Qwen3-8B-Chemistry-RLSD-TR/artifacts/queue.log

[2026-07-02 23:56:17] Waiting for PID 999999999.
[2026-07-02 23:56:17] PID 999999999 finished. Starting Qwen3 generalization queue.
[2026-07-02 23:56:17] Model: Qwen/Qwen3-8B
[2026-07-02 23:56:17] Datasets: chemistry physics biology material tooluse
[2026-07-02 23:56:17] Methods: grpo rlsd_tr
[2026-07-02 23:56:17] Starting Qwen/Qwen3-8B chemistry grpo.
Experiment: qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8
Dataset:    chemistry
Method:     grpo
Config:     baseline_grpo
Model:      Qwen/Qwen3-8B
Train:      /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet
Val:        /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet
Ray tmp:    /tmp/ray_q3g_chemistry_grpo_Qwen3_8B
2026-07-02 23:56:19,612	INFO scripts.py:1364 -- Did not find any active Ray processes.
Experiment: qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8
Config: baseline_grpo
Task: datasets/sciknoweval/chemistry
Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-GRPO-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_chemistry_grpo_Qwen3_8B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/chemistry +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-8B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 trainer.max_actor_ckpt_to_keep=100 actor_rollout_ref.actor.policy_loss.loss_mode=vanilla actor_rollout_ref.actor.kl_loss_coef=0.0
ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_chemistry_grpo_Qwen3_8B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/chemistry', 'EXPERIMENT': 'qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}}
2026-07-02 23:56:29,407	INFO worker.py:2007 -- Started a local Ray instance.
/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
  warnings.warn(
(TaskRunner pid=2224264) TaskRunner hostname: mole-workspace-rw, PID: 2224264
(TaskRunner pid=2224264) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig',
(TaskRunner pid=2224264)                                  'calculate_entropy': False,
(TaskRunner pid=2224264)                                  'calculate_sum_pi_squared': False,
(TaskRunner pid=2224264)                                  'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig',
(TaskRunner pid=2224264)                                                 'async_save': False,
(TaskRunner pid=2224264)                                                 'load_contents': ['model',
(TaskRunner pid=2224264)                                                                   'optimizer',
(TaskRunner pid=2224264)                                                                   'extra'],
(TaskRunner pid=2224264)                                                 'save_contents': ['model',
(TaskRunner pid=2224264)                                                                   'optimizer',
(TaskRunner pid=2224264)                                                                   'extra']},
(TaskRunner pid=2224264)                                  'clip_ratio': 0.2,
(TaskRunner pid=2224264)                                  'clip_ratio_c': 3.0,
(TaskRunner pid=2224264)                                  'clip_ratio_high': 0.28,
(TaskRunner pid=2224264)                                  'clip_ratio_low': 0.2,
(TaskRunner pid=2224264)                                  'data_loader_seed': 42,
(TaskRunner pid=2224264)                                  'entropy_checkpointing': False,
(TaskRunner pid=2224264)                                  'entropy_coeff': 0,
(TaskRunner pid=2224264)                                  'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2224264)                                  'freeze_vision_tower': False,
(TaskRunner pid=2224264)                                  'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2224264)                                                  'dtype': 'bfloat16',
(TaskRunner pid=2224264)                                                  'entropy_checkpointing': False,
(TaskRunner pid=2224264)                                                  'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2224264)                                                  'forward_only': False,
(TaskRunner pid=2224264)                                                  'forward_prefetch': False,
(TaskRunner pid=2224264)                                                  'fsdp_size': -1,
(TaskRunner pid=2224264)                                                  'full_determinism': False,
(TaskRunner pid=2224264)                                                  'model_dtype': 'fp32',
(TaskRunner pid=2224264)                                                  'offload_policy': False,
(TaskRunner pid=2224264)                                                  'optimizer_offload': False,
(TaskRunner pid=2224264)                                                  'param_offload': False,
(TaskRunner pid=2224264)                                                  'reshard_after_forward': True,
(TaskRunner pid=2224264)                                                  'seed': 42,
(TaskRunner pid=2224264)                                                  'strategy': 'fsdp',
(TaskRunner pid=2224264)                                                  'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2224264)                                                  'use_orig_params': False,
(TaskRunner pid=2224264)                                                  'use_torch_compile': True,
(TaskRunner pid=2224264)                                                  'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2224264)                                  'grad_clip': 1.0,
(TaskRunner pid=2224264)                                  'kl_loss_coef': 0.0,
(TaskRunner pid=2224264)                                  'kl_loss_type': 'low_var_kl',
(TaskRunner pid=2224264)                                  'loss_agg_mode': 'token-mean',
(TaskRunner pid=2224264)                                  'loss_scale_factor': None,
(TaskRunner pid=2224264)                                  'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig',
(TaskRunner pid=2224264)                                            'betas': [0.9, 0.999],
(TaskRunner pid=2224264)                                            'clip_grad': 1.0,
(TaskRunner pid=2224264)                                            'lr': 1e-06,
(TaskRunner pid=2224264)                                            'lr_scheduler_type': 'constant',
(TaskRunner pid=2224264)                                            'lr_warmup_steps': 10,
(TaskRunner pid=2224264)                                            'lr_warmup_steps_ratio': 0.0,
(TaskRunner pid=2224264)                                            'min_lr_ratio': 0.0,
(TaskRunner pid=2224264)                                            'num_cycles': 0.5,
(TaskRunner pid=2224264)                                            'optimizer': 'AdamW',
(TaskRunner pid=2224264)                                            'optimizer_impl': 'torch.optim',
(TaskRunner pid=2224264)                                            'override_optimizer_config': None,
(TaskRunner pid=2224264)                                            'total_training_steps': -1,
(TaskRunner pid=2224264)                                            'warmup_style': None,
(TaskRunner pid=2224264)                                            'weight_decay': 0.01},
(TaskRunner pid=2224264)                                  'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig',
(TaskRunner pid=2224264)                                                  'clip_cov_lb': 1.0,
(TaskRunner pid=2224264)                                                  'clip_cov_ratio': 0.0002,
(TaskRunner pid=2224264)                                                  'clip_cov_ub': 5.0,
(TaskRunner pid=2224264)                                                  'kl_cov_ratio': 0.0002,
(TaskRunner pid=2224264)                                                  'loss_mode': 'vanilla',
(TaskRunner pid=2224264)                                                  'ppo_kl_coef': 0.1},
(TaskRunner pid=2224264)                                  'ppo_epochs': 1,
(TaskRunner pid=2224264)                                  'ppo_max_token_len_per_gpu': 10240,
(TaskRunner pid=2224264)                                  'ppo_micro_batch_size': None,
(TaskRunner pid=2224264)                                  'ppo_micro_batch_size_per_gpu': 1,
(TaskRunner pid=2224264)                                  'ppo_mini_batch_size': 8,
(TaskRunner pid=2224264)                                  'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2224264)                                               'all_ranks': False,
(TaskRunner pid=2224264)                                               'enable': False,
(TaskRunner pid=2224264)                                               'ranks': [],
(TaskRunner pid=2224264)                                               'save_path': 'outputs/profile',
(TaskRunner pid=2224264)                                               'tool': None,
(TaskRunner pid=2224264)                                               'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2224264)                                                                       'analysis': True,
(TaskRunner pid=2224264)                                                                       'contents': [],
(TaskRunner pid=2224264)                                                                       'discrete': False,
(TaskRunner pid=2224264)                                                                       'level': 'level0'},
(TaskRunner pid=2224264)                                                               'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2224264)                                                                        'discrete': False},
(TaskRunner pid=2224264)                                                               'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2224264)                                                                         'step_end': None,
(TaskRunner pid=2224264)                                                                         'step_start': 0},
(TaskRunner pid=2224264)                                                               'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2224264)                                                                                'stack_depth': 32,
(TaskRunner pid=2224264)                                                                                'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2224264)                                  'rollout_n': 8,
(TaskRunner pid=2224264)                                  'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig',
(TaskRunner pid=2224264)                                                    'mode': 'disabled',
(TaskRunner pid=2224264)                                                    'record_file': None,
(TaskRunner pid=2224264)                                                    'replay_file': None},
(TaskRunner pid=2224264)                                  'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig',
(TaskRunner pid=2224264)                                                        'alpha': 0.5,
(TaskRunner pid=2224264)                                                        'distillation_add_tail': True,
(TaskRunner pid=2224264)                                                        'distillation_topk': 100,
(TaskRunner pid=2224264)                                                        'dont_reprompt_on_self_success': True,
(TaskRunner pid=2224264)                                                        'environment_feedback_only_without_solution': True,
(TaskRunner pid=2224264)                                                        'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig',
(TaskRunner pid=2224264)                                                                             'enable': False,
(TaskRunner pid=2224264)                                                                             'loss_weight': 0.0,
(TaskRunner pid=2224264)                                                                             'mask': 'all'},
(TaskRunner pid=2224264)                                                        'feedback_template': '\n'
(TaskRunner pid=2224264)                                                                             'The '
(TaskRunner pid=2224264)                                                                             'following '
(TaskRunner pid=2224264)                                                                             'is '
(TaskRunner pid=2224264)                                                                             'feedback '
(TaskRunner pid=2224264)                                                                             'from '
(TaskRunner pid=2224264)                                                                             'your '
(TaskRunner pid=2224264)                                                                             'unsuccessful '
(TaskRunner pid=2224264)                                                                             'earlier '
(TaskRunner pid=2224264)                                                                             'attempt:\n'
(TaskRunner pid=2224264)                                                                             '\n'
(TaskRunner pid=2224264)                                                                             '{feedback_raw}',
(TaskRunner pid=2224264)                                                        'full_logit_distillation': True,
(TaskRunner pid=2224264)                                                        'include_environment_feedback': True,
(TaskRunner pid=2224264)                                                        'is_clip': 2,
(TaskRunner pid=2224264)                                                        'max_reprompt_len': 22528,
(TaskRunner pid=2224264)                                                        'remove_thinking_from_demonstration': False,
(TaskRunner pid=2224264)                                                        'reprompt_template': '{prompt}{solution}{feedback}\n'
(TaskRunner pid=2224264)                                                                             '\n'
(TaskRunner pid=2224264)                                                                             'Correctly '
(TaskRunner pid=2224264)                                                                             'solve '
(TaskRunner pid=2224264)                                                                             'the '
(TaskRunner pid=2224264)                                                                             'original '
(TaskRunner pid=2224264)                                                                             'question.',
(TaskRunner pid=2224264)                                                        'reprompt_truncation': 'right',
(TaskRunner pid=2224264)                                                        'solution_template': '\n'
(TaskRunner pid=2224264)                                                                             'Correct '
(TaskRunner pid=2224264)                                                                             'solution:\n'
(TaskRunner pid=2224264)                                                                             '\n'
(TaskRunner pid=2224264)                                                                             '{successful_previous_attempt}',
(TaskRunner pid=2224264)                                                        'srpo_dynamic_weighting': False,
(TaskRunner pid=2224264)                                                        'srpo_dynamic_weighting_temperature': 1.0,
(TaskRunner pid=2224264)                                                        'success_reward_threshold': 0.5,
(TaskRunner pid=2224264)                                                        'teacher_regularization': 'ema',
(TaskRunner pid=2224264)                                                        'teacher_update_rate': 0.0,
(TaskRunner pid=2224264)                                                        'token_reweight_decay_steps': None,
(TaskRunner pid=2224264)                                                        'token_reweight_eps_w': 0.2,
(TaskRunner pid=2224264)                                                        'token_reweight_lambda': 0.5},
(TaskRunner pid=2224264)                                  'shuffle': False,
(TaskRunner pid=2224264)                                  'strategy': 'fsdp',
(TaskRunner pid=2224264)                                  'sum_pi_squared_checkpointing': False,
(TaskRunner pid=2224264)                                  'tau_neg': 1.05,
(TaskRunner pid=2224264)                                  'tau_pos': 1.0,
(TaskRunner pid=2224264)                                  'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2224264)                                  'use_dynamic_bsz': False,
(TaskRunner pid=2224264)                                  'use_fused_kernels': False,
(TaskRunner pid=2224264)                                  'use_kl_loss': False,
(TaskRunner pid=2224264)                                  'use_prefix_grouper': False,
(TaskRunner pid=2224264)                                  'use_remove_padding': True,
(TaskRunner pid=2224264)                                  'use_torch_compile': True},
(TaskRunner pid=2224264)                        'hybrid_engine': True,
(TaskRunner pid=2224264)                        'model': {'_target_': 'verl.workers.config.HFModelConfig',
(TaskRunner pid=2224264)                                  'custom_chat_template': None,
(TaskRunner pid=2224264)                                  'enable_activation_offload': False,
(TaskRunner pid=2224264)                                  'enable_gradient_checkpointing': True,
(TaskRunner pid=2224264)                                  'exclude_modules': None,
(TaskRunner pid=2224264)                                  'external_lib': None,
(TaskRunner pid=2224264)                                  'fused_kernel_options': {'impl_backend': 'torch'},
(TaskRunner pid=2224264)                                  'hf_config_path': None,
(TaskRunner pid=2224264)                                  'lora_adapter_path': None,
(TaskRunner pid=2224264)                                  'lora_alpha': 16,
(TaskRunner pid=2224264)                                  'lora_rank': 0,
(TaskRunner pid=2224264)                                  'override_config': {},
(TaskRunner pid=2224264)                                  'path': 'Qwen/Qwen3-8B',
(TaskRunner pid=2224264)                                  'target_modules': 'all-linear',
(TaskRunner pid=2224264)                                  'tiled_mlp': {'enabled': False,
(TaskRunner pid=2224264)                                                'num_shards': 4},
(TaskRunner pid=2224264)                                  'tokenizer_path': None,
(TaskRunner pid=2224264)                                  'trust_remote_code': True,
(TaskRunner pid=2224264)                                  'use_fused_kernels': False,
(TaskRunner pid=2224264)                                  'use_liger': False,
(TaskRunner pid=2224264)                                  'use_remove_padding': True,
(TaskRunner pid=2224264)                                  'use_shm': False},
(TaskRunner pid=2224264)                        'nccl_timeout': 600,
(TaskRunner pid=2224264)                        'ref': {'_target_': 'verl.workers.config.FSDPActorConfig',
(TaskRunner pid=2224264)                                'entropy_checkpointing': False,
(TaskRunner pid=2224264)                                'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2224264)                                'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2224264)                                                'dtype': 'bfloat16',
(TaskRunner pid=2224264)                                                'entropy_checkpointing': False,
(TaskRunner pid=2224264)                                                'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2224264)                                                'forward_only': True,
(TaskRunner pid=2224264)                                                'forward_prefetch': False,
(TaskRunner pid=2224264)                                                'fsdp_size': -1,
(TaskRunner pid=2224264)                                                'full_determinism': False,
(TaskRunner pid=2224264)                                                'model_dtype': 'fp32',
(TaskRunner pid=2224264)                                                'offload_policy': False,
(TaskRunner pid=2224264)                                                'optimizer_offload': False,
(TaskRunner pid=2224264)                                                'param_offload': False,
(TaskRunner pid=2224264)                                                'reshard_after_forward': True,
(TaskRunner pid=2224264)                                                'seed': 42,
(TaskRunner pid=2224264)                                                'strategy': 'fsdp',
(TaskRunner pid=2224264)                                                'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2224264)                                                'use_orig_params': False,
(TaskRunner pid=2224264)                                                'use_torch_compile': True,
(TaskRunner pid=2224264)                                                'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2224264)                                'log_prob_max_token_len_per_gpu': 10240,
(TaskRunner pid=2224264)                                'log_prob_micro_batch_size': None,
(TaskRunner pid=2224264)                                'log_prob_micro_batch_size_per_gpu': 1,
(TaskRunner pid=2224264)                                'log_prob_use_dynamic_bsz': False,
(TaskRunner pid=2224264)                                'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2224264)                                             'all_ranks': False,
(TaskRunner pid=2224264)                                             'enable': False,
(TaskRunner pid=2224264)                                             'ranks': [],
(TaskRunner pid=2224264)                                             'save_path': 'outputs/profile',
(TaskRunner pid=2224264)                                             'tool': None,
(TaskRunner pid=2224264)                                             'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2224264)                                                                     'analysis': True,
(TaskRunner pid=2224264)                                                                     'contents': [],
(TaskRunner pid=2224264)                                                                     'discrete': False,
(TaskRunner pid=2224264)                                                                     'level': 'level0'},
(TaskRunner pid=2224264)                                                             'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2224264)                                                                      'discrete': False},
(TaskRunner pid=2224264)                                                             'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2224264)                                                                       'step_end': None,
(TaskRunner pid=2224264)                                                                       'step_start': 0},
(TaskRunner pid=2224264)                                                             'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2224264)                                                                              'stack_depth': 32,
(TaskRunner pid=2224264)                                                                              'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2224264)                                'rollout_n': 8,
(TaskRunner pid=2224264)                                'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig',
(TaskRunner pid=2224264)                                                  'mode': 'disabled',
(TaskRunner pid=2224264)                                                  'record_file': None,
(TaskRunner pid=2224264)                                                  'replay_file': None},
(TaskRunner pid=2224264)                                'strategy': 'fsdp',
(TaskRunner pid=2224264)                                'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2224264)                                'use_torch_compile': True},
(TaskRunner pid=2224264)                        'rollout': {'_target_': 'verl.workers.config.RolloutConfig',
(TaskRunner pid=2224264)                                    'agent': {'_target_': 'verl.workers.config.AgentLoopConfig',
(TaskRunner pid=2224264)                                              'agent_loop_config_path': None,
(TaskRunner pid=2224264)                                              'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig',
(TaskRunner pid=2224264)                                                                      'name': None,
(TaskRunner pid=2224264)                                                                      'path': None},
(TaskRunner pid=2224264)                                              'default_agent_loop': 'single_turn_agent',
(TaskRunner pid=2224264)                                              'num_workers': 8},
(TaskRunner pid=2224264)                                    'calculate_log_probs': True,
(TaskRunner pid=2224264)                                    'cudagraph_capture_sizes': None,
(TaskRunner pid=2224264)                                    'data_parallel_size': 1,
(TaskRunner pid=2224264)                                    'disable_log_stats': True,
(TaskRunner pid=2224264)                                    'do_sample': True,
(TaskRunner pid=2224264)                                    'dtype': 'bfloat16',
(TaskRunner pid=2224264)                                    'enable_chunked_prefill': True,
(TaskRunner pid=2224264)                                    'enable_prefix_caching': True,
(TaskRunner pid=2224264)                                    'enable_rollout_routing_replay': False,
(TaskRunner pid=2224264)                                    'enforce_eager': False,
(TaskRunner pid=2224264)                                    'engine_kwargs': {'sglang': {}, 'vllm': {}},
(TaskRunner pid=2224264)                                    'expert_parallel_size': 1,
(TaskRunner pid=2224264)                                    'free_cache_engine': True,
(TaskRunner pid=2224264)                                    'gpu_memory_utilization': 0.8,
(TaskRunner pid=2224264)                                    'ignore_eos': False,
(TaskRunner pid=2224264)                                    'layered_summon': False,
(TaskRunner pid=2224264)                                    'load_format': 'dummy',
(TaskRunner pid=2224264)                                    'log_prob_max_token_len_per_gpu': 10240,
(TaskRunner pid=2224264)                                    'log_prob_micro_batch_size': None,
(TaskRunner pid=2224264)                                    'log_prob_micro_batch_size_per_gpu': 1,
(TaskRunner pid=2224264)                                    'log_prob_use_dynamic_bsz': False,
(TaskRunner pid=2224264)                                    'logprobs_mode': 'processed_logprobs',
(TaskRunner pid=2224264)                                    'max_model_len': 10240,
(TaskRunner pid=2224264)                                    'max_num_batched_tokens': 10240,
(TaskRunner pid=2224264)                                    'max_num_seqs': 1024,
(TaskRunner pid=2224264)                                    'mode': 'async',
(TaskRunner pid=2224264)                                    'multi_stage_wake_up': False,
(TaskRunner pid=2224264)                                    'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig',
(TaskRunner pid=2224264)                                                   'enable': False,
(TaskRunner pid=2224264)                                                   'format': 'hermes',
(TaskRunner pid=2224264)                                                   'interaction_config_path': None,
(TaskRunner pid=2224264)                                                   'max_assistant_turns': None,
(TaskRunner pid=2224264)                                                   'max_parallel_calls': 1,
(TaskRunner pid=2224264)                                                   'max_tool_response_length': 256,
(TaskRunner pid=2224264)                                                   'max_user_turns': None,
(TaskRunner pid=2224264)                                                   'num_repeat_rollouts': None,
(TaskRunner pid=2224264)                                                   'tokenization_sanity_check_mode': 'strict',
(TaskRunner pid=2224264)                                                   'tool_config_path': None,
(TaskRunner pid=2224264)                                                   'tool_response_truncate_side': 'middle',
(TaskRunner pid=2224264)                                                   'use_inference_chat_template': False},
(TaskRunner pid=2224264)                                    'n': 8,
(TaskRunner pid=2224264)                                    'name': 'vllm',
(TaskRunner pid=2224264)                                    'over_sample_rate': 0,
(TaskRunner pid=2224264)                                    'pipeline_model_parallel_size': 1,
(TaskRunner pid=2224264)                                    'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2224264)                                                 'all_ranks': False,
(TaskRunner pid=2224264)                                                 'enable': False,
(TaskRunner pid=2224264)                                                 'ranks': [],
(TaskRunner pid=2224264)                                                 'save_path': 'outputs/profile',
(TaskRunner pid=2224264)                                                 'tool': None,
(TaskRunner pid=2224264)                                                 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2224264)                                                                         'analysis': True,
(TaskRunner pid=2224264)                                                                         'contents': [],
(TaskRunner pid=2224264)                                                                         'discrete': False,
(TaskRunner pid=2224264)                                                                         'level': 'level0'},
(TaskRunner pid=2224264)                                                                 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2224264)                                                                          'discrete': False},
(TaskRunner pid=2224264)                                                                 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2224264)                                                                           'step_end': None,
(TaskRunner pid=2224264)                                                                           'step_start': 0},
(TaskRunner pid=2224264)                                                                 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2224264)                                                                                  'stack_depth': 32,
(TaskRunner pid=2224264)                                                                                  'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2224264)                                    'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig',
(TaskRunner pid=2224264)                                                   'enable': False,
(TaskRunner pid=2224264)                                                   'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml',
(TaskRunner pid=2224264)                                                   'port': 9090,
(TaskRunner pid=2224264)                                                   'served_model_name': 'Qwen/Qwen3-8B'},
(TaskRunner pid=2224264)                                    'prompt_length': 2048,
(TaskRunner pid=2224264)                                    'quantization': None,
(TaskRunner pid=2224264)                                    'quantization_config_file': None,
(TaskRunner pid=2224264)                                    'response_length': 8192,
(TaskRunner pid=2224264)                                    'scheduling_policy': 'fcfs',
(TaskRunner pid=2224264)                                    'skip_dump_dir': '/tmp/rollout_dump',
(TaskRunner pid=2224264)                                    'skip_rollout': False,
(TaskRunner pid=2224264)                                    'skip_tokenizer_init': True,
(TaskRunner pid=2224264)                                    'temperature': 1.0,
(TaskRunner pid=2224264)                                    'tensor_model_parallel_size': 2,
(TaskRunner pid=2224264)                                    'top_k': -1,
(TaskRunner pid=2224264)                                    'top_p': 1.0,
(TaskRunner pid=2224264)                                    'trace': {'_target_': 'verl.workers.config.TraceConfig',
(TaskRunner pid=2224264)                                              'backend': None,
(TaskRunner pid=2224264)                                              'max_samples_per_step_per_worker': None,
(TaskRunner pid=2224264)                                              'token2text': False},
(TaskRunner pid=2224264)                                    'update_weights_bucket_megabytes': 512,
(TaskRunner pid=2224264)                                    'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig',
(TaskRunner pid=2224264)                                                   'do_sample': True,
(TaskRunner pid=2224264)                                                   'n': 16,
(TaskRunner pid=2224264)                                                   'temperature': 0.6,
(TaskRunner pid=2224264)                                                   'top_k': -1,
(TaskRunner pid=2224264)                                                   'top_p': 0.95}}},
(TaskRunner pid=2224264)  'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig',
(TaskRunner pid=2224264)                'adv_estimator': 'grpo',
(TaskRunner pid=2224264)                'gamma': 1.0,
(TaskRunner pid=2224264)                'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig',
(TaskRunner pid=2224264)                            'horizon': 10000,
(TaskRunner pid=2224264)                            'kl_coef': 0.001,
(TaskRunner pid=2224264)                            'target_kl': 0.1,
(TaskRunner pid=2224264)                            'type': 'fixed'},
(TaskRunner pid=2224264)                'kl_penalty': 'kl',
(TaskRunner pid=2224264)                'lam': 1.0,
(TaskRunner pid=2224264)                'norm_adv_by_std_in_grpo': False,
(TaskRunner pid=2224264)                'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0},
(TaskRunner pid=2224264)                'rollout_correction': {'bypass_mode': False,
(TaskRunner pid=2224264)                                       'loss_type': 'ppo_clip',
(TaskRunner pid=2224264)                                       'rollout_is': 'token',
(TaskRunner pid=2224264)                                       'rollout_is_batch_normalize': False,
(TaskRunner pid=2224264)                                       'rollout_is_threshold': 2.0,
(TaskRunner pid=2224264)                                       'rollout_rs': None,
(TaskRunner pid=2224264)                                       'rollout_rs_threshold': None},
(TaskRunner pid=2224264)                'use_kl_in_reward': False,
(TaskRunner pid=2224264)                'use_pf_ppo': False},
(TaskRunner pid=2224264)  'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig',
(TaskRunner pid=2224264)             'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig',
(TaskRunner pid=2224264)                            'async_save': False,
(TaskRunner pid=2224264)                            'load_contents': ['model', 'optimizer', 'extra'],
(TaskRunner pid=2224264)                            'save_contents': ['model', 'optimizer', 'extra']},
(TaskRunner pid=2224264)             'cliprange_value': 0.5,
(TaskRunner pid=2224264)             'data_loader_seed': 42,
(TaskRunner pid=2224264)             'enable': None,
(TaskRunner pid=2224264)             'forward_max_token_len_per_gpu': 32768,
(TaskRunner pid=2224264)             'forward_micro_batch_size': None,
(TaskRunner pid=2224264)             'forward_micro_batch_size_per_gpu': None,
(TaskRunner pid=2224264)             'grad_clip': 1.0,
(TaskRunner pid=2224264)             'loss_agg_mode': 'token-mean',
(TaskRunner pid=2224264)             'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg',
(TaskRunner pid=2224264)                       'enable_activation_offload': False,
(TaskRunner pid=2224264)                       'enable_gradient_checkpointing': True,
(TaskRunner pid=2224264)                       'external_lib': None,
(TaskRunner pid=2224264)                       'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2224264)                                       'dtype': 'bfloat16',
(TaskRunner pid=2224264)                                       'entropy_checkpointing': False,
(TaskRunner pid=2224264)                                       'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2224264)                                       'forward_only': False,
(TaskRunner pid=2224264)                                       'forward_prefetch': False,
(TaskRunner pid=2224264)                                       'fsdp_size': -1,
(TaskRunner pid=2224264)                                       'full_determinism': False,
(TaskRunner pid=2224264)                                       'model_dtype': 'fp32',
(TaskRunner pid=2224264)                                       'offload_policy': False,
(TaskRunner pid=2224264)                                       'optimizer_offload': False,
(TaskRunner pid=2224264)                                       'param_offload': False,
(TaskRunner pid=2224264)                                       'reshard_after_forward': True,
(TaskRunner pid=2224264)                                       'seed': 42,
(TaskRunner pid=2224264)                                       'strategy': 'fsdp',
(TaskRunner pid=2224264)                                       'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2224264)                                       'use_orig_params': False,
(TaskRunner pid=2224264)                                       'use_torch_compile': True,
(TaskRunner pid=2224264)                                       'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2224264)                       'lora_alpha': 16,
(TaskRunner pid=2224264)                       'lora_rank': 0,
(TaskRunner pid=2224264)                       'override_config': {},
(TaskRunner pid=2224264)                       'path': 'Qwen/Qwen3-8B',
(TaskRunner pid=2224264)                       'target_modules': 'all-linear',
(TaskRunner pid=2224264)                       'tiled_mlp': {'enabled': False, 'num_shards': 4},
(TaskRunner pid=2224264)                       'tokenizer_path': 'Qwen/Qwen3-8B',
(TaskRunner pid=2224264)                       'trust_remote_code': True,
(TaskRunner pid=2224264)                       'use_remove_padding': False,
(TaskRunner pid=2224264)                       'use_shm': False},
(TaskRunner pid=2224264)             'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig',
(TaskRunner pid=2224264)                       'betas': [0.9, 0.999],
(TaskRunner pid=2224264)                       'clip_grad': 1.0,
(TaskRunner pid=2224264)                       'lr': 1e-05,
(TaskRunner pid=2224264)                       'lr_scheduler_type': 'constant',
(TaskRunner pid=2224264)                       'lr_warmup_steps': -1,
(TaskRunner pid=2224264)                       'lr_warmup_steps_ratio': 0.0,
(TaskRunner pid=2224264)                       'min_lr_ratio': 0.0,
(TaskRunner pid=2224264)                       'num_cycles': 0.5,
(TaskRunner pid=2224264)                       'optimizer': 'AdamW',
(TaskRunner pid=2224264)                       'optimizer_impl': 'torch.optim',
(TaskRunner pid=2224264)                       'override_optimizer_config': None,
(TaskRunner pid=2224264)                       'total_training_steps': -1,
(TaskRunner pid=2224264)                       'warmup_style': None,
(TaskRunner pid=2224264)                       'weight_decay': 0.01},
(TaskRunner pid=2224264)             'ppo_epochs': 1,
(TaskRunner pid=2224264)             'ppo_max_token_len_per_gpu': 32768,
(TaskRunner pid=2224264)             'ppo_micro_batch_size': None,
(TaskRunner pid=2224264)             'ppo_micro_batch_size_per_gpu': None,
(TaskRunner pid=2224264)             'ppo_mini_batch_size': 8,
(TaskRunner pid=2224264)             'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2224264)                          'all_ranks': False,
(TaskRunner pid=2224264)                          'enable': False,
(TaskRunner pid=2224264)                          'ranks': [],
(TaskRunner pid=2224264)                          'save_path': 'outputs/profile',
(TaskRunner pid=2224264)                          'tool': None,
(TaskRunner pid=2224264)                          'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2224264)                                                  'analysis': True,
(TaskRunner pid=2224264)                                                  'contents': [],
(TaskRunner pid=2224264)                                                  'discrete': False,
(TaskRunner pid=2224264)                                                  'level': 'level0'},
(TaskRunner pid=2224264)                                          'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2224264)                                                   'discrete': False},
(TaskRunner pid=2224264)                                          'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2224264)                                                    'step_end': None,
(TaskRunner pid=2224264)                                                    'step_start': 0},
(TaskRunner pid=2224264)                                          'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2224264)                                                           'stack_depth': 32,
(TaskRunner pid=2224264)                                                           'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2224264)             'rollout_n': 8,
(TaskRunner pid=2224264)             'shuffle': False,
(TaskRunner pid=2224264)             'strategy': 'fsdp',
(TaskRunner pid=2224264)             'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2224264)             'use_dynamic_bsz': False},
(TaskRunner pid=2224264)  'custom_reward_function': {'name': 'compute_score',
(TaskRunner pid=2224264)                             'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'},
(TaskRunner pid=2224264)  'data': {'apply_chat_template_kwargs': {'enable_thinking': False},
(TaskRunner pid=2224264)           'custom_cls': {'name': None, 'path': None},
(TaskRunner pid=2224264)           'datagen': {'name': None, 'path': None},
(TaskRunner pid=2224264)           'dataloader_num_workers': 8,
(TaskRunner pid=2224264)           'filter_overlong_prompts': True,
(TaskRunner pid=2224264)           'filter_overlong_prompts_workers': 1,
(TaskRunner pid=2224264)           'image_key': 'images',
(TaskRunner pid=2224264)           'image_patch_size': 14,
(TaskRunner pid=2224264)           'max_prompt_length': 2048,
(TaskRunner pid=2224264)           'max_response_length': 8192,
(TaskRunner pid=2224264)           'prompt_key': 'prompt',
(TaskRunner pid=2224264)           'return_full_prompt': False,
(TaskRunner pid=2224264)           'return_multi_modal_inputs': True,
(TaskRunner pid=2224264)           'return_raw_chat': True,
(TaskRunner pid=2224264)           'return_raw_input_ids': False,
(TaskRunner pid=2224264)           'reward_fn_key': 'data_source',
(TaskRunner pid=2224264)           'sampler': {'class_name': None, 'class_path': None},
(TaskRunner pid=2224264)           'seed': None,
(TaskRunner pid=2224264)           'shuffle': True,
(TaskRunner pid=2224264)           'tokenizer': None,
(TaskRunner pid=2224264)           'tool_config_path': None,
(TaskRunner pid=2224264)           'train_batch_size': 32,
(TaskRunner pid=2224264)           'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet'],
(TaskRunner pid=2224264)           'train_max_samples': 3200,
(TaskRunner pid=2224264)           'truncation': 'error',
(TaskRunner pid=2224264)           'trust_remote_code': True,
(TaskRunner pid=2224264)           'use_shm': False,
(TaskRunner pid=2224264)           'val_batch_size': None,
(TaskRunner pid=2224264)           'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet'],
(TaskRunner pid=2224264)           'val_max_samples': -1,
(TaskRunner pid=2224264)           'validation_shuffle': False,
(TaskRunner pid=2224264)           'video_key': 'videos'},
(TaskRunner pid=2224264)  'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2224264)                      'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2224264)                                                      'controller_nsight_options': {'cuda-graph-trace': 'graph',
(TaskRunner pid=2224264)                                                                                    'cuda-memory-usage': 'true',
(TaskRunner pid=2224264)                                                                                    'trace': 'cuda,nvtx,cublas,ucx'},
(TaskRunner pid=2224264)                                                      'discrete': False,
(TaskRunner pid=2224264)                                                      'worker_nsight_options': {'capture-range': 'cudaProfilerApi',
(TaskRunner pid=2224264)                                                                                'capture-range-end': None,
(TaskRunner pid=2224264)                                                                                'cuda-graph-trace': 'graph',
(TaskRunner pid=2224264)                                                                                'cuda-memory-usage': 'true',
(TaskRunner pid=2224264)                                                                                'kill': 'none',
(TaskRunner pid=2224264)                                                                                'trace': 'cuda,nvtx,cublas,ucx'}},
(TaskRunner pid=2224264)                                             'torch_memory': {'context': 'all',
(TaskRunner pid=2224264)                                                              'kw_args': {},
(TaskRunner pid=2224264)                                                              'stack_depth': 32,
(TaskRunner pid=2224264)                                                              'stacks': 'all',
(TaskRunner pid=2224264)                                                              'trace_alloc_max_entries': 100000}},
(TaskRunner pid=2224264)                      'profile_continuous_steps': False,
(TaskRunner pid=2224264)                      'save_path': 'outputs/profile',
(TaskRunner pid=2224264)                      'steps': None,
(TaskRunner pid=2224264)                      'tool': None},
(TaskRunner pid=2224264)  'max_model_len': 10240,
(TaskRunner pid=2224264)  'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_chemistry_grpo_Qwen3_8B',
(TaskRunner pid=2224264)                              'include_dashboard': False,
(TaskRunner pid=2224264)                              'num_cpus': None,
(TaskRunner pid=2224264)                              'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8',
(TaskRunner pid=2224264)                                                           'TASK': 'datasets/sciknoweval/chemistry',
(TaskRunner pid=2224264)                                                           'USER': 'root'}}},
(TaskRunner pid=2224264)                 'timeline_json_file': None},
(TaskRunner pid=2224264)  'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig',
(TaskRunner pid=2224264)                     'module': {'_target_': 'verl.trainer.config.config.ModuleConfig',
(TaskRunner pid=2224264)                                'name': 'custom_reward_manager',
(TaskRunner pid=2224264)                                'path': None},
(TaskRunner pid=2224264)                     'name': 'naive',
(TaskRunner pid=2224264)                     'source': 'register'},
(TaskRunner pid=2224264)  'reward_model': {'enable': False,
(TaskRunner pid=2224264)                   'enable_resource_pool': False,
(TaskRunner pid=2224264)                   'forward_max_token_len_per_gpu': 32768,
(TaskRunner pid=2224264)                   'launch_reward_fn_async': False,
(TaskRunner pid=2224264)                   'max_length': None,
(TaskRunner pid=2224264)                   'micro_batch_size': None,
(TaskRunner pid=2224264)                   'micro_batch_size_per_gpu': None,
(TaskRunner pid=2224264)                   'model': {'external_lib': None,
(TaskRunner pid=2224264)                             'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2224264)                                             'forward_prefetch': False,
(TaskRunner pid=2224264)                                             'fsdp_size': -1,
(TaskRunner pid=2224264)                                             'param_offload': False,
(TaskRunner pid=2224264)                                             'reshard_after_forward': True,
(TaskRunner pid=2224264)                                             'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2224264)                             'input_tokenizer': 'Qwen/Qwen3-8B',
(TaskRunner pid=2224264)                             'override_config': {},
(TaskRunner pid=2224264)                             'path': '~/models/FsfairX-LLaMA3-RM-v0.1',
(TaskRunner pid=2224264)                             'trust_remote_code': False,
(TaskRunner pid=2224264)                             'use_fused_kernels': False,
(TaskRunner pid=2224264)                             'use_remove_padding': False,
(TaskRunner pid=2224264)                             'use_shm': False},
(TaskRunner pid=2224264)                   'n_gpus_per_node': 8,
(TaskRunner pid=2224264)                   'nnodes': 0,
(TaskRunner pid=2224264)                   'num_workers': 1,
(TaskRunner pid=2224264)                   'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2224264)                                'all_ranks': False,
(TaskRunner pid=2224264)                                'enable': False,
(TaskRunner pid=2224264)                                'ranks': [],
(TaskRunner pid=2224264)                                'save_path': 'outputs/profile',
(TaskRunner pid=2224264)                                'tool': None,
(TaskRunner pid=2224264)                                'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2224264)                                                        'analysis': True,
(TaskRunner pid=2224264)                                                        'contents': [],
(TaskRunner pid=2224264)                                                        'discrete': False,
(TaskRunner pid=2224264)                                                        'level': 'level0'},
(TaskRunner pid=2224264)                                                'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2224264)                                                         'discrete': False},
(TaskRunner pid=2224264)                                                'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2224264)                                                          'step_end': None,
(TaskRunner pid=2224264)                                                          'step_start': 0},
(TaskRunner pid=2224264)                                                'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2224264)                                                                 'stack_depth': 32,
(TaskRunner pid=2224264)                                                                 'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2224264)                   'reward_loop_class_name': None,
(TaskRunner pid=2224264)                   'reward_loop_module_path': None,
(TaskRunner pid=2224264)                   'reward_loop_source': 'register',
(TaskRunner pid=2224264)                   'reward_manager': 'naive',
(TaskRunner pid=2224264)                   'rollout': {'_target_': 'verl.workers.config.RolloutConfig',
(TaskRunner pid=2224264)                               'cudagraph_capture_sizes': None,
(TaskRunner pid=2224264)                               'data_parallel_size': 1,
(TaskRunner pid=2224264)                               'disable_log_stats': True,
(TaskRunner pid=2224264)                               'dtype': 'bfloat16',
(TaskRunner pid=2224264)                               'enable_chunked_prefill': True,
(TaskRunner pid=2224264)                               'enable_prefix_caching': True,
(TaskRunner pid=2224264)                               'enforce_eager': True,
(TaskRunner pid=2224264)                               'engine_kwargs': {},
(TaskRunner pid=2224264)                               'expert_parallel_size': 1,
(TaskRunner pid=2224264)                               'free_cache_engine': True,
(TaskRunner pid=2224264)                               'gpu_memory_utilization': 0.5,
(TaskRunner pid=2224264)                               'limit_images': None,
(TaskRunner pid=2224264)                               'load_format': 'auto',
(TaskRunner pid=2224264)                               'max_model_len': None,
(TaskRunner pid=2224264)                               'max_num_batched_tokens': 8192,
(TaskRunner pid=2224264)                               'max_num_seqs': 1024,
(TaskRunner pid=2224264)                               'name': '???',
(TaskRunner pid=2224264)                               'prompt_length': 2048,
(TaskRunner pid=2224264)                               'response_length': 2048,
(TaskRunner pid=2224264)                               'skip_tokenizer_init': False,
(TaskRunner pid=2224264)                               'tensor_model_parallel_size': 2},
(TaskRunner pid=2224264)                   'sandbox_fusion': {'max_concurrent': 64,
(TaskRunner pid=2224264)                                      'memory_limit_mb': 1024,
(TaskRunner pid=2224264)                                      'url': None},
(TaskRunner pid=2224264)                   'strategy': 'fsdp',
(TaskRunner pid=2224264)                   'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2224264)                   'use_dynamic_bsz': False,
(TaskRunner pid=2224264)                   'use_reward_loop': False},
(TaskRunner pid=2224264)  'trainer': {'balance_batch': True,
(TaskRunner pid=2224264)              'critic_warmup': 0,
(TaskRunner pid=2224264)              'default_hdfs_dir': None,
(TaskRunner pid=2224264)              'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8',
(TaskRunner pid=2224264)              'del_local_ckpt_after_load': False,
(TaskRunner pid=2224264)              'device': 'cuda',
(TaskRunner pid=2224264)              'esi_redundant_time': 0,
(TaskRunner pid=2224264)              'experiment_name': 'qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8',
(TaskRunner pid=2224264)              'group_name': 'QWEN3-GRPO-generalization',
(TaskRunner pid=2224264)              'log_val_generations': 0,
(TaskRunner pid=2224264)              'logger': ['console', 'wandb'],
(TaskRunner pid=2224264)              'max_actor_ckpt_to_keep': 100,
(TaskRunner pid=2224264)              'max_critic_ckpt_to_keep': None,
(TaskRunner pid=2224264)              'n_gpus_per_node': 8,
(TaskRunner pid=2224264)              'nnodes': 1,
(TaskRunner pid=2224264)              'project_name': 'SDPO-root',
(TaskRunner pid=2224264)              'ray_wait_register_center_timeout': 300,
(TaskRunner pid=2224264)              'resume_from_path': None,
(TaskRunner pid=2224264)              'resume_mode': 'auto',
(TaskRunner pid=2224264)              'rollout_data_dir': None,
(TaskRunner pid=2224264)              'save_freq': 10,
(TaskRunner pid=2224264)              'test_freq': 10,
(TaskRunner pid=2224264)              'total_epochs': 30,
(TaskRunner pid=2224264)              'total_training_steps': 100,
(TaskRunner pid=2224264)              'use_legacy_worker_impl': 'auto',
(TaskRunner pid=2224264)              'val_before_train': False,
(TaskRunner pid=2224264)              'val_only': False,
(TaskRunner pid=2224264)              'validation_data_dir': None},
(TaskRunner pid=2224264)  'transfer_queue': {'enable': False},
(TaskRunner pid=2224264)  'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/chemistry',
(TaskRunner pid=2224264)           'dir': '/users/root/SDPO',
(TaskRunner pid=2224264)           'log_dir': '/users/root/output',
(TaskRunner pid=2224264)           'task': 'datasets/sciknoweval/chemistry'}}
(TaskRunner pid=2224264) [validate_config] All configuration checks passed successfully!
(TaskRunner pid=2224264) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True
(TaskRunner pid=2224264)   use_critic=need_critic(config),
(TaskRunner pid=2224264) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now.
(TaskRunner pid=2224264)   warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1)
(TaskRunner pid=2224264) Using dataset class: RLHFDataset
(TaskRunner pid=2224264) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(TaskRunner pid=2224264)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(TaskRunner pid=2224264) dataset len: 1890
(TaskRunner pid=2224264) Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2224264) WARNING:2026-07-02 23:56:38,560:Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2224264) 
Filtering prompts longer than 2048 tokens (num_proc=1):   0%|          | 0/1890 [00:00<?, ? examples/s]
(TaskRunner pid=2224264) 
Filtering prompts longer than 2048 tokens (num_proc=1):  53%|█████▎    | 1000/1890 [00:00<00:00, 1041.98 examples/s]
(TaskRunner pid=2224264) 
Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 1890/1890 [00:01<00:00, 1506.61 examples/s]
(TaskRunner pid=2224264) filter dataset len: 1890
(TaskRunner pid=2224264) Using dataset class: RLHFDataset
(TaskRunner pid=2224264) 
Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 1890/1890 [00:01<00:00, 1304.24 examples/s]
(TaskRunner pid=2224264) dataset len: 210
(TaskRunner pid=2224264) Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2224264) WARNING:2026-07-02 23:56:40,469:Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2224264) 
Filtering prompts longer than 2048 tokens (num_proc=1):   0%|          | 0/210 [00:00<?, ? examples/s]
(TaskRunner pid=2224264) 
Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 210/210 [00:00<00:00, 349.33 examples/s]
(TaskRunner pid=2224264) filter dataset len: 210
(TaskRunner pid=2224264) Size of train dataloader: 59, Size of val dataloader: 1
(TaskRunner pid=2224264) Total training steps: 100
(TaskRunner pid=2224264) 
Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 210/210 [00:00<00:00, 298.23 examples/s]
(TaskRunner pid=2224264) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True
(TaskRunner pid=2224264)   self.use_critic = need_critic(self.config)
(TaskRunner pid=2224264) colocated worker base class <class 'verl.single_controller.base.worker.Worker'>
(TaskRunner pid=2224264) bind role actor_rollout method chat_completion to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2224264) bind role actor_rollout method generate to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2224264) bind role actor_rollout method get_zeromq_address to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2224264) bind role actor_rollout method sleep to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2224264) bind role actor_rollout method wake_up to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(WorkerDict pid=2224678) [W702 23:56:48.468298935 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:38433 (errno: 97 - Address family not supported by protocol).
(WorkerDict pid=2224673) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7
(WorkerDict pid=2224673) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(WorkerDict pid=2224673)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(WorkerDict pid=2224680) [W702 23:56:48.948596036 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:38433 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.)
(WorkerDict pid=2224673) Model config after override: Qwen3Config {
(WorkerDict pid=2224673)   "architectures": [
(WorkerDict pid=2224673)     "Qwen3ForCausalLM"
(WorkerDict pid=2224673)   ],
(WorkerDict pid=2224673)   "attention_bias": false,
(WorkerDict pid=2224673)   "attention_dropout": 0.0,
(WorkerDict pid=2224673)   "dtype": "bfloat16",
(WorkerDict pid=2224673)   "eos_token_id": 151645,
(WorkerDict pid=2224673)   "head_dim": 128,
(WorkerDict pid=2224673)   "hidden_act": "silu",
(WorkerDict pid=2224673)   "hidden_size": 4096,
(WorkerDict pid=2224673)   "initializer_range": 0.02,
(WorkerDict pid=2224673)   "intermediate_size": 12288,
(WorkerDict pid=2224673)   "layer_types": [
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention",
(WorkerDict pid=2224673)     "full_attention"
(WorkerDict pid=2224673)   ],
(WorkerDict pid=2224673)   "max_position_embeddings": 40960,
(WorkerDict pid=2224673)   "max_window_layers": 36,
(WorkerDict pid=2224673)   "model_type": "qwen3",
(WorkerDict pid=2224673)   "num_attention_heads": 32,
(WorkerDict pid=2224673)   "num_hidden_layers": 36,
(WorkerDict pid=2224673)   "num_key_value_heads": 8,
(WorkerDict pid=2224673)   "pad_token_id": 151643,
(WorkerDict pid=2224673)   "rms_norm_eps": 1e-06,
(WorkerDict pid=2224673)   "rope_scaling": null,
(WorkerDict pid=2224673)   "rope_theta": 1000000,
(WorkerDict pid=2224673)   "sliding_window": null,
(WorkerDict pid=2224673)   "tie_word_embeddings": false,
(WorkerDict pid=2224673)   "transformers_version": "4.57.1",
(WorkerDict pid=2224673)   "use_cache": true,
(WorkerDict pid=2224673)   "use_sliding_window": false,
(WorkerDict pid=2224673)   "vocab_size": 151936
(WorkerDict pid=2224673) }
(WorkerDict pid=2224673) 
(WorkerDict pid=2224673) `torch_dtype` is deprecated! Use `dtype` instead!
(WorkerDict pid=2224673) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)`
(WorkerDict pid=2224673) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)`
(WorkerDict pid=2224673) 
Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s]
(WorkerDict pid=2224673) 
Loading checkpoint shards:  20%|██        | 1/5 [00:02<00:11,  2.96s/it]
(WorkerDict pid=2224680) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster]
(WorkerDict pid=2224680)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster]
(WorkerDict pid=2224680) `torch_dtype` is deprecated! Use `dtype` instead! [repeated 7x across cluster]
(WorkerDict pid=2224680) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` [repeated 14x across cluster]
(WorkerDict pid=2224680) 
Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s] [repeated 7x across cluster]
(WorkerDict pid=2224673) 
Loading checkpoint shards:  60%|██████    | 3/5 [00:08<00:05,  2.94s/it] [repeated 16x across cluster]
(WorkerDict pid=2224673) 
Loading checkpoint shards: 100%|██████████| 5/5 [00:12<00:00,  2.06s/it]
Loading checkpoint shards: 100%|██████████| 5/5 [00:12<00:00,  2.42s/it]
(WorkerDict pid=2224673) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention
(WorkerDict pid=2224673) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch
(WorkerDict pid=2224680) [Gloo] Rank 7 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 [repeated 7x across cluster]
(WorkerDict pid=2224673) Qwen3ForCausalLM contains 8.19B parameters
(WorkerDict pid=2224673) wrap_policy: functools.partial(<function _or_policy at 0x7283ffa8c540>, policies=[functools.partial(<function transformer_auto_wrap_policy at 0x7283ffa8c400>, transformer_layer_cls={<class 'transformers.models.qwen3.modeling_qwen3.Qwen3DecoderLayer'>})])
(WorkerDict pid=2224673) NCCL version 2.27.5+cuda12.9
(WorkerDict pid=2224674) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now.
(WorkerDict pid=2224674)   warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1)
(WorkerDict pid=2224680) 
Loading checkpoint shards:  80%|████████  | 4/5 [00:12<00:02,  2.93s/it] [repeated 15x across cluster]
(WorkerDict pid=2224680) 
Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00,  2.22s/it]
Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00,  2.62s/it] [repeated 7x across cluster]
(WorkerDict pid=2224673) Total steps: 100, num_warmup_steps: 10
(WorkerDict pid=2224673) Actor use_remove_padding=True
(WorkerDict pid=2224673) Actor use_fused_kernels=False
(WorkerDict pid=2224673) Actor use_prefix_grouper=False
(WorkerDict pid=2224680) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster]
(WorkerDict pid=2224680) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster]
(WorkerDict pid=2224674) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(WorkerDict pid=2224674)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(WorkerDict pid=2224674) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3
(WorkerDict pid=2224673) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html .
(WorkerDict pid=2224673)   warnings.warn(
(WorkerDict pid=2224673) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster]
(WorkerDict pid=2224673)   warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster]
(WorkerDict pid=2224673) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster]
(WorkerDict pid=2224673)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster]
(WorkerDict pid=2224680) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster]
(WorkerDict pid=2224680)   warnings.warn( [repeated 7x across cluster]
(TaskRunner pid=2224264) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now.
(TaskRunner pid=2224264)   warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1)
(vLLMHttpServer pid=2225569) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(vLLMHttpServer pid=2225569)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(vLLMHttpServer pid=2225570) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(vLLMHttpServer pid=2225570)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(vLLMHttpServer pid=2225569) ['serve',
(vLLMHttpServer pid=2225569)  'Qwen/Qwen3-8B',
(vLLMHttpServer pid=2225569)  '--dtype',
(vLLMHttpServer pid=2225569)  'bfloat16',
(vLLMHttpServer pid=2225569)  '--load_format',
(vLLMHttpServer pid=2225569)  'dummy',
(vLLMHttpServer pid=2225569)  '--trust_remote_code',
(vLLMHttpServer pid=2225569)  '--max_model_len',
(vLLMHttpServer pid=2225569)  '40960',
(vLLMHttpServer pid=2225569)  '--max_num_seqs',
(vLLMHttpServer pid=2225569)  '1024',
(vLLMHttpServer pid=2225569)  '--enable_chunked_prefill',
(vLLMHttpServer pid=2225569)  '--max_num_batched_tokens',
(vLLMHttpServer pid=2225569)  '10240',
(vLLMHttpServer pid=2225569)  '--enable_prefix_caching',
(vLLMHttpServer pid=2225569)  '--enable_sleep_mode',
(vLLMHttpServer pid=2225569)  '--logprobs_mode',
(vLLMHttpServer pid=2225569)  'processed_logprobs',
(vLLMHttpServer pid=2225569)  '--disable_custom_all_reduce',
(vLLMHttpServer pid=2225569)  '--gpu_memory_utilization',
(vLLMHttpServer pid=2225569)  '0.8',
(vLLMHttpServer pid=2225569)  '--disable_log_stats',
(vLLMHttpServer pid=2225569)  '--tensor_parallel_size',
(vLLMHttpServer pid=2225569)  '2',
(vLLMHttpServer pid=2225569)  '--seed',
(vLLMHttpServer pid=2225569)  '0',
(vLLMHttpServer pid=2225569)  '--override_generation_config',
(vLLMHttpServer pid=2225569)  '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, '
(vLLMHttpServer pid=2225569)  '"max_new_tokens": 8192}',
(vLLMHttpServer pid=2225569)  '--hf_overrides',
(vLLMHttpServer pid=2225569)  '{}',
(vLLMHttpServer pid=2225569)  '--scheduling_policy',
(vLLMHttpServer pid=2225569)  'fcfs']
(WorkerDict pid=2224680) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster]
(vLLMHttpServer pid=2225569) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead.
(vLLMHttpServer pid=2225569) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(vLLMHttpServer pid=2225569) WARNING 07-02 23:57:38 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned
(WorkerDict pid=2224673) WARNING 07-02 23:57:46 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'.
(vLLMHttpServer pid=2225571) WARNING 07-02 23:57:39 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster]
(WorkerDict pid=2224675) NCCL version 2.27.5+cuda12.9
(WorkerDict pid=2224677) 2026-07-02 23:58:01,108 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ...
(vLLMHttpServer pid=2225571) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 2x across cluster]
(vLLMHttpServer pid=2225571)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 2x across cluster]
(vLLMHttpServer pid=2225571) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster]
(vLLMHttpServer pid=2225571) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster]
(WorkerDict pid=2224678) 2026-07-02 23:58:01,122 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   0%|          | 0/51 [00:00<?, ?it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   2%|▏         | 1/51 [00:00<00:05,  9.84it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   6%|▌         | 3/51 [00:00<00:03, 15.33it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  10%|▉         | 5/51 [00:00<00:02, 16.95it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  14%|█▎        | 7/51 [00:00<00:02, 17.32it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  18%|█▊        | 9/51 [00:00<00:02, 17.07it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  22%|██▏       | 11/51 [00:00<00:02, 17.25it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  25%|██▌       | 13/51 [00:00<00:02, 17.04it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  29%|██▉       | 15/51 [00:00<00:02, 16.86it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  33%|███▎      | 17/51 [00:01<00:02, 16.82it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  37%|███▋      | 19/51 [00:01<00:01, 17.13it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  41%|████      | 21/51 [00:01<00:01, 17.13it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  45%|████▌     | 23/51 [00:01<00:01, 16.62it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  49%|████▉     | 25/51 [00:01<00:01, 16.48it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  53%|█████▎    | 27/51 [00:01<00:01, 16.36it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  57%|█████▋    | 29/51 [00:01<00:01, 15.74it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  61%|██████    | 31/51 [00:01<00:01, 15.80it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  65%|██████▍   | 33/51 [00:02<00:01, 15.50it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  69%|██████▊   | 35/51 [00:02<00:01, 15.58it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  73%|███████▎  | 37/51 [00:02<00:00, 15.29it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  76%|███████▋  | 39/51 [00:02<00:00, 14.89it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  80%|████████  | 41/51 [00:02<00:00, 14.86it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  84%|████████▍ | 43/51 [00:02<00:00, 13.86it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  88%|████████▊ | 45/51 [00:02<00:00, 13.92it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  92%|█████████▏| 47/51 [00:03<00:00, 13.81it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  96%|█████████▌| 49/51 [00:03<00:00, 12.90it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 51/51 [00:03<00:00, 13.17it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 51/51 [00:03<00:00, 15.25it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (decode, FULL):   0%|          | 0/51 [00:00<?, ?it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (decode, FULL):   4%|▍         | 2/51 [00:00<00:02, 18.50it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (decode, FULL):  10%|▉         | 5/51 [00:00<00:02, 20.71it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (decode, FULL):  16%|█▌        | 8/51 [00:00<00:02, 20.25it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (decode, FULL):  22%|██▏       | 11/51 [00:00<00:01, 21.00it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (decode, FULL):  27%|██▋       | 14/51 [00:00<00:01, 21.25it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (decode, FULL):  33%|███▎      | 17/51 [00:00<00:01, 21.37it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (decode, FULL):  39%|███▉      | 20/51 [00:00<00:01, 21.58it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (decode, FULL):  45%|████▌     | 23/51 [00:01<00:01, 21.93it/s]
(WorkerDict pid=2224674) 2026-07-02 23:58:01,335 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... [repeated 7x across cluster]
(WorkerDict pid=2224674) 2026-07-02 23:58:01,351 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends [repeated 7x across cluster]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (decode, FULL):  51%|█████     | 26/51 [00:01<00:01, 22.21it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (decode, FULL):  57%|█████▋    | 29/51 [00:01<00:00, 22.31it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (decode, FULL):  63%|██████▎   | 32/51 [00:01<00:00, 22.42it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (decode, FULL):  69%|██████▊   | 35/51 [00:01<00:00, 22.45it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (decode, FULL):  75%|███████▍  | 38/51 [00:01<00:00, 22.29it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (decode, FULL):  80%|████████  | 41/51 [00:01<00:00, 21.34it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (decode, FULL):  86%|████████▋ | 44/51 [00:02<00:00, 21.18it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (decode, FULL):  92%|█████████▏| 47/51 [00:02<00:00, 20.82it/s]
(WorkerDict pid=2224673) 
Capturing CUDA graphs (decode, FULL):  98%|█████████▊| 50/51 [00:02<00:00, 21.22it/s]
Capturing CUDA graphs (decode, FULL): 100%|██████████| 51/51 [00:02<00:00, 21.51it/s]
(vLLMHttpServer pid=2225571) WARNING 07-02 23:58:09 [api_server.py:1358] LoRA dynamic loading & unloading is enabled in the API server. This should ONLY be used for local development!
(WorkerDict pid=2224676) WARNING 07-02 23:57:47 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. [repeated 7x across cluster]
(WorkerDict pid=2224679) NCCL version 2.27.5+cuda12.9 [repeated 2x across cluster]
(vLLMHttpServer pid=2225571) WARNING 07-02 23:58:09 [model.py:1576] Default sampling parameters have been overridden by the model's Hugging Face generation config recommended from the model creator. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`.
(TaskRunner pid=2224264) AgentLoopManager: ['198.19.44.212:42387', '198.19.44.212:39253', '198.19.44.212:42941', '198.19.44.212:38755']
(TaskRunner pid=2224264) wandb: Currently logged in as: seongryongjung (seongryongjung-chung-ang-university) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
(TaskRunner pid=2224264) wandb: setting up run ffovp19e
(TaskRunner pid=2224264) wandb: Tracking run with wandb version 0.23.1
(TaskRunner pid=2224264) wandb: Run data is saved locally in /mnt/mole/SDPO/L2T/wandb/run-20260702_235814-ffovp19e
(TaskRunner pid=2224264) wandb: Run `wandb offline` to turn off syncing.
(TaskRunner pid=2224264) wandb: Syncing run qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8
(TaskRunner pid=2224264) wandb: ⭐️ View project at https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root
(TaskRunner pid=2224264) wandb: 🚀 View run at https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/ffovp19e
(TaskRunner pid=2224264) Checkpoint tracker file does not exist: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/latest_checkpointed_iteration.txt
(TaskRunner pid=2224264) Training from scratch
(vLLMHttpServer pid=2225569) WARNING 07-02 23:58:09 [api_server.py:1358] LoRA dynamic loading & unloading is enabled in the API server. This should ONLY be used for local development! [repeated 3x across cluster]
(vLLMHttpServer pid=2225569) WARNING 07-02 23:58:10 [model.py:1576] Default sampling parameters have been overridden by the model's Hugging Face generation config recommended from the model creator. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`. [repeated 3x across cluster]
(TaskRunner pid=2224264) wandb: Detected [openai] in use.
(TaskRunner pid=2224264) wandb: Use W&B Weave for improved LLM call tracing. Install Weave with `pip install weave` then add `import weave` to the top of your script.
(TaskRunner pid=2224264) wandb: For more information, check out the docs at: https://weave-docs.wandb.ai/
(TaskRunner pid=2224264) 
Training Progress:   0%|          | 0/100 [00:00<?, ?it/s]
(AgentLoopWorker pid=2226513) Using dataset class: RLHFDataset
(AgentLoopWorker pid=2226513) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(AgentLoopWorker pid=2226513)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(AgentLoopWorker pid=2226515) You're using a Qwen2TokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.
(AgentLoopWorker pid=2226522) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster]
(AgentLoopWorker pid=2226522)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster]
(TaskRunner pid=2224264) step:1 - global_seqlen/min:19322 - global_seqlen/max:25315 - global_seqlen/minmax_diff:5993 - global_seqlen/balanced_min:22987 - global_seqlen/balanced_max:23220 - global_seqlen/mean:23039.25 - actor/entropy:0.5288177132606506 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21512234210968018 - training/rollout_probs_diff_mean:0.004612532444298267 - training/rollout_probs_diff_std:0.009421703405678272 - training/rollout_actor_probs_pearson_corr:0.9995055794715881 - rollout_corr/rollout_is_mean:0.9999278783798218 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.5366545742144808e-05 - rollout_corr/rollout_is_max:1.599510669708252 - rollout_corr/rollout_is_min:0.4775051474571228 - rollout_corr/rollout_is_std:0.03466796875 - rollout_corr/rollout_is_eff_sample_size:0.9987994557723341 - rollout_corr/rollout_is_seq_mean:0.9999526739120483 - rollout_corr/rollout_is_seq_std:0.0017850211588665843 - rollout_corr/rollout_is_seq_max:1.0056862831115723 - rollout_corr/rollout_is_seq_min:0.9945246577262878 - rollout_corr/rollout_is_seq_max_deviation:0.005686283111572266 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7541180718690157) - rollout_corr/training_log_ppl:np.float64(0.5497869531391189) - rollout_corr/kl:np.float64(0.00068277636619829) - rollout_corr/k3_kl:np.float64(0.0006354950018021555) - rollout_corr/rollout_ppl:np.float64(1.7529067401774228) - rollout_corr/rollout_log_ppl:np.float64(0.5491041730856523) - rollout_corr/log_ppl_diff:np.float64(0.0006827800534665585) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015149706741794944) - rollout_corr/log_ppl_diff_max:np.float64(0.005993187427520752) - rollout_corr/log_ppl_diff_min:np.float64(-0.004957437515258789) - rollout_corr/ppl_ratio:np.float64(1.0006846021860838) - rollout_corr/chi2_token:np.float64(0.0011827307753264904) - rollout_corr/chi2_seq:np.float64(0.5518562712240964) - actor/pg_loss:np.float64(-0.00024519627913832664) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(0.0) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3594253621995449) - perf/mfu/actor:np.float64(0.08964318930500031) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.935546875) - perf/cpu_memory_used_gb:np.float64(102.53416061401367) - actor/lr:np.float64(0.0) - training/global_step:1 - training/epoch:0 - critic/score/mean:0.48046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0064135086722671986 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0064135086722671986 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:461.9765625 - response_length/max:1716.0 - response_length/min:149.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:461.9765625 - response_length_non_aborted/max:1716.0 - response_length_non_aborted/min:149.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.0 - prompt_length/max:491.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00023651309311389923 - timing_s/agent_loop/generate_sequences/min:np.float64(13.904150538146496) - timing_s/agent_loop/generate_sequences/max:np.float64(27.40781383961439) - timing_s/agent_loop/generate_sequences/mean:np.float64(18.873495450563496) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.40781383961439) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_
(AgentLoopWorker pid=2226520) Using dataset class: RLHFDataset [repeated 7x across cluster]
(TaskRunner pid=2224264) 
Training Progress:   1%|          | 1/100 [00:56<1:33:50, 56.88s/it]
(AgentLoopWorker pid=2226513) You're using a Qwen2TokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding. [repeated 7x across cluster]
(TaskRunner pid=2224264) step:2 - global_seqlen/min:18367 - global_seqlen/max:28812 - global_seqlen/minmax_diff:10445 - global_seqlen/balanced_min:24319 - global_seqlen/balanced_max:24347 - global_seqlen/mean:24339.125 - actor/entropy:0.5474216938018799 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5073403120040894 - training/rollout_probs_diff_mean:0.00468479422852397 - training/rollout_probs_diff_std:0.00983211025595665 - training/rollout_actor_probs_pearson_corr:0.9994499683380127 - rollout_corr/rollout_is_mean:1.00003981590271 - rollout_corr/rollout_is_ratio_fraction_high:8.855121450324077e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.855121450324077e-06 - rollout_corr/rollout_is_max:2.8898818492889404 - rollout_corr/rollout_is_min:0.4623408317565918 - rollout_corr/rollout_is_std:0.03588784113526344 - rollout_corr/rollout_is_eff_sample_size:0.9987138383870012 - rollout_corr/rollout_is_seq_mean:1.0000025033950806 - rollout_corr/rollout_is_seq_std:0.002084122272208333 - rollout_corr/rollout_is_seq_max:1.0075422525405884 - rollout_corr/rollout_is_seq_min:0.9931289553642273 - rollout_corr/rollout_is_seq_max_deviation:0.007542252540588379 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.808347711339593) - rollout_corr/training_log_ppl:np.float64(0.5770619873364922) - rollout_corr/kl:np.float64(0.0005897953656575083) - rollout_corr/k3_kl:np.float64(0.000690597933726167) - rollout_corr/rollout_ppl:np.float64(1.8072920083068311) - rollout_corr/rollout_log_ppl:np.float64(0.5764721901505254) - rollout_corr/log_ppl_diff:np.float64(0.000589797185966745) - rollout_corr/log_ppl_abs_diff:np.float64(0.001534482667921111) - rollout_corr/log_ppl_diff_max:np.float64(0.00782063603401184) - rollout_corr/log_ppl_diff_min:np.float64(-0.004696309566497803) - rollout_corr/ppl_ratio:np.float64(1.0005918587557971) - rollout_corr/chi2_token:np.float64(0.0016041339840739965) - rollout_corr/chi2_seq:np.float64(0.7937973756343126) - actor/pg_loss:np.float64(1.557217910885811e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00031911871974443784) - actor/ppo_kl:np.float64(-0.00011317147449219078) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3977566659450531) - perf/mfu/actor:np.float64(0.09740370282378204) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(101.18880462646484) - actor/lr:np.float64(1e-07) - training/global_step:2 - training/epoch:0 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014309875667095184 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.014309875667095184 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:441.12890625 - response_length/max:1401.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:441.12890625 - response_length_non_aborted/max:1401.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:319.46875 - prompt_length/max:818.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001631416380405426 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2865754403173923) - timing_s/agent_loop/generate_sequences/max:np.float64(10.83000554330647) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.861061018127657) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.83000554330647) - timing_s/agent_loop/slowest/too
(TaskRunner pid=2224264) 
Training Progress:   2%|▏         | 2/100 [01:31<1:11:14, 43.61s/it]
(TaskRunner pid=2224264) step:3 - global_seqlen/min:19797 - global_seqlen/max:28335 - global_seqlen/minmax_diff:8538 - global_seqlen/balanced_min:24282 - global_seqlen/balanced_max:24340 - global_seqlen/mean:24297.5 - actor/entropy:0.5263562202453613 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.13323187828063965 - training/rollout_probs_diff_mean:0.004719756543636322 - training/rollout_probs_diff_std:0.009652746841311455 - training/rollout_actor_probs_pearson_corr:0.9994633197784424 - rollout_corr/rollout_is_mean:0.9999251961708069 - rollout_corr/rollout_is_ratio_fraction_high:8.550809070584364e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.3356199264526367 - rollout_corr/rollout_is_min:0.5226753354072571 - rollout_corr/rollout_is_std:0.03476754575967789 - rollout_corr/rollout_is_eff_sample_size:0.998792439351094 - rollout_corr/rollout_is_seq_mean:0.9998748302459717 - rollout_corr/rollout_is_seq_std:0.001937155844643712 - rollout_corr/rollout_is_seq_max:1.0051332712173462 - rollout_corr/rollout_is_seq_min:0.994463324546814 - rollout_corr/rollout_is_seq_max_deviation:0.005536675453186035 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7754999860189855) - rollout_corr/training_log_ppl:np.float64(0.5597236861940473) - rollout_corr/kl:np.float64(0.000738800011342633) - rollout_corr/k3_kl:np.float64(0.000668586852611952) - rollout_corr/rollout_ppl:np.float64(1.7741392091847956) - rollout_corr/rollout_log_ppl:np.float64(0.5589848817326128) - rollout_corr/log_ppl_diff:np.float64(0.0007388044614344835) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017042472027242184) - rollout_corr/log_ppl_diff_max:np.float64(0.0062329769134521484) - rollout_corr/log_ppl_diff_min:np.float64(-0.005337923765182495) - rollout_corr/ppl_ratio:np.float64(1.0007410887628794) - rollout_corr/chi2_token:np.float64(0.0012068876530975103) - rollout_corr/chi2_seq:np.float64(1.152974521741271) - actor/pg_loss:np.float64(-0.00012802286073565483) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003878075890497712) - actor/ppo_kl:np.float64(-4.74486657751072e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4122833386063576) - perf/mfu/actor:np.float64(0.09657854222960446) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(100.6053695678711) - actor/lr:np.float64(2e-07) - training/global_step:3 - training/epoch:0 - critic/score/mean:0.44921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00122917874250561 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00122917874250561 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:456.828125 - response_length/max:1459.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:456.828125 - response_length_non_aborted/max:1459.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.46875 - prompt_length/max:557.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.80570912361145e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4546770006418228) - timing_s/agent_loop/generate_sequences/max:np.float64(11.504085250198841) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.162285889819032) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.504085250198841) - timing_s/agent_loop/slowest/tool_calls:np.f
(TaskRunner pid=2224264) 
Training Progress:   3%|▎         | 3/100 [02:06<1:04:33, 39.93s/it]
(TaskRunner pid=2224264) step:4 - global_seqlen/min:22442 - global_seqlen/max:26662 - global_seqlen/minmax_diff:4220 - global_seqlen/balanced_min:24730 - global_seqlen/balanced_max:24744 - global_seqlen/mean:24738.75 - actor/entropy:0.5523660778999329 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6050522327423096 - training/rollout_probs_diff_mean:0.004622706677764654 - training/rollout_probs_diff_std:0.009672068059444427 - training/rollout_actor_probs_pearson_corr:0.9995031356811523 - rollout_corr/rollout_is_mean:1.0001981258392334 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.5127732442342676e-05 - rollout_corr/rollout_is_max:1.8165526390075684 - rollout_corr/rollout_is_min:0.13162213563919067 - rollout_corr/rollout_is_std:0.03505437821149826 - rollout_corr/rollout_is_eff_sample_size:0.998773174430212 - rollout_corr/rollout_is_seq_mean:1.0002080202102661 - rollout_corr/rollout_is_seq_std:0.0018218032782897353 - rollout_corr/rollout_is_seq_max:1.005950689315796 - rollout_corr/rollout_is_seq_min:0.9943360090255737 - rollout_corr/rollout_is_seq_max_deviation:0.0059506893157958984 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8033743798732758) - rollout_corr/training_log_ppl:np.float64(0.5734791937866248) - rollout_corr/kl:np.float64(0.0005666991043140612) - rollout_corr/k3_kl:np.float64(0.0006585608642808438) - rollout_corr/rollout_ppl:np.float64(1.8023865297436714) - rollout_corr/rollout_log_ppl:np.float64(0.5729124903446063) - rollout_corr/log_ppl_diff:np.float64(0.0005667034420184791) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015296083292923868) - rollout_corr/log_ppl_diff_max:np.float64(0.006748795509338379) - rollout_corr/log_ppl_diff_min:np.float64(-0.005822420120239258) - rollout_corr/ppl_ratio:np.float64(1.000568695133552) - rollout_corr/chi2_token:np.float64(0.001501972321420908) - rollout_corr/chi2_seq:np.float64(1.0207945266738534) - actor/pg_loss:np.float64(2.6238849386572838e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014007428308104863) - actor/ppo_kl:np.float64(0.00010998321266719735) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3790104538202286) - perf/mfu/actor:np.float64(0.10117752971397821) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(99.83869171142578) - actor/lr:np.float64(3e-07) - training/global_step:4 - training/epoch:0 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008165465667843819 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008165465667843819 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:466.3671875 - response_length/max:1192.0 - response_length/min:113.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:466.3671875 - response_length_non_aborted/max:1192.0 - response_length_non_aborted/min:113.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:306.71875 - prompt_length/max:604.0 - prompt_length/min:173.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015220977365970612 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4423759039491415) - timing_s/agent_loop/generate_sequences/max:np.float64(10.007848922163248) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.2785777673416305) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.007848922163248) - timing_s/agent_loop/slowest/too
(TaskRunner pid=2224264) 
Training Progress:   4%|▍         | 4/100 [02:39<59:31, 37.20s/it]  
(TaskRunner pid=2224264) step:5 - global_seqlen/min:18776 - global_seqlen/max:28441 - global_seqlen/minmax_diff:9665 - global_seqlen/balanced_min:22325 - global_seqlen/balanced_max:29119 - global_seqlen/mean:23183.375 - actor/entropy:0.5180944204330444 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2772417664527893 - training/rollout_probs_diff_mean:0.004665252286940813 - training/rollout_probs_diff_std:0.009622659534215927 - training/rollout_actor_probs_pearson_corr:0.9994248151779175 - rollout_corr/rollout_is_mean:0.9999130964279175 - rollout_corr/rollout_is_ratio_fraction_high:8.664685083203949e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.664685083203949e-06 - rollout_corr/rollout_is_max:2.8200924396514893 - rollout_corr/rollout_is_min:0.4454187750816345 - rollout_corr/rollout_is_std:0.03461463004350662 - rollout_corr/rollout_is_eff_sample_size:0.9988030234819574 - rollout_corr/rollout_is_seq_mean:0.9998463988304138 - rollout_corr/rollout_is_seq_std:0.001816977746784687 - rollout_corr/rollout_is_seq_max:1.0054938793182373 - rollout_corr/rollout_is_seq_min:0.9929466843605042 - rollout_corr/rollout_is_seq_max_deviation:0.00705331563949585 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7746925223618746) - rollout_corr/training_log_ppl:np.float64(0.5630445900387713) - rollout_corr/kl:np.float64(0.0009524658903821148) - rollout_corr/k3_kl:np.float64(0.0006796251157652478) - rollout_corr/rollout_ppl:np.float64(1.7729679476469755) - rollout_corr/rollout_log_ppl:np.float64(0.5620921208319487) - rollout_corr/log_ppl_diff:np.float64(0.0009524692068225704) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016687612733221613) - rollout_corr/log_ppl_diff_max:np.float64(0.006459951400756836) - rollout_corr/log_ppl_diff_min:np.float64(-0.003880620002746582) - rollout_corr/ppl_ratio:np.float64(1.000954748596996) - rollout_corr/chi2_token:np.float64(0.0008346459362655878) - rollout_corr/chi2_seq:np.float64(0.2616440069396049) - actor/pg_loss:np.float64(-1.638184767216444e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003420987316076207) - actor/ppo_kl:np.float64(0.00012373590199388218) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.38139186054468155) - perf/mfu/actor:np.float64(0.08868156608274999) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(99.51568698883057) - actor/lr:np.float64(4e-07) - training/global_step:5 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0017578480765223503 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0017578480765223503 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:450.82421875 - response_length/max:8192.0 - response_length/min:93.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:450.82421875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:273.65625 - prompt_length/max:520.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010846368968486786 - timing_s/agent_loop/generate_sequences/min:np.float64(1.193764079362154) - timing_s/agent_loop/generate_sequences/max:np.float64(59.1775236427784) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.851797078110394) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(59.1775236427784) - 
(TaskRunner pid=2224264) 
Training Progress:   5%|▌         | 5/100 [04:03<1:25:12, 53.81s/it]
(TaskRunner pid=2224264) step:6 - global_seqlen/min:20366 - global_seqlen/max:28896 - global_seqlen/minmax_diff:8530 - global_seqlen/balanced_min:23779 - global_seqlen/balanced_max:24033 - global_seqlen/mean:23815.375 - actor/entropy:0.5188326835632324 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553183436393738 - training/rollout_probs_diff_mean:0.004564165603369474 - training/rollout_probs_diff_std:0.00953100249171257 - training/rollout_actor_probs_pearson_corr:0.9994511604309082 - rollout_corr/rollout_is_mean:0.9998612403869629 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.322029316332191e-06 - rollout_corr/rollout_is_max:1.6952842473983765 - rollout_corr/rollout_is_min:0.40082022547721863 - rollout_corr/rollout_is_std:0.03409762680530548 - rollout_corr/rollout_is_eff_sample_size:0.998838464114971 - rollout_corr/rollout_is_seq_mean:0.9999702572822571 - rollout_corr/rollout_is_seq_std:0.0019584151450544596 - rollout_corr/rollout_is_seq_max:1.0068446397781372 - rollout_corr/rollout_is_seq_min:0.9937593340873718 - rollout_corr/rollout_is_seq_max_deviation:0.006844639778137207 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.75880296388641) - rollout_corr/training_log_ppl:np.float64(0.5500174012850039) - rollout_corr/kl:np.float64(0.0007605231542111568) - rollout_corr/k3_kl:np.float64(0.0006334543155617212) - rollout_corr/rollout_ppl:np.float64(1.7574654794298112) - rollout_corr/rollout_log_ppl:np.float64(0.5492568750632927) - rollout_corr/log_ppl_diff:np.float64(0.0007605262217111886) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016004611388780177) - rollout_corr/log_ppl_diff_max:np.float64(0.006067186594009399) - rollout_corr/log_ppl_diff_min:np.float64(-0.005928128957748413) - rollout_corr/ppl_ratio:np.float64(1.000762639567256) - rollout_corr/chi2_token:np.float64(0.0010106135159730911) - rollout_corr/chi2_seq:np.float64(0.4114875444211066) - actor/pg_loss:np.float64(-0.000129641848616302) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000321736315072485) - actor/ppo_kl:np.float64(9.994746215014061e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.36745208501815796) - perf/mfu/actor:np.float64(0.09673023323806183) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(99.67207098007202) - actor/lr:np.float64(5e-07) - training/global_step:6 - training/epoch:0 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0014771601418033242 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0014771601418033242 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:469.38671875 - response_length/max:1729.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:469.38671875 - response_length_non_aborted/max:1729.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.84375 - prompt_length/max:450.0 - prompt_length/min:173.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010452978312969208 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3636087086051702) - timing_s/agent_loop/generate_sequences/max:np.float64(12.901104992255569) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.271141361357877) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.901104992255569) - timing_s/agent_loop/slowest/too
(TaskRunner pid=2224264) 
Training Progress:   6%|▌         | 6/100 [04:39<1:15:06, 47.94s/it]
(TaskRunner pid=2224264) step:7 - global_seqlen/min:20657 - global_seqlen/max:26337 - global_seqlen/minmax_diff:5680 - global_seqlen/balanced_min:23234 - global_seqlen/balanced_max:23251 - global_seqlen/mean:23239.0 - actor/entropy:0.5476059913635254 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24462002515792847 - training/rollout_probs_diff_mean:0.004715604707598686 - training/rollout_probs_diff_std:0.009818820282816887 - training/rollout_actor_probs_pearson_corr:0.9994401335716248 - rollout_corr/rollout_is_mean:1.0001124143600464 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5858569145202637 - rollout_corr/rollout_is_min:0.5473191738128662 - rollout_corr/rollout_is_std:0.03527304157614708 - rollout_corr/rollout_is_eff_sample_size:0.9987575965499879 - rollout_corr/rollout_is_seq_mean:1.0001611709594727 - rollout_corr/rollout_is_seq_std:0.0020254836417734623 - rollout_corr/rollout_is_seq_max:1.0101655721664429 - rollout_corr/rollout_is_seq_min:0.9944638013839722 - rollout_corr/rollout_is_seq_max_deviation:0.010165572166442871 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7864349940791726) - rollout_corr/training_log_ppl:np.float64(0.5668349982588552) - rollout_corr/kl:np.float64(0.0004977610002434574) - rollout_corr/k3_kl:np.float64(0.000654406503088012) - rollout_corr/rollout_ppl:np.float64(1.7855724152177572) - rollout_corr/rollout_log_ppl:np.float64(0.5663372349808924) - rollout_corr/log_ppl_diff:np.float64(0.0004977632779628038) - rollout_corr/log_ppl_abs_diff:np.float64(0.001586579019203782) - rollout_corr/log_ppl_diff_max:np.float64(0.006533026695251465) - rollout_corr/log_ppl_diff_min:np.float64(-0.008994758129119873) - rollout_corr/ppl_ratio:np.float64(1.0005000154487789) - rollout_corr/chi2_token:np.float64(0.001633110223338008) - rollout_corr/chi2_seq:np.float64(1.2687437802087516) - actor/pg_loss:np.float64(0.000499273301102221) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00025790440895434585) - actor/ppo_kl:np.float64(-1.094456836625568e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.37454719841480255) - perf/mfu/actor:np.float64(0.09487462357860618) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(99.51276779174805) - actor/lr:np.float64(6e-07) - training/global_step:7 - training/epoch:0 - critic/score/mean:0.28125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.28125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004087449051439762 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.004087449051439762 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:437.21875 - response_length/max:1408.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:437.21875 - response_length_non_aborted/max:1408.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:289.0 - prompt_length/max:662.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010518915951251984 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2608289290219545) - timing_s/agent_loop/generate_sequences/max:np.float64(11.097910234704614) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.5351159983474645) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.097910234704614) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timin
(TaskRunner pid=2224264) 
Training Progress:   7%|▋         | 7/100 [05:13<1:07:20, 43.45s/it]
(TaskRunner pid=2224264) step:8 - global_seqlen/min:18447 - global_seqlen/max:32287 - global_seqlen/minmax_diff:13840 - global_seqlen/balanced_min:25765 - global_seqlen/balanced_max:25781 - global_seqlen/mean:25775.5 - actor/entropy:0.5252702832221985 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31757450103759766 - training/rollout_probs_diff_mean:0.004371982999145985 - training/rollout_probs_diff_std:0.009270070120692253 - training/rollout_actor_probs_pearson_corr:0.9994732737541199 - rollout_corr/rollout_is_mean:1.0000965595245361 - rollout_corr/rollout_is_ratio_fraction_high:7.615218237333465e-06 - rollout_corr/rollout_is_ratio_fraction_low:7.615218237333465e-06 - rollout_corr/rollout_is_max:2.049854278564453 - rollout_corr/rollout_is_min:0.36485040187835693 - rollout_corr/rollout_is_std:0.03379738703370094 - rollout_corr/rollout_is_eff_sample_size:0.9988592777394362 - rollout_corr/rollout_is_seq_mean:1.0001250505447388 - rollout_corr/rollout_is_seq_std:0.0016558379866182804 - rollout_corr/rollout_is_seq_max:1.005513310432434 - rollout_corr/rollout_is_seq_min:0.9954277276992798 - rollout_corr/rollout_is_seq_max_deviation:0.005513310432434082 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7645341814495623) - rollout_corr/training_log_ppl:np.float64(0.5513433935120702) - rollout_corr/kl:np.float64(0.0004816958155657858) - rollout_corr/k3_kl:np.float64(0.0006227592039635965) - rollout_corr/rollout_ppl:np.float64(1.763689087703824) - rollout_corr/rollout_log_ppl:np.float64(0.5508616954903118) - rollout_corr/log_ppl_diff:np.float64(0.00048169802175834775) - rollout_corr/log_ppl_abs_diff:np.float64(0.001352956343907863) - rollout_corr/log_ppl_diff_max:np.float64(0.005591928958892822) - rollout_corr/log_ppl_diff_min:np.float64(-0.0054190754890441895) - rollout_corr/ppl_ratio:np.float64(1.0004832921549678) - rollout_corr/chi2_token:np.float64(0.0015316253993660212) - rollout_corr/chi2_seq:np.float64(0.7357629749458283) - actor/pg_loss:np.float64(-0.0002484882716089487) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003153111326810176) - actor/ppo_kl:np.float64(-1.135815940678242e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.33920033276081085) - perf/mfu/actor:np.float64(0.10468805371453549) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(99.58919906616211) - actor/lr:np.float64(7e-07) - training/global_step:8 - training/epoch:0 - critic/score/mean:0.48828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0034925295040011406 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0034925295040011406 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:512.953125 - response_length/max:1337.0 - response_length/min:140.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:512.953125 - response_length_non_aborted/max:1337.0 - response_length_non_aborted/min:140.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:292.53125 - prompt_length/max:720.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.801797568798065e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7887007519602776) - timing_s/agent_loop/generate_sequences/max:np.float64(11.159991431981325) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.707926993221918) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.159991431981325) - timing_s/a
(TaskRunner pid=2224264) 
Training Progress:   8%|▊         | 8/100 [05:48<1:02:17, 40.63s/it]
(TaskRunner pid=2224264) step:9 - global_seqlen/min:21454 - global_seqlen/max:27246 - global_seqlen/minmax_diff:5792 - global_seqlen/balanced_min:24307 - global_seqlen/balanced_max:24315 - global_seqlen/mean:24311.875 - actor/entropy:0.5553610920906067 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6999252438545227 - training/rollout_probs_diff_mean:0.004598262254148722 - training/rollout_probs_diff_std:0.009674612432718277 - training/rollout_actor_probs_pearson_corr:0.9994522333145142 - rollout_corr/rollout_is_mean:0.999959409236908 - rollout_corr/rollout_is_ratio_fraction_high:8.295244697364978e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.3180978789459914e-05 - rollout_corr/rollout_is_max:2.1562492847442627 - rollout_corr/rollout_is_min:0.04011661186814308 - rollout_corr/rollout_is_std:0.034607741981744766 - rollout_corr/rollout_is_eff_sample_size:0.9988037370269406 - rollout_corr/rollout_is_seq_mean:0.9998873472213745 - rollout_corr/rollout_is_seq_std:0.001812683418393135 - rollout_corr/rollout_is_seq_max:1.006127119064331 - rollout_corr/rollout_is_seq_min:0.9938367009162903 - rollout_corr/rollout_is_seq_max_deviation:0.006163299083709717 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8204427626915276) - rollout_corr/training_log_ppl:np.float64(0.579765857430175) - rollout_corr/kl:np.float64(0.0006254475316893604) - rollout_corr/k3_kl:np.float64(0.0006774885101776817) - rollout_corr/rollout_ppl:np.float64(1.8192923632450402) - rollout_corr/rollout_log_ppl:np.float64(0.5791404079063796) - rollout_corr/log_ppl_diff:np.float64(0.0006254495237953961) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016019002650864422) - rollout_corr/log_ppl_diff_max:np.float64(0.007795929908752441) - rollout_corr/log_ppl_diff_min:np.float64(-0.004667341709136963) - rollout_corr/ppl_ratio:np.float64(1.0006277253851295) - rollout_corr/chi2_token:np.float64(0.001466952497139573) - rollout_corr/chi2_seq:np.float64(0.8160945861600339) - actor/pg_loss:np.float64(0.00012180034536868334) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00034742705497592397) - actor/ppo_kl:np.float64(-0.0001504568158097186) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.38103173673152924) - perf/mfu/actor:np.float64(0.09934898847820164) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(99.49935913085938) - actor/lr:np.float64(8e-07) - training/global_step:9 - training/epoch:0 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003955794498324394 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003955794498324394 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:470.90234375 - response_length/max:1203.0 - response_length/min:145.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:470.90234375 - response_length_non_aborted/max:1203.0 - response_length_non_aborted/min:145.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:288.84375 - prompt_length/max:735.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010239146649837494 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8441594671458006) - timing_s/agent_loop/generate_sequences/max:np.float64(10.078126607462764) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.1269525975803845) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.078126607462764) - timi
(TaskRunner pid=2224264) 
Training Progress:   9%|▉         | 9/100 [06:21<58:03, 38.28s/it]  
(TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 10}
(TaskRunner pid=2224264) validation generation end
(TaskRunner pid=2224264) [prompt] system
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2224264) <reasoning>
(TaskRunner pid=2224264) ...
(TaskRunner pid=2224264) </reasoning>
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) ...
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) A
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) user
(TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2224264) Please reason step by step.
(TaskRunner pid=2224264) assistant
(TaskRunner pid=2224264) <think>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) </think>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) [response] <reasoning>
(TaskRunner pid=2224264) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to analyze the structure of the product and identify what functional groups or atoms are present that must be introduced in the synthesis.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) The product has the following key features:
(TaskRunner pid=2224264) - A nitrogen-containing ring (indole-like structure)
(TaskRunner pid=2224264) - A chlorine atom attached to a benzene ring
(TaskRunner pid=2224264) - A trifluoromethyl group (C(F)(F)F) attached to a benzene ring
(TaskRunner pid=2224264) - A substituent group that includes a nitrogen atom bonded to a carbon chain
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) We are given four options (A, B, C, D) and need to identify which one is the only correct reactant that can be used in the synthesis of this product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) Let’s analyze each option:
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) **Option A:**
(TaskRunner pid=2224264) O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1  
(TaskRunner pid=2224264) This option includes the same ring structure as the product, but it has an extra oxygen and a double bond. It seems to be a more complex derivative and may not be a direct reactant for the synthesis of the product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) **Option B:**
(TaskRunner pid=2224264) O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1  
(TaskRunner pid=2224264) This option has a chlorine atom attached to a carbon that is also bonded to a nitrogen. It includes a ring structure and a chlorine atom, but it does not match the exact structure of the product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) **Option C:**
(TaskRunner pid=2224264) O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1  
(TaskRunner pid=2224264) This option includes a trifluoromethyl group (C(F)(F)F), a chlorine atom, and a ring structure. However, it has an oxygen atom attached to a carbon that is also bonded to a ring. This structure is more complex and seems to be a derivative of the product, not a direct reactant.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) **Option D:**
(TaskRunner pid=2224264) O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F  
(TaskRunner pid=2224264) This option contains a ring structure, a trifluoromethyl group (C(F)(F)F), and a carbon chain with a double bond. It matches the necessary components of the product, including the trifluoromethyl group and the ring structure. It is the only option that contains the trifluoromethyl group and a carbon chain that could be used to build the product's structure.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) Thus, the only correct reactant that matches the necessary components of the product is **Option D**.
(TaskRunner pid=2224264) </reasoning>
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) D
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) [ground_truth] A
(TaskRunner pid=2224264) [score] 0.0
(TaskRunner pid=2224264) [acc] 0.0
(TaskRunner pid=2224264) [pred] D
(TaskRunner pid=2224264) [incorrect_format] 1
(TaskRunner pid=2224264) [feedback] 
(TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10
(TaskRunner pid=2224264) step:10 - global_seqlen/min:20418 - global_seqlen/max:28068 - global_seqlen/minmax_diff:7650 - global_seqlen/balanced_min:23873 - global_seqlen/balanced_max:23970 - global_seqlen/mean:23892.875 - actor/entropy:0.5462746024131775 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553977370262146 - training/rollout_probs_diff_mean:0.004709707107394934 - training/rollout_probs_diff_std:0.009677772410213947 - training/rollout_actor_probs_pearson_corr:0.9994706511497498 - rollout_corr/rollout_is_mean:0.9998794198036194 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.18444459582679e-06 - rollout_corr/rollout_is_max:1.622496247291565 - rollout_corr/rollout_is_min:0.4166650176048279 - rollout_corr/rollout_is_std:0.034952208399772644 - rollout_corr/rollout_is_eff_sample_size:0.9987795959879308 - rollout_corr/rollout_is_seq_mean:0.9998565912246704 - rollout_corr/rollout_is_seq_std:0.0018025203607976437 - rollout_corr/rollout_is_seq_max:1.0046859979629517 - rollout_corr/rollout_is_seq_min:0.9944931268692017 - rollout_corr/rollout_is_seq_max_deviation:0.00550687313079834 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7906524906866252) - rollout_corr/training_log_ppl:np.float64(0.5681954241008498) - rollout_corr/kl:np.float64(0.000816460652924178) - rollout_corr/k3_kl:np.float64(0.0006859729254529157) - rollout_corr/rollout_ppl:np.float64(1.789125895127654) - rollout_corr/rollout_log_ppl:np.float64(0.5673789661959745) - rollout_corr/log_ppl_diff:np.float64(0.0008164579048752785) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015062049496918917) - rollout_corr/log_ppl_diff_max:np.float64(0.007841110229492188) - rollout_corr/log_ppl_diff_min:np.float64(-0.004483282566070557) - rollout_corr/ppl_ratio:np.float64(1.0008185552433133) - rollout_corr/chi2_token:np.float64(0.0011271920520812273) - rollout_corr/chi2_seq:np.float64(0.35812679631635547) - actor/pg_loss:np.float64(-9.637698531150818e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003609476652854937) - actor/ppo_kl:np.float64(1.5247296355269668e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.38365066051483154) - perf/mfu/actor:np.float64(0.09690410487686144) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(99.56629180908203) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.42410714285714285) - val-aux/sciknoweval/reward/std@16:np.float64(0.3064350931481341) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5519095238095238) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.2843083193630651) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.2967190476190476) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.23547573506551264) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.42388571428571425) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.3061951415792409) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6715952380952382) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.229658540263386) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.21003333333333332) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1505707856063924) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.43492857142857144) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.25799645877611216) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.765652380952381) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.1562366777957225) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.1566142857142857) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09130497085868691) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.44051904761904753) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.20808730912319104) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8
(TaskRunner pid=2224264) 
Training Progress:  10%|█         | 10/100 [09:38<2:10:46, 87.18s/it]
(TaskRunner pid=2224264) step:11 - global_seqlen/min:20294 - global_seqlen/max:26725 - global_seqlen/minmax_diff:6431 - global_seqlen/balanced_min:24058 - global_seqlen/balanced_max:24076 - global_seqlen/mean:24070.25 - actor/entropy:0.5913231372833252 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23963481187820435 - training/rollout_probs_diff_mean:0.004752492066472769 - training/rollout_probs_diff_std:0.009357922710478306 - training/rollout_actor_probs_pearson_corr:0.9995139241218567 - rollout_corr/rollout_is_mean:1.0000724792480469 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.7242271496797912e-05 - rollout_corr/rollout_is_max:1.639595627784729 - rollout_corr/rollout_is_min:0.41323140263557434 - rollout_corr/rollout_is_std:0.03530850633978844 - rollout_corr/rollout_is_eff_sample_size:0.9987550993777402 - rollout_corr/rollout_is_seq_mean:1.0001287460327148 - rollout_corr/rollout_is_seq_std:0.001979757333174348 - rollout_corr/rollout_is_seq_max:1.0079063177108765 - rollout_corr/rollout_is_seq_min:0.9954867959022522 - rollout_corr/rollout_is_seq_max_deviation:0.007906317710876465 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8813741342164576) - rollout_corr/training_log_ppl:np.float64(0.6172284478088841) - rollout_corr/kl:np.float64(0.0006792962190935725) - rollout_corr/k3_kl:np.float64(0.0007297945928712579) - rollout_corr/rollout_ppl:np.float64(1.8801453905180097) - rollout_corr/rollout_log_ppl:np.float64(0.6165491492720321) - rollout_corr/log_ppl_diff:np.float64(0.0006792985368520021) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017337198369204998) - rollout_corr/log_ppl_diff_max:np.float64(0.00989621877670288) - rollout_corr/log_ppl_diff_min:np.float64(-0.006999552249908447) - rollout_corr/ppl_ratio:np.float64(1.000681943492964) - rollout_corr/chi2_token:np.float64(0.0015983630437403917) - rollout_corr/chi2_seq:np.float64(0.8742716582491994) - actor/pg_loss:np.float64(-0.00019931478891521692) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003874804629049322) - actor/ppo_kl:np.float64(0.000144808165869037) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4361106902360916) - perf/mfu/actor:np.float64(0.09751332153344734) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(105.7747859954834) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.5078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005713657476007938 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005713657476007938 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:453.1015625 - response_length/max:1181.0 - response_length/min:128.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:453.1015625 - response_length_non_aborted/max:1181.0 - response_length_non_aborted/min:128.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.09375 - prompt_length/max:620.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.448345422744751e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.323524136096239) - timing_s/agent_loop/generate_sequences/max:np.float64(8.916998555883765) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.45590021499811) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.916998555883765) - timing_s/agent_loop/slowest/tool_
(TaskRunner pid=2224264) 
Training Progress:  11%|█         | 11/100 [10:09<1:44:10, 70.23s/it]
(TaskRunner pid=2224264) step:12 - global_seqlen/min:22279 - global_seqlen/max:28767 - global_seqlen/minmax_diff:6488 - global_seqlen/balanced_min:25313 - global_seqlen/balanced_max:25329 - global_seqlen/mean:25325.625 - actor/entropy:0.5468142628669739 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30783188343048096 - training/rollout_probs_diff_mean:0.004490185528993607 - training/rollout_probs_diff_std:0.009273474104702473 - training/rollout_actor_probs_pearson_corr:0.9995062351226807 - rollout_corr/rollout_is_mean:1.0000845193862915 - rollout_corr/rollout_is_ratio_fraction_high:7.435773568431614e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.075246810913086 - rollout_corr/rollout_is_min:0.5100579261779785 - rollout_corr/rollout_is_std:0.03405739739537239 - rollout_corr/rollout_is_eff_sample_size:0.9988416753032981 - rollout_corr/rollout_is_seq_mean:1.0000801086425781 - rollout_corr/rollout_is_seq_std:0.0016866717487573624 - rollout_corr/rollout_is_seq_max:1.0045456886291504 - rollout_corr/rollout_is_seq_min:0.9936270713806152 - rollout_corr/rollout_is_seq_max_deviation:0.006372928619384766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7922574547119439) - rollout_corr/training_log_ppl:np.float64(0.5712134827044792) - rollout_corr/kl:np.float64(0.0005522373108419742) - rollout_corr/k3_kl:np.float64(0.0006720679998579726) - rollout_corr/rollout_ppl:np.float64(1.7912592180073261) - rollout_corr/rollout_log_ppl:np.float64(0.5706612397334538) - rollout_corr/log_ppl_diff:np.float64(0.0005522429710254073) - rollout_corr/log_ppl_abs_diff:np.float64(0.001352141029201448) - rollout_corr/log_ppl_diff_max:np.float64(0.007192671298980713) - rollout_corr/log_ppl_diff_min:np.float64(-0.003842592239379883) - rollout_corr/ppl_ratio:np.float64(1.0005537504330277) - rollout_corr/chi2_token:np.float64(0.0016048443503677845) - rollout_corr/chi2_seq:np.float64(0.7803806539159268) - actor/pg_loss:np.float64(-5.2671064622700214e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002636122353578685) - actor/ppo_kl:np.float64(2.2634695866941e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3025449775159359) - perf/mfu/actor:np.float64(0.09925995879832478) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.3304214477539) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.44140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003494813572615385 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003494813572615385 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:525.33203125 - response_length/max:1340.0 - response_length/min:219.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:525.33203125 - response_length_non_aborted/max:1340.0 - response_length_non_aborted/min:219.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.09375 - prompt_length/max:665.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001250188797712326 - timing_s/agent_loop/generate_sequences/min:np.float64(2.742388065904379) - timing_s/agent_loop/generate_sequences/max:np.float64(11.409071335569024) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.0465768424546695) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.409071335569024) - timing_s/agent_loop/slowe
(TaskRunner pid=2224264) 
Training Progress:  12%|█▏        | 12/100 [10:45<1:27:35, 59.72s/it]
(TaskRunner pid=2224264) step:13 - global_seqlen/min:22139 - global_seqlen/max:26070 - global_seqlen/minmax_diff:3931 - global_seqlen/balanced_min:23703 - global_seqlen/balanced_max:23955 - global_seqlen/mean:23768.875 - actor/entropy:0.6082448959350586 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8117851614952087 - training/rollout_probs_diff_mean:0.004782064817845821 - training/rollout_probs_diff_std:0.009643850848078728 - training/rollout_actor_probs_pearson_corr:0.9995208978652954 - rollout_corr/rollout_is_mean:0.9998765587806702 - rollout_corr/rollout_is_ratio_fraction_high:8.387642992602196e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.677528598520439e-05 - rollout_corr/rollout_is_max:2.2560038566589355 - rollout_corr/rollout_is_min:0.1834239512681961 - rollout_corr/rollout_is_std:0.03520030155777931 - rollout_corr/rollout_is_eff_sample_size:0.9987622341887205 - rollout_corr/rollout_is_seq_mean:0.9998448491096497 - rollout_corr/rollout_is_seq_std:0.0017469775630161166 - rollout_corr/rollout_is_seq_max:1.0048439502716064 - rollout_corr/rollout_is_seq_min:0.9949353933334351 - rollout_corr/rollout_is_seq_max_deviation:0.005064606666564941 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8870359733700752) - rollout_corr/training_log_ppl:np.float64(0.6238076707813889) - rollout_corr/kl:np.float64(0.0008689244228179405) - rollout_corr/k3_kl:np.float64(0.0006925447481762603) - rollout_corr/rollout_ppl:np.float64(1.8853362454101443) - rollout_corr/rollout_log_ppl:np.float64(0.6229387456551194) - rollout_corr/log_ppl_diff:np.float64(0.0008689251262694597) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016076937317848206) - rollout_corr/log_ppl_diff_max:np.float64(0.007483184337615967) - rollout_corr/log_ppl_diff_min:np.float64(-0.0051155686378479) - rollout_corr/ppl_ratio:np.float64(1.00087097636424) - rollout_corr/chi2_token:np.float64(0.001052779145538807) - rollout_corr/chi2_seq:np.float64(0.7514310674741864) - actor/pg_loss:np.float64(7.94560182839632e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002786977966025006) - actor/ppo_kl:np.float64(6.474458803040761e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.397162064909935) - perf/mfu/actor:np.float64(0.09482056304606831) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.62731170654297) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.37109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.37109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0025341168511658907 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0025341168511658907 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:465.71484375 - response_length/max:1517.0 - response_length/min:174.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:465.71484375 - response_length_non_aborted/max:1517.0 - response_length_non_aborted/min:174.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.0625 - prompt_length/max:612.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010670162737369537 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1883809324353933) - timing_s/agent_loop/generate_sequences/max:np.float64(11.769639683887362) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.302525144354149) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.769639683887362) - timing_s/a
(TaskRunner pid=2224264) 
Training Progress:  13%|█▎        | 13/100 [11:21<1:15:59, 52.41s/it]
(TaskRunner pid=2224264) step:14 - global_seqlen/min:21108 - global_seqlen/max:28388 - global_seqlen/minmax_diff:7280 - global_seqlen/balanced_min:24288 - global_seqlen/balanced_max:24367 - global_seqlen/mean:24304.0 - actor/entropy:0.5927061438560486 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2932337522506714 - training/rollout_probs_diff_mean:0.004692592658102512 - training/rollout_probs_diff_std:0.00925843883305788 - training/rollout_actor_probs_pearson_corr:0.9995315670967102 - rollout_corr/rollout_is_mean:0.9999940991401672 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.629692554473877 - rollout_corr/rollout_is_min:0.5129373669624329 - rollout_corr/rollout_is_std:0.03493344411253929 - rollout_corr/rollout_is_eff_sample_size:0.998781141930825 - rollout_corr/rollout_is_seq_mean:0.9999703168869019 - rollout_corr/rollout_is_seq_std:0.0018831780180335045 - rollout_corr/rollout_is_seq_max:1.007323145866394 - rollout_corr/rollout_is_seq_min:0.9952291250228882 - rollout_corr/rollout_is_seq_max_deviation:0.007323145866394043 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8519614487886429) - rollout_corr/training_log_ppl:np.float64(0.6055547567666508) - rollout_corr/kl:np.float64(0.0006922458704323731) - rollout_corr/k3_kl:np.float64(0.0006847217604217803) - rollout_corr/rollout_ppl:np.float64(1.850690824445337) - rollout_corr/rollout_log_ppl:np.float64(0.604862505744677) - rollout_corr/log_ppl_diff:np.float64(0.0006922510219737887) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016473830910399556) - rollout_corr/log_ppl_diff_max:np.float64(0.005706071853637695) - rollout_corr/log_ppl_diff_min:np.float64(-0.006285727024078369) - rollout_corr/ppl_ratio:np.float64(1.00069451588206) - rollout_corr/chi2_token:np.float64(0.0013829697854816914) - rollout_corr/chi2_seq:np.float64(1.670509943505749) - actor/pg_loss:np.float64(4.7211069613695145e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002791377244193427) - actor/ppo_kl:np.float64(2.9361409389139226e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.39511965960264206) - perf/mfu/actor:np.float64(0.09905766411058545) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.6309814453125) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.39453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.39453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001839753007516265 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.001839753007516265 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:492.0625 - response_length/max:1506.0 - response_length/min:184.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:492.0625 - response_length_non_aborted/max:1506.0 - response_length_non_aborted/min:184.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.4375 - prompt_length/max:666.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010376237332820892 - timing_s/agent_loop/generate_sequences/min:np.float64(2.323818039149046) - timing_s/agent_loop/generate_sequences/max:np.float64(11.8414571788162) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.66649279979174) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.8414571788162) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/
(TaskRunner pid=2224264) 
Training Progress:  14%|█▍        | 14/100 [11:56<1:07:37, 47.17s/it]
(TaskRunner pid=2224264) step:15 - global_seqlen/min:22401 - global_seqlen/max:28174 - global_seqlen/minmax_diff:5773 - global_seqlen/balanced_min:25422 - global_seqlen/balanced_max:25432 - global_seqlen/mean:25428.5 - actor/entropy:0.644464373588562 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21453610062599182 - training/rollout_probs_diff_mean:0.004607322160154581 - training/rollout_probs_diff_std:0.008982905186712742 - training/rollout_actor_probs_pearson_corr:0.9995591640472412 - rollout_corr/rollout_is_mean:0.9999948143959045 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.693491170357447e-06 - rollout_corr/rollout_is_max:1.5525251626968384 - rollout_corr/rollout_is_min:0.4232032895088196 - rollout_corr/rollout_is_std:0.0347040556371212 - rollout_corr/rollout_is_eff_sample_size:0.9987969583907628 - rollout_corr/rollout_is_seq_mean:1.000001072883606 - rollout_corr/rollout_is_seq_std:0.001659586327150464 - rollout_corr/rollout_is_seq_max:1.0077872276306152 - rollout_corr/rollout_is_seq_min:0.994013249874115 - rollout_corr/rollout_is_seq_max_deviation:0.007787227630615234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.001432800665498) - rollout_corr/training_log_ppl:np.float64(0.6742313308641315) - rollout_corr/kl:np.float64(0.0006003224210004987) - rollout_corr/k3_kl:np.float64(0.0006980515448731239) - rollout_corr/rollout_ppl:np.float64(2.0002612890675664) - rollout_corr/rollout_log_ppl:np.float64(0.673631009995006) - rollout_corr/log_ppl_diff:np.float64(0.0006003208691254258) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014080120017752051) - rollout_corr/log_ppl_diff_max:np.float64(0.00482177734375) - rollout_corr/log_ppl_diff_min:np.float64(-0.007164597511291504) - rollout_corr/ppl_ratio:np.float64(1.0006020197179168) - rollout_corr/chi2_token:np.float64(0.001624844502657652) - rollout_corr/chi2_seq:np.float64(1.5461109012831002) - actor/pg_loss:np.float64(0.00015092291869223118) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017774101524992147) - actor/ppo_kl:np.float64(-3.1671785084341764e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3353484757244587) - perf/mfu/actor:np.float64(0.10308922077230062) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.59129333496094) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006222110823728144 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006222110823728144 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:507.734375 - response_length/max:1102.0 - response_length/min:186.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:507.734375 - response_length_non_aborted/max:1102.0 - response_length_non_aborted/min:186.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.90625 - prompt_length/max:494.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000116739422082901 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3703849464654922) - timing_s/agent_loop/generate_sequences/max:np.float64(9.983016459271312) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.813961479689169) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.983016459271312) - timing_s/agent_loop/slowest/tool_calls:np.f
(TaskRunner pid=2224264) 
Training Progress:  15%|█▌        | 15/100 [12:29<1:00:58, 43.04s/it]
(TaskRunner pid=2224264) step:16 - global_seqlen/min:21410 - global_seqlen/max:32573 - global_seqlen/minmax_diff:11163 - global_seqlen/balanced_min:26094 - global_seqlen/balanced_max:26108 - global_seqlen/mean:26103.625 - actor/entropy:0.6026089787483215 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1841939091682434 - training/rollout_probs_diff_mean:0.004535277374088764 - training/rollout_probs_diff_std:0.008988428860902786 - training/rollout_actor_probs_pearson_corr:0.9995559453964233 - rollout_corr/rollout_is_mean:1.000097632408142 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6370495557785034 - rollout_corr/rollout_is_min:0.5366882681846619 - rollout_corr/rollout_is_std:0.034516338258981705 - rollout_corr/rollout_is_eff_sample_size:0.9988102779034601 - rollout_corr/rollout_is_seq_mean:1.0000861883163452 - rollout_corr/rollout_is_seq_std:0.00168060592841357 - rollout_corr/rollout_is_seq_max:1.0056310892105103 - rollout_corr/rollout_is_seq_min:0.9956023097038269 - rollout_corr/rollout_is_seq_max_deviation:0.005631089210510254 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8813290460966527) - rollout_corr/training_log_ppl:np.float64(0.6181044758995995) - rollout_corr/kl:np.float64(0.0006577574873949388) - rollout_corr/k3_kl:np.float64(0.0006780388332572329) - rollout_corr/rollout_ppl:np.float64(1.880119965877384) - rollout_corr/rollout_log_ppl:np.float64(0.617446719086729) - rollout_corr/log_ppl_diff:np.float64(0.0006577568128705025) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014887393917888403) - rollout_corr/log_ppl_diff_max:np.float64(0.009718984365463257) - rollout_corr/log_ppl_diff_min:np.float64(-0.005425512790679932) - rollout_corr/ppl_ratio:np.float64(1.0006597633473575) - rollout_corr/chi2_token:np.float64(0.0014098691754043102) - rollout_corr/chi2_seq:np.float64(2.303532388759777) - actor/pg_loss:np.float64(-0.00024759862571954727) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005523854347302404) - actor/ppo_kl:np.float64(0.00011259556639942048) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4002230614423752) - perf/mfu/actor:np.float64(0.10440002990848575) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.71824264526367) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.48828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004673639312386513 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004673639312386513 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:535.01953125 - response_length/max:1208.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:535.01953125 - response_length_non_aborted/max:1208.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.71875 - prompt_length/max:859.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.0385330319404602e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0918461363762617) - timing_s/agent_loop/generate_sequences/max:np.float64(10.83283576183021) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.100973397937196) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.83283576183021) - timing_s/agent_loop/slowest/tool_calls:np.flo
(TaskRunner pid=2224264) 
Training Progress:  16%|█▌        | 16/100 [13:04<56:35, 40.42s/it]  
(TaskRunner pid=2224264) step:17 - global_seqlen/min:25658 - global_seqlen/max:31580 - global_seqlen/minmax_diff:5922 - global_seqlen/balanced_min:28427 - global_seqlen/balanced_max:28619 - global_seqlen/mean:28480.125 - actor/entropy:0.6307497024536133 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24447917938232422 - training/rollout_probs_diff_mean:0.004340663086622953 - training/rollout_probs_diff_std:0.008799507282674313 - training/rollout_actor_probs_pearson_corr:0.9995622634887695 - rollout_corr/rollout_is_mean:1.0000449419021606 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.015722076175734e-06 - rollout_corr/rollout_is_max:1.5821566581726074 - rollout_corr/rollout_is_min:0.4949151575565338 - rollout_corr/rollout_is_std:0.03397679701447487 - rollout_corr/rollout_is_eff_sample_size:0.998847027329727 - rollout_corr/rollout_is_seq_mean:1.0000169277191162 - rollout_corr/rollout_is_seq_std:0.0016468465328216553 - rollout_corr/rollout_is_seq_max:1.006962537765503 - rollout_corr/rollout_is_seq_min:0.9948194622993469 - rollout_corr/rollout_is_seq_max_deviation:0.00696253776550293 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0382777974009514) - rollout_corr/training_log_ppl:np.float64(0.6885525062098168) - rollout_corr/kl:np.float64(0.0007926652352825414) - rollout_corr/k3_kl:np.float64(0.000665645356832556) - rollout_corr/rollout_ppl:np.float64(2.0366465840488672) - rollout_corr/rollout_log_ppl:np.float64(0.687759839813225) - rollout_corr/log_ppl_diff:np.float64(0.0007926663965918124) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014466078137047589) - rollout_corr/log_ppl_diff_max:np.float64(0.0072629451751708984) - rollout_corr/log_ppl_diff_min:np.float64(-0.004837632179260254) - rollout_corr/ppl_ratio:np.float64(1.0007944458629936) - rollout_corr/chi2_token:np.float64(0.0010911454446613789) - rollout_corr/chi2_seq:np.float64(0.8875894017983228) - actor/pg_loss:np.float64(4.744785837829113e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019340229709996493) - actor/ppo_kl:np.float64(0.00017584012368843105) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3084394074976444) - perf/mfu/actor:np.float64(0.1127631964826109) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.56375503540039) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.3984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005952840205281973 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005952840205281973 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:556.78515625 - response_length/max:1943.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:556.78515625 - response_length_non_aborted/max:1943.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:333.21875 - prompt_length/max:617.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012286566197872162 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8024302516132593) - timing_s/agent_loop/generate_sequences/max:np.float64(15.304179856553674) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.37867253353761) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.304179856553674) - timing_s/agent_loop/slowest/t
(TaskRunner pid=2224264) 
Training Progress:  17%|█▋        | 17/100 [13:42<55:10, 39.88s/it]
(TaskRunner pid=2224264) step:18 - global_seqlen/min:21346 - global_seqlen/max:28688 - global_seqlen/minmax_diff:7342 - global_seqlen/balanced_min:24408 - global_seqlen/balanced_max:24420 - global_seqlen/mean:24413.25 - actor/entropy:0.6875190734863281 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4119926691055298 - training/rollout_probs_diff_mean:0.0047507998533546925 - training/rollout_probs_diff_std:0.009137631393969059 - training/rollout_actor_probs_pearson_corr:0.9995434284210205 - rollout_corr/rollout_is_mean:0.9999079704284668 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.3933751435833983e-05 - rollout_corr/rollout_is_max:1.649957537651062 - rollout_corr/rollout_is_min:0.2244005799293518 - rollout_corr/rollout_is_std:0.03564453125 - rollout_corr/rollout_is_eff_sample_size:0.9987308417828792 - rollout_corr/rollout_is_seq_mean:0.9998347759246826 - rollout_corr/rollout_is_seq_std:0.0020229120273143053 - rollout_corr/rollout_is_seq_max:1.005630373954773 - rollout_corr/rollout_is_seq_min:0.9910240769386292 - rollout_corr/rollout_is_seq_max_deviation:0.00897592306137085 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0345947919413447) - rollout_corr/training_log_ppl:np.float64(0.6928353062248789) - rollout_corr/kl:np.float64(0.0008856774063996653) - rollout_corr/k3_kl:np.float64(0.0006838213192850162) - rollout_corr/rollout_ppl:np.float64(2.0328480429016054) - rollout_corr/rollout_log_ppl:np.float64(0.6919496294576675) - rollout_corr/log_ppl_diff:np.float64(0.0008856767672114074) - rollout_corr/log_ppl_abs_diff:np.float64(0.001704056456219405) - rollout_corr/log_ppl_diff_max:np.float64(0.00990024209022522) - rollout_corr/log_ppl_diff_min:np.float64(-0.004864394664764404) - rollout_corr/ppl_ratio:np.float64(1.0008882319089025) - rollout_corr/chi2_token:np.float64(0.0009777550585567951) - rollout_corr/chi2_seq:np.float64(0.8223780991975218) - actor/pg_loss:np.float64(0.00021487544290721416) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000207112968837464) - actor/ppo_kl:np.float64(5.649118667616637e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.38337405771017075) - perf/mfu/actor:np.float64(0.09893896312961671) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.62813949584961) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.4609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0014918704982846975 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0014918704982846975 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:489.6328125 - response_length/max:1187.0 - response_length/min:151.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:489.6328125 - response_length_non_aborted/max:1187.0 - response_length_non_aborted/min:151.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.28125 - prompt_length/max:540.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001486111432313919 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7897587660700083) - timing_s/agent_loop/generate_sequences/max:np.float64(10.173580570146441) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.444523438432952) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.173580570146441) - timing_s/agent_loop/slowest/tool_c
(TaskRunner pid=2224264) 
Training Progress:  18%|█▊        | 18/100 [14:16<51:58, 38.03s/it]
(TaskRunner pid=2224264) step:19 - global_seqlen/min:21987 - global_seqlen/max:30303 - global_seqlen/minmax_diff:8316 - global_seqlen/balanced_min:26121 - global_seqlen/balanced_max:26132 - global_seqlen/mean:26128.5 - actor/entropy:0.6984733939170837 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2413516640663147 - training/rollout_probs_diff_mean:0.004623448941856623 - training/rollout_probs_diff_std:0.008824565447866917 - training/rollout_actor_probs_pearson_corr:0.9995649456977844 - rollout_corr/rollout_is_mean:0.9998148083686829 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.593936970806681e-06 - rollout_corr/rollout_is_max:1.9665265083312988 - rollout_corr/rollout_is_min:0.4158303737640381 - rollout_corr/rollout_is_std:0.035356584936380386 - rollout_corr/rollout_is_eff_sample_size:0.9987510563634333 - rollout_corr/rollout_is_seq_mean:0.9998022317886353 - rollout_corr/rollout_is_seq_std:0.001719683175906539 - rollout_corr/rollout_is_seq_max:1.005003571510315 - rollout_corr/rollout_is_seq_min:0.9942381978034973 - rollout_corr/rollout_is_seq_max_deviation:0.0057618021965026855 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1011477317661047) - rollout_corr/training_log_ppl:np.float64(0.7186182939331047) - rollout_corr/kl:np.float64(0.000854842338526618) - rollout_corr/k3_kl:np.float64(0.0006763917490957283) - rollout_corr/rollout_ppl:np.float64(2.0992823550477624) - rollout_corr/rollout_log_ppl:np.float64(0.7177634529653005) - rollout_corr/log_ppl_diff:np.float64(0.0008548409678041935) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015009507769718766) - rollout_corr/log_ppl_diff_max:np.float64(0.0058748722076416016) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035433173179626465) - rollout_corr/ppl_ratio:np.float64(1.000856626080349) - rollout_corr/chi2_token:np.float64(0.0010158277582377195) - rollout_corr/chi2_seq:np.float64(0.722900487948209) - actor/pg_loss:np.float64(0.00023403007071465254) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021148656901459617) - actor/ppo_kl:np.float64(1.1429972420096135e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.30054471269249916) - perf/mfu/actor:np.float64(0.10550384895416316) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.6054801940918) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012871723156422377 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0012871723156422377 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:514.390625 - response_length/max:1276.0 - response_length/min:169.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:514.390625 - response_length_non_aborted/max:1276.0 - response_length_non_aborted/min:169.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.125 - prompt_length/max:836.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001385621726512909 - timing_s/agent_loop/generate_sequences/min:np.float64(1.989379409700632) - timing_s/agent_loop/generate_sequences/max:np.float64(10.74332383275032) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.9351765791507205) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.74332383275032) - timing_s/agent_loop/slowest/tool_
(TaskRunner pid=2224264) 
Training Progress:  19%|█▉        | 19/100 [14:50<49:38, 36.77s/it]
(TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20}
(TaskRunner pid=2224264) validation generation end
(TaskRunner pid=2224264) [prompt] system
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2224264) <reasoning>
(TaskRunner pid=2224264) ...
(TaskRunner pid=2224264) </reasoning>
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) ...
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) A
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) user
(TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2224264) Please reason step by step.
(TaskRunner pid=2224264) assistant
(TaskRunner pid=2224264) <think>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) </think>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) [response] <reasoning>
(TaskRunner pid=2224264) To determine which molecule is the only correct reactant used in the synthesis of the product **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"**, we need to analyze the structure and functional groups of the product and match it with the structure of the reactant.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Step 1: Analyze the product
(TaskRunner pid=2224264) The product is:
(TaskRunner pid=2224264) ```
(TaskRunner pid=2224264) Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2224264) ```
(TaskRunner pid=2224264) This can be interpreted as:
(TaskRunner pid=2224264) - A nitrogen atom (N) is attached to a ring.
(TaskRunner pid=2224264) - The ring is a pyrrole-like structure (N in the ring).
(TaskRunner pid=2224264) - The ring is substituted with a group: `Cc2ccc(Cl)cc2C(F)(F)F`, which is a benzene ring with a chlorine atom and a trifluoromethyl group.
(TaskRunner pid=2224264) - The structure is a pyrrole derivative with a substituent on the ring.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Step 2: Analyze the options
(TaskRunner pid=2224264) Let’s examine the options one by one to see which one matches the structure of the product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) #### Option A:
(TaskRunner pid=2224264) ```
(TaskRunner pid=2224264) O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2224264) ```
(TaskRunner pid=2224264) This structure contains:
(TaskRunner pid=2224264) - A carbonyl group (O=C), which is not in the product.
(TaskRunner pid=2224264) - Two carbonyl groups (C=O), which are not in the product.
(TaskRunner pid=2224264) - The part `c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1` matches the pyrrole-like structure of the product, but the presence of two carbonyl groups makes this a different compound.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) #### Option B:
(TaskRunner pid=2224264) ```
(TaskRunner pid=2224264) O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2224264) ```
(TaskRunner pid=2224264) This contains:
(TaskRunner pid=2224264) - A carbonyl group (O=C), which is not in the product.
(TaskRunner pid=2224264) - A chlorine atom (Cl) attached to the ring.
(TaskRunner pid=2224264) - The rest of the structure does not match the product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) #### Option C:
(TaskRunner pid=2224264) ```
(TaskRunner pid=2224264) O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2224264) ```
(TaskRunner pid=2224264) This contains:
(TaskRunner pid=2224264) - Two carbonyl groups (C=O), which are not in the product.
(TaskRunner pid=2224264) - A hydroxyl group (O), which is not in the product.
(TaskRunner pid=2224264) - The part `c2ccc(Cl)c(C(F)(F)F)c2` is similar to the product, but the overall structure is not matching the product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) #### Option D:
(TaskRunner pid=2224264) ```
(TaskRunner pid=2224264) O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2224264) ```
(TaskRunner pid=2224264) This contains:
(TaskRunner pid=2224264) - Two carbonyl groups (C=O), which are not in the product.
(TaskRunner pid=2224264) - A fluorinated group (C(F)(F)F), which is present in the product.
(TaskRunner pid=2224264) - The structure is not matching the product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Step 3: Conclusion
(TaskRunner pid=2224264) None of the options exactly match the structure of the product. However, **Option A** is the only one that contains the core structure of the product: `c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1`. The other groups in Option A (carbonyl groups) are not part of the product, but the rest of the structure is the closest match.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) Thus, **Option A** is the best candidate as the correct reactant.
(TaskRunner pid=2224264) </reasoning>
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) A
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) [ground_truth] A
(TaskRunner pid=2224264) [score] 1.0
(TaskRunner pid=2224264) [acc] 1.0
(TaskRunner pid=2224264) [pred] A
(TaskRunner pid=2224264) [incorrect_format] 1
(TaskRunner pid=2224264) [feedback] 
(TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20
(TaskRunner pid=2224264) step:20 - global_seqlen/min:22848 - global_seqlen/max:30898 - global_seqlen/minmax_diff:8050 - global_seqlen/balanced_min:26847 - global_seqlen/balanced_max:26876 - global_seqlen/mean:26867.0 - actor/entropy:0.6798826456069946 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967540204524994 - training/rollout_probs_diff_mean:0.004492460284382105 - training/rollout_probs_diff_std:0.008851698599755764 - training/rollout_actor_probs_pearson_corr:0.9995761513710022 - rollout_corr/rollout_is_mean:0.9998908638954163 - rollout_corr/rollout_is_ratio_fraction_high:1.4051259313418996e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.54838228225708 - rollout_corr/rollout_is_min:0.5467735528945923 - rollout_corr/rollout_is_std:0.034748680889606476 - rollout_corr/rollout_is_eff_sample_size:0.9987937474899505 - rollout_corr/rollout_is_seq_mean:0.9998890161514282 - rollout_corr/rollout_is_seq_std:0.0017418391071259975 - rollout_corr/rollout_is_seq_max:1.0073792934417725 - rollout_corr/rollout_is_seq_min:0.9951199293136597 - rollout_corr/rollout_is_seq_max_deviation:0.007379293441772461 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.033839966636151) - rollout_corr/training_log_ppl:np.float64(0.6936833750805818) - rollout_corr/kl:np.float64(0.0007494871477833343) - rollout_corr/k3_kl:np.float64(0.0006487431917321373) - rollout_corr/rollout_ppl:np.float64(2.0322980885393918) - rollout_corr/rollout_log_ppl:np.float64(0.6929338824702427) - rollout_corr/log_ppl_diff:np.float64(0.0007494926103390753) - rollout_corr/log_ppl_abs_diff:np.float64(0.001494129013735801) - rollout_corr/log_ppl_diff_max:np.float64(0.005956470966339111) - rollout_corr/log_ppl_diff_min:np.float64(-0.006281614303588867) - rollout_corr/ppl_ratio:np.float64(1.000751372659579) - rollout_corr/chi2_token:np.float64(0.0011060182005167007) - rollout_corr/chi2_seq:np.float64(0.7747026174329221) - actor/pg_loss:np.float64(-0.00010517542250454426) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014331985744320264) - actor/ppo_kl:np.float64(1.3654966693366077e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.31467055529356003) - perf/mfu/actor:np.float64(0.10809592356334449) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.8171272277832) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.48392857142857143) - val-aux/sciknoweval/reward/std@16:np.float64(0.3171121217029239) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6190095238095239) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.2728610239160176) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.3498904761904762) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2656159472576298) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.48431904761904765) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.3171144172517289) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.729252380952381) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.20041976709665926) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.24425714285714284) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1861647179153658) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.5091) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.2646122540406198) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8068619047619048) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.12410804929329204) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.17493809523809525) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.11508084398834663) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.5237238095238095) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.20562351058186512) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8522714285714
(TaskRunner pid=2224264) 
Training Progress:  20%|██        | 20/100 [18:19<1:58:06, 88.58s/it]
(TaskRunner pid=2224264) step:21 - global_seqlen/min:21817 - global_seqlen/max:33018 - global_seqlen/minmax_diff:11201 - global_seqlen/balanced_min:27703 - global_seqlen/balanced_max:27729 - global_seqlen/mean:27721.25 - actor/entropy:0.6702685952186584 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30163779854774475 - training/rollout_probs_diff_mean:0.004457509610801935 - training/rollout_probs_diff_std:0.00877729244530201 - training/rollout_actor_probs_pearson_corr:0.9996225833892822 - rollout_corr/rollout_is_mean:1.0001065731048584 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7989544868469238 - rollout_corr/rollout_is_min:0.6091647744178772 - rollout_corr/rollout_is_std:0.03465593233704567 - rollout_corr/rollout_is_eff_sample_size:0.9988006450060433 - rollout_corr/rollout_is_seq_mean:1.0000840425491333 - rollout_corr/rollout_is_seq_std:0.001599725685082376 - rollout_corr/rollout_is_seq_max:1.0058624744415283 - rollout_corr/rollout_is_seq_min:0.9920676350593567 - rollout_corr/rollout_is_seq_max_deviation:0.00793236494064331 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.009537471458316) - rollout_corr/training_log_ppl:np.float64(0.6828376608900726) - rollout_corr/kl:np.float64(0.000724996415337742) - rollout_corr/k3_kl:np.float64(0.0006654245569848172) - rollout_corr/rollout_ppl:np.float64(2.008096869569272) - rollout_corr/rollout_log_ppl:np.float64(0.6821126618888229) - rollout_corr/log_ppl_diff:np.float64(0.000724999001249671) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013606392312794924) - rollout_corr/log_ppl_diff_max:np.float64(0.009048700332641602) - rollout_corr/log_ppl_diff_min:np.float64(-0.004755973815917969) - rollout_corr/ppl_ratio:np.float64(1.0007265841122717) - rollout_corr/chi2_token:np.float64(0.001218994613736868) - rollout_corr/chi2_seq:np.float64(0.7947670572903007) - actor/pg_loss:np.float64(-0.00015212141443043947) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016183609022846213) - actor/ppo_kl:np.float64(0.00017459750991832834) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3394359387457371) - perf/mfu/actor:np.float64(0.10795117855645653) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.55794906616211) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003770750481635332 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003770750481635332 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:583.1015625 - response_length/max:1506.0 - response_length/min:126.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:583.1015625 - response_length_non_aborted/max:1506.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.1875 - prompt_length/max:669.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011318549513816833 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5702421497553587) - timing_s/agent_loop/generate_sequences/max:np.float64(12.702512258663774) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.579333163579577) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.702512258663774) - timing_s/agent_loop/slowest/tool_calls:np.floa
(TaskRunner pid=2224264) 
Training Progress:  21%|██        | 21/100 [18:55<1:35:47, 72.76s/it]
(TaskRunner pid=2224264) step:22 - global_seqlen/min:20941 - global_seqlen/max:26032 - global_seqlen/minmax_diff:5091 - global_seqlen/balanced_min:23791 - global_seqlen/balanced_max:27438 - global_seqlen/mean:24253.875 - actor/entropy:0.6328612565994263 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18033698201179504 - training/rollout_probs_diff_mean:0.004449577536433935 - training/rollout_probs_diff_std:0.00884386245161295 - training/rollout_actor_probs_pearson_corr:0.9996188879013062 - rollout_corr/rollout_is_mean:0.9999265074729919 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.831578016281128 - rollout_corr/rollout_is_min:0.5145671367645264 - rollout_corr/rollout_is_std:0.0340801440179348 - rollout_corr/rollout_is_eff_sample_size:0.9988397723743896 - rollout_corr/rollout_is_seq_mean:0.9999299049377441 - rollout_corr/rollout_is_seq_std:0.0016637897351756692 - rollout_corr/rollout_is_seq_max:1.00577974319458 - rollout_corr/rollout_is_seq_min:0.9933801293373108 - rollout_corr/rollout_is_seq_max_deviation:0.006619870662689209 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9823644189164042) - rollout_corr/training_log_ppl:np.float64(0.6632540230639279) - rollout_corr/kl:np.float64(0.0008493472823345449) - rollout_corr/k3_kl:np.float64(0.000727977444228145) - rollout_corr/rollout_ppl:np.float64(1.9806274939328432) - rollout_corr/rollout_log_ppl:np.float64(0.6624046702054329) - rollout_corr/log_ppl_diff:np.float64(0.0008493528584949672) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016184973646886647) - rollout_corr/log_ppl_diff_max:np.float64(0.007628798484802246) - rollout_corr/log_ppl_diff_min:np.float64(-0.00557255744934082) - rollout_corr/ppl_ratio:np.float64(1.000851531745866) - rollout_corr/chi2_token:np.float64(0.0012481804005801678) - rollout_corr/chi2_seq:np.float64(0.8115007842425257) - actor/pg_loss:np.float64(0.0003393040969967842) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005306760692747048) - actor/ppo_kl:np.float64(0.00014998488964579337) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.35529135540127754) - perf/mfu/actor:np.float64(0.09394398118409722) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.99365997314453) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.44921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01673314906656742 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01673314906656742 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:502.37109375 - response_length/max:4901.0 - response_length/min:131.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:502.37109375 - response_length_non_aborted/max:4901.0 - response_length_non_aborted/min:131.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.5625 - prompt_length/max:592.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013022124767303467 - timing_s/agent_loop/generate_sequences/min:np.float64(1.669778237119317) - timing_s/agent_loop/generate_sequences/max:np.float64(30.597046691924334) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.5186303148948355) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(30.597046691924334) - timing_s/agent_loop/slowest/tool_calls:np.float64(
(TaskRunner pid=2224264) 
Training Progress:  22%|██▏       | 22/100 [19:49<1:27:27, 67.28s/it]
(TaskRunner pid=2224264) step:23 - global_seqlen/min:20573 - global_seqlen/max:31570 - global_seqlen/minmax_diff:10997 - global_seqlen/balanced_min:28209 - global_seqlen/balanced_max:28226 - global_seqlen/mean:28218.75 - actor/entropy:0.6778538823127747 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7014397382736206 - training/rollout_probs_diff_mean:0.0045140404254198074 - training/rollout_probs_diff_std:0.008873715065419674 - training/rollout_actor_probs_pearson_corr:0.9996016025543213 - rollout_corr/rollout_is_mean:0.999861478805542 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.2932093341078144e-05 - rollout_corr/rollout_is_max:1.5868637561798096 - rollout_corr/rollout_is_min:0.09759196639060974 - rollout_corr/rollout_is_std:0.0343623049557209 - rollout_corr/rollout_is_eff_sample_size:0.9988203866993153 - rollout_corr/rollout_is_seq_mean:0.9999100565910339 - rollout_corr/rollout_is_seq_std:0.0015114216366782784 - rollout_corr/rollout_is_seq_max:1.004460334777832 - rollout_corr/rollout_is_seq_min:0.9952100515365601 - rollout_corr/rollout_is_seq_max_deviation:0.004789948463439941 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.013377800118178) - rollout_corr/training_log_ppl:np.float64(0.6853225293452851) - rollout_corr/kl:np.float64(0.0008045039580757418) - rollout_corr/k3_kl:np.float64(0.0007065492574156451) - rollout_corr/rollout_ppl:np.float64(2.0117388791404665) - rollout_corr/rollout_log_ppl:np.float64(0.684518019144889) - rollout_corr/log_ppl_diff:np.float64(0.0008045102003961802) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014397446066141129) - rollout_corr/log_ppl_diff_max:np.float64(0.007155239582061768) - rollout_corr/log_ppl_diff_min:np.float64(-0.004554927349090576) - rollout_corr/ppl_ratio:np.float64(1.0008063388522714) - rollout_corr/chi2_token:np.float64(0.0012174241710454226) - rollout_corr/chi2_seq:np.float64(1.7728085950948298) - actor/pg_loss:np.float64(0.0003834259696304798) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000311835868842536) - actor/ppo_kl:np.float64(9.214319616290823e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.32652221247553825) - perf/mfu/actor:np.float64(0.11250225764410283) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.84423065185547) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:0 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0047743674367666245 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0047743674367666245 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:604.1171875 - response_length/max:1391.0 - response_length/min:180.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:604.1171875 - response_length_non_aborted/max:1391.0 - response_length_non_aborted/min:180.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.71875 - prompt_length/max:766.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010936148464679718 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4273153580725193) - timing_s/agent_loop/generate_sequences/max:np.float64(12.947387943044305) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.094733837271633) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.947387943044305) - timing_s/agent_loop/slowest/to
(TaskRunner pid=2224264) 
Training Progress:  23%|██▎       | 23/100 [20:26<1:14:27, 58.02s/it]
(TaskRunner pid=2224264) step:24 - global_seqlen/min:25634 - global_seqlen/max:31704 - global_seqlen/minmax_diff:6070 - global_seqlen/balanced_min:29137 - global_seqlen/balanced_max:29209 - global_seqlen/mean:29156.5 - actor/entropy:0.712544858455658 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3460633456707001 - training/rollout_probs_diff_mean:0.004510445520281792 - training/rollout_probs_diff_std:0.008606033399701118 - training/rollout_actor_probs_pearson_corr:0.9996117949485779 - rollout_corr/rollout_is_mean:0.9999745488166809 - rollout_corr/rollout_is_ratio_fraction_high:6.192870841914555e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.238574168382911e-05 - rollout_corr/rollout_is_max:2.087355136871338 - rollout_corr/rollout_is_min:0.47689226269721985 - rollout_corr/rollout_is_std:0.03445065766572952 - rollout_corr/rollout_is_eff_sample_size:0.9988145592508434 - rollout_corr/rollout_is_seq_mean:0.999997615814209 - rollout_corr/rollout_is_seq_std:0.0016162468818947673 - rollout_corr/rollout_is_seq_max:1.0054336786270142 - rollout_corr/rollout_is_seq_min:0.9961015582084656 - rollout_corr/rollout_is_seq_max_deviation:0.00543367862701416 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.082198773045093) - rollout_corr/training_log_ppl:np.float64(0.7211795629700646) - rollout_corr/kl:np.float64(0.0007120691560231762) - rollout_corr/k3_kl:np.float64(0.0006404822171930391) - rollout_corr/rollout_ppl:np.float64(2.0806773733347654) - rollout_corr/rollout_log_ppl:np.float64(0.7204674915410578) - rollout_corr/log_ppl_diff:np.float64(0.0007120714290067554) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014570994535461068) - rollout_corr/log_ppl_diff_max:np.float64(0.005695939064025879) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035497546195983887) - rollout_corr/ppl_ratio:np.float64(1.000713731860742) - rollout_corr/chi2_token:np.float64(0.0011395812034606934) - rollout_corr/chi2_seq:np.float64(1.328517090762034) - actor/pg_loss:np.float64(-4.311627708375454e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001907837179260241) - actor/ppo_kl:np.float64(0.0001019216878037632) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3508700765669346) - perf/mfu/actor:np.float64(0.11424736196001828) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.2998194694519) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:0 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002073063515126705 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002073063515126705 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:630.765625 - response_length/max:1901.0 - response_length/min:207.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:630.765625 - response_length_non_aborted/max:1901.0 - response_length_non_aborted/min:207.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.375 - prompt_length/max:848.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011662021279335022 - timing_s/agent_loop/generate_sequences/min:np.float64(2.381055638194084) - timing_s/agent_loop/generate_sequences/max:np.float64(15.576675411313772) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.741719769590418) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.576675411313772) - timing_s/agent_loop/
(TaskRunner pid=2224264) 
Training Progress:  24%|██▍       | 24/100 [21:06<1:06:38, 52.62s/it]
(TaskRunner pid=2224264) step:25 - global_seqlen/min:23500 - global_seqlen/max:34274 - global_seqlen/minmax_diff:10774 - global_seqlen/balanced_min:27954 - global_seqlen/balanced_max:27980 - global_seqlen/mean:27971.25 - actor/entropy:0.7009389400482178 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18294647336006165 - training/rollout_probs_diff_mean:0.004573621321469545 - training/rollout_probs_diff_std:0.008723455481231213 - training/rollout_actor_probs_pearson_corr:0.999584436416626 - rollout_corr/rollout_is_mean:0.9999302625656128 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.836859029135667e-06 - rollout_corr/rollout_is_max:1.4950411319732666 - rollout_corr/rollout_is_min:0.4878106117248535 - rollout_corr/rollout_is_std:0.03484973683953285 - rollout_corr/rollout_is_eff_sample_size:0.9987866122287246 - rollout_corr/rollout_is_seq_mean:0.9999464154243469 - rollout_corr/rollout_is_seq_std:0.001561047974973917 - rollout_corr/rollout_is_seq_max:1.004056453704834 - rollout_corr/rollout_is_seq_min:0.9949167370796204 - rollout_corr/rollout_is_seq_max_deviation:0.005083262920379639 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1113617275841534) - rollout_corr/training_log_ppl:np.float64(0.7249491217080504) - rollout_corr/kl:np.float64(0.0006290591016089309) - rollout_corr/k3_kl:np.float64(0.0006558882103604446) - rollout_corr/rollout_ppl:np.float64(2.1101012001745403) - rollout_corr/rollout_log_ppl:np.float64(0.7243200563825667) - rollout_corr/log_ppl_diff:np.float64(0.0006290653254836798) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012821299023926258) - rollout_corr/log_ppl_diff_max:np.float64(0.005804777145385742) - rollout_corr/log_ppl_diff_min:np.float64(-0.004645586013793945) - rollout_corr/ppl_ratio:np.float64(1.0006304802373052) - rollout_corr/chi2_token:np.float64(0.00137246772646904) - rollout_corr/chi2_seq:np.float64(0.4346559832338244) - actor/pg_loss:np.float64(-0.00020399875938892365) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024524242849111033) - actor/ppo_kl:np.float64(-6.53956497700392e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.31579897925257683) - perf/mfu/actor:np.float64(0.10990411377383731) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(108.17280960083008) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:0 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002174121094867587 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002174121094867587 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:571.3515625 - response_length/max:1596.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:571.3515625 - response_length_non_aborted/max:1596.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.75 - prompt_length/max:646.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010717101395130157 - timing_s/agent_loop/generate_sequences/min:np.float64(3.3154151998460293) - timing_s/agent_loop/generate_sequences/max:np.float64(15.293862147256732) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.137247967235453) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.293862147256732) - timing_s/agent_loop/slowest/tool_calls:np
(TaskRunner pid=2224264) 
Training Progress:  25%|██▌       | 25/100 [21:45<1:00:38, 48.52s/it]
(TaskRunner pid=2224264) step:26 - global_seqlen/min:22426 - global_seqlen/max:29951 - global_seqlen/minmax_diff:7525 - global_seqlen/balanced_min:26952 - global_seqlen/balanced_max:26964 - global_seqlen/mean:26960.375 - actor/entropy:0.7369614243507385 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31862038373947144 - training/rollout_probs_diff_mean:0.004758655559271574 - training/rollout_probs_diff_std:0.00890500657260418 - training/rollout_actor_probs_pearson_corr:0.9995867609977722 - rollout_corr/rollout_is_mean:0.9999259114265442 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4305845070339274e-05 - rollout_corr/rollout_is_max:1.8682441711425781 - rollout_corr/rollout_is_min:0.27226293087005615 - rollout_corr/rollout_is_std:0.03548027575016022 - rollout_corr/rollout_is_eff_sample_size:0.9987426137038081 - rollout_corr/rollout_is_seq_mean:0.9999041557312012 - rollout_corr/rollout_is_seq_std:0.001717878389172256 - rollout_corr/rollout_is_seq_max:1.005202293395996 - rollout_corr/rollout_is_seq_min:0.9952166080474854 - rollout_corr/rollout_is_seq_max_deviation:0.005202293395996094 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1379281943663955) - rollout_corr/training_log_ppl:np.float64(0.7447412704350427) - rollout_corr/kl:np.float64(0.0007289938001822094) - rollout_corr/k3_kl:np.float64(0.0007074658968804215) - rollout_corr/rollout_ppl:np.float64(2.136324817314744) - rollout_corr/rollout_log_ppl:np.float64(0.7440122718689963) - rollout_corr/log_ppl_diff:np.float64(0.0007289985660463572) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015073230024427176) - rollout_corr/log_ppl_diff_max:np.float64(0.005815982818603516) - rollout_corr/log_ppl_diff_min:np.float64(-0.004653453826904297) - rollout_corr/ppl_ratio:np.float64(1.00073087005876) - rollout_corr/chi2_token:np.float64(0.0013713838998228312) - rollout_corr/chi2_seq:np.float64(0.9597642624285072) - actor/pg_loss:np.float64(0.00014449155423790216) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012465850477383356) - actor/ppo_kl:np.float64(-1.70074631995476e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2908688709139824) - perf/mfu/actor:np.float64(0.105823819821739) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.64182186126709) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:0 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0029881333466619253 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0029881333466619253 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:546.10546875 - response_length/max:1208.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:546.10546875 - response_length_non_aborted/max:1208.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.40625 - prompt_length/max:709.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015214644372463226 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7672749813646078) - timing_s/agent_loop/generate_sequences/max:np.float64(10.418701702728868) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.990022340542055) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.418701702728868) - timing_s/agent_loop/slowe
(TaskRunner pid=2224264) 
Training Progress:  26%|██▌       | 26/100 [22:20<54:56, 44.54s/it]  
(TaskRunner pid=2224264) step:27 - global_seqlen/min:23952 - global_seqlen/max:34841 - global_seqlen/minmax_diff:10889 - global_seqlen/balanced_min:28921 - global_seqlen/balanced_max:28949 - global_seqlen/mean:28940.875 - actor/entropy:0.701714038848877 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17479756474494934 - training/rollout_probs_diff_mean:0.004477211274206638 - training/rollout_probs_diff_std:0.008556351996958256 - training/rollout_actor_probs_pearson_corr:0.9996283054351807 - rollout_corr/rollout_is_mean:0.9999887943267822 - rollout_corr/rollout_is_ratio_fraction_high:6.0401425798772834e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.0401425798772834e-06 - rollout_corr/rollout_is_max:2.401174545288086 - rollout_corr/rollout_is_min:0.40530839562416077 - rollout_corr/rollout_is_std:0.034324124455451965 - rollout_corr/rollout_is_eff_sample_size:0.9988231220557214 - rollout_corr/rollout_is_seq_mean:0.9999889135360718 - rollout_corr/rollout_is_seq_std:0.0014754357980564237 - rollout_corr/rollout_is_seq_max:1.0065122842788696 - rollout_corr/rollout_is_seq_min:0.9951512813568115 - rollout_corr/rollout_is_seq_max_deviation:0.006512284278869629 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.07450102083385) - rollout_corr/training_log_ppl:np.float64(0.7144390830071643) - rollout_corr/kl:np.float64(0.0007389031823423764) - rollout_corr/k3_kl:np.float64(0.0006497853174778356) - rollout_corr/rollout_ppl:np.float64(2.0729061695747077) - rollout_corr/rollout_log_ppl:np.float64(0.713700178428553) - rollout_corr/log_ppl_diff:np.float64(0.0007389045786112547) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013226103037595749) - rollout_corr/log_ppl_diff_max:np.float64(0.0057825446128845215) - rollout_corr/log_ppl_diff_min:np.float64(-0.005777716636657715) - rollout_corr/ppl_ratio:np.float64(1.0007404030766338) - rollout_corr/chi2_token:np.float64(0.001133262412622571) - rollout_corr/chi2_seq:np.float64(0.8787905650679022) - actor/pg_loss:np.float64(-2.4800654500722885e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022700108638673555) - actor/ppo_kl:np.float64(0.00011755238243083) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.32230544835329056) - perf/mfu/actor:np.float64(0.11335348091929753) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.77265930175781) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006488623563200235 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006488623563200235 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:646.71484375 - response_length/max:1758.0 - response_length/min:222.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:646.71484375 - response_length_non_aborted/max:1758.0 - response_length_non_aborted/min:222.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:257.6875 - prompt_length/max:380.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.4581090807914734e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.452285857871175) - timing_s/agent_loop/generate_sequences/max:np.float64(14.636356772854924) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.195729885730543) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.636356772854924) - timing_
(TaskRunner pid=2224264) 
Training Progress:  27%|██▋       | 27/100 [22:58<51:55, 42.68s/it]
(TaskRunner pid=2224264) step:28 - global_seqlen/min:25216 - global_seqlen/max:34360 - global_seqlen/minmax_diff:9144 - global_seqlen/balanced_min:29787 - global_seqlen/balanced_max:30195 - global_seqlen/mean:29843.75 - actor/entropy:0.7055036425590515 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1455976366996765 - training/rollout_probs_diff_mean:0.004539164714515209 - training/rollout_probs_diff_std:0.008678900077939034 - training/rollout_actor_probs_pearson_corr:0.9996199011802673 - rollout_corr/rollout_is_mean:1.0000463724136353 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.916669579164591e-06 - rollout_corr/rollout_is_max:1.6682544946670532 - rollout_corr/rollout_is_min:0.4844903349876404 - rollout_corr/rollout_is_std:0.0349300317466259 - rollout_corr/rollout_is_eff_sample_size:0.9987814986875572 - rollout_corr/rollout_is_seq_mean:1.0001145601272583 - rollout_corr/rollout_is_seq_std:0.0015147364465519786 - rollout_corr/rollout_is_seq_max:1.0062413215637207 - rollout_corr/rollout_is_seq_min:0.9959177374839783 - rollout_corr/rollout_is_seq_max_deviation:0.006241321563720703 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.081616838928312) - rollout_corr/training_log_ppl:np.float64(0.716681465273723) - rollout_corr/kl:np.float64(0.0007082077879232429) - rollout_corr/k3_kl:np.float64(0.0006868325160667155) - rollout_corr/rollout_ppl:np.float64(2.0801399582996964) - rollout_corr/rollout_log_ppl:np.float64(0.7159732520813122) - rollout_corr/log_ppl_diff:np.float64(0.0007082131924107671) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013121537631377578) - rollout_corr/log_ppl_diff_max:np.float64(0.00504302978515625) - rollout_corr/log_ppl_diff_min:np.float64(-0.0041460394859313965) - rollout_corr/ppl_ratio:np.float64(1.0007096340414137) - rollout_corr/chi2_token:np.float64(0.0013347528874874115) - rollout_corr/chi2_seq:np.float64(4.891642920905724) - actor/pg_loss:np.float64(-6.637291517108679e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002813626040278905) - actor/ppo_kl:np.float64(0.00019132641280006357) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.33150169998407364) - perf/mfu/actor:np.float64(0.11585717187378697) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.45846176147461) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:0 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007486805785447359 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007486805785447359 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:660.2109375 - response_length/max:2519.0 - response_length/min:204.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:660.2109375 - response_length_non_aborted/max:2519.0 - response_length_non_aborted/min:204.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.40625 - prompt_length/max:443.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011588819324970245 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6954549103975296) - timing_s/agent_loop/generate_sequences/max:np.float64(18.969659883528948) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.485852991718275) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.969659883528948) - timing_s/agent_loop/slowest/to
(TaskRunner pid=2224264) 
Training Progress:  28%|██▊       | 28/100 [23:41<51:12, 42.67s/it]
(TaskRunner pid=2224264) step:29 - global_seqlen/min:26637 - global_seqlen/max:40228 - global_seqlen/minmax_diff:13591 - global_seqlen/balanced_min:33532 - global_seqlen/balanced_max:33558 - global_seqlen/mean:33551.125 - actor/entropy:0.6751730442047119 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30162957310676575 - training/rollout_probs_diff_mean:0.004225059412419796 - training/rollout_probs_diff_std:0.008393223397433758 - training/rollout_actor_probs_pearson_corr:0.9996880292892456 - rollout_corr/rollout_is_mean:0.9999781847000122 - rollout_corr/rollout_is_ratio_fraction_high:5.061727733846055e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.012345546769211e-05 - rollout_corr/rollout_is_max:2.110851764678955 - rollout_corr/rollout_is_min:0.42815104126930237 - rollout_corr/rollout_is_std:0.033269479870796204 - rollout_corr/rollout_is_eff_sample_size:0.9988943655271626 - rollout_corr/rollout_is_seq_mean:1.0000176429748535 - rollout_corr/rollout_is_seq_std:0.001319664646871388 - rollout_corr/rollout_is_seq_max:1.0051316022872925 - rollout_corr/rollout_is_seq_min:0.9961720108985901 - rollout_corr/rollout_is_seq_max_deviation:0.0051316022872924805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.010434144642204) - rollout_corr/training_log_ppl:np.float64(0.6835667735431343) - rollout_corr/kl:np.float64(0.0005298591240814332) - rollout_corr/k3_kl:np.float64(0.0006146891930143283) - rollout_corr/rollout_ppl:np.float64(2.009353395085782) - rollout_corr/rollout_log_ppl:np.float64(0.6830369132512715) - rollout_corr/log_ppl_diff:np.float64(0.0005298602918628603) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011505558213684708) - rollout_corr/log_ppl_diff_max:np.float64(0.0041373372077941895) - rollout_corr/log_ppl_diff_min:np.float64(-0.005634099245071411) - rollout_corr/ppl_ratio:np.float64(1.0005309600383043) - rollout_corr/chi2_token:np.float64(0.0014236378483474255) - rollout_corr/chi2_seq:np.float64(0.9517913018353283) - actor/pg_loss:np.float64(7.968046702444553e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001298733541261754) - actor/ppo_kl:np.float64(-2.9945098688699545e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21987130120396614) - perf/mfu/actor:np.float64(0.11725733884439407) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(109.95436000823975) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:0 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00392979895696044 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00392979895696044 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:771.72265625 - response_length/max:1889.0 - response_length/min:208.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:771.72265625 - response_length_non_aborted/max:1889.0 - response_length_non_aborted/min:208.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.75 - prompt_length/max:492.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.993037045001984e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6769527215510607) - timing_s/agent_loop/generate_sequences/max:np.float64(16.69622828066349) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.844669617952604) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.69622828066349) - timing_
(TaskRunner pid=2224264) 
Training Progress:  29%|██▉       | 29/100 [24:24<50:44, 42.88s/it]
(TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30}
(TaskRunner pid=2224264) validation generation end
(TaskRunner pid=2224264) [prompt] system
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2224264) <reasoning>
(TaskRunner pid=2224264) ...
(TaskRunner pid=2224264) </reasoning>
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) ...
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) A
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) user
(TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2224264) Please reason step by step.
(TaskRunner pid=2224264) assistant
(TaskRunner pid=2224264) <think>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) </think>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) [response] <reasoning>
(TaskRunner pid=2224264) To determine which of the given options is the correct reactant for the synthesis of the product **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"**, we need to analyze the structure and functional groups involved.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) The target product is:
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) **Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This can be interpreted as:
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) - A **nitrile group (–CN)** attached to a ring (likely a pyrrole or similar heterocycle).
(TaskRunner pid=2224264) - The ring has a **chloro group (–Cl)** on a carbon and a **trifluoromethyl group (–CF₃)** on another carbon.
(TaskRunner pid=2224264) - The structure suggests that the molecule is a **pyrrole derivative** with a **chloro-substituted benzene ring** and a **trifluoromethyl group** attached to the same ring.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) Now, let's examine the options one by one to see which one could be a **reactant** that leads to this product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Option A:
(TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This is a **complex molecule**. It contains:
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) - A **carbonyl group (C=O)** and a **nitrile group (–CN)**.
(TaskRunner pid=2224264) - A **benzene ring** with a **chloro group (–Cl)** and a **trifluoromethyl group (–CF₃)**.
(TaskRunner pid=2224264) - A **pyrrole-like ring** (as suggested by the nitrile group and the ring structure).
(TaskRunner pid=2224264) - The **nitrile group** is attached to the ring in a way that matches the target product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This option **matches the structure** of the target molecule. It is **likely the correct reactant** that could be used in the synthesis of the product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Option B:
(TaskRunner pid=2224264) **O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This structure contains:
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) - A **carbonyl group (C=O)**.
(TaskRunner pid=2224264) - A **chloro group (–Cl)**.
(TaskRunner pid=2224264) - A **pyrrole-like ring**.
(TaskRunner pid=2224264) - However, the **trifluoromethyl group (–CF₃)** is **missing**.
(TaskRunner pid=2224264) - The **nitrile group (–CN)** is **not clearly present**.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This molecule is **not a good match** for the target product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Option C:
(TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This structure contains:
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) - A **carbonyl group (C=O)**.
(TaskRunner pid=2224264) - A **trifluoromethyl group (–CF₃)**.
(TaskRunner pid=2224264) - A **benzene ring** with a **chloro group (–Cl)**.
(TaskRunner pid=2224264) - A **pyrrole-like ring**.
(TaskRunner pid=2224264) - However, the **nitrile group (–CN)** is **missing**.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This molecule is **not a good match** for the target product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Option D:
(TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This structure contains:
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) - A **carbonyl group (C=O)**.
(TaskRunner pid=2224264) - A **trifluoromethyl group (–CF₃)**.
(TaskRunner pid=2224264) - A **benzene ring**.
(TaskRunner pid=2224264) - A **pyrrole-like ring**.
(TaskRunner pid=2224264) - However, the **nitrile group (–CN)** is **missing**.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) Again, this molecule is **not a good match** for the target product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Final Analysis:
(TaskRunner pid=2224264) Only **Option A** contains all the necessary functional groups and structure to be a **valid reactant** for the synthesis of the target molecule **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"**.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) </reasoning>
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) A
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) [ground_truth] A
(TaskRunner pid=2224264) [score] 1.0
(TaskRunner pid=2224264) [acc] 1.0
(TaskRunner pid=2224264) [pred] A
(TaskRunner pid=2224264) [incorrect_format] 1
(TaskRunner pid=2224264) [feedback] 
(TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30
(TaskRunner pid=2224264) step:30 - global_seqlen/min:28624 - global_seqlen/max:34380 - global_seqlen/minmax_diff:5756 - global_seqlen/balanced_min:31823 - global_seqlen/balanced_max:31938 - global_seqlen/mean:31860.75 - actor/entropy:0.7115141153335571 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5642241835594177 - training/rollout_probs_diff_mean:0.004424138925969601 - training/rollout_probs_diff_std:0.008687857538461685 - training/rollout_actor_probs_pearson_corr:0.999598503112793 - rollout_corr/rollout_is_mean:0.9999087452888489 - rollout_corr/rollout_is_ratio_fraction_high:5.690936632163357e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.1381873264326714e-05 - rollout_corr/rollout_is_max:2.0644965171813965 - rollout_corr/rollout_is_min:0.22554616630077362 - rollout_corr/rollout_is_std:0.034556034952402115 - rollout_corr/rollout_is_eff_sample_size:0.9988071858420651 - rollout_corr/rollout_is_seq_mean:0.9999098777770996 - rollout_corr/rollout_is_seq_std:0.0014933171914890409 - rollout_corr/rollout_is_seq_max:1.0051254034042358 - rollout_corr/rollout_is_seq_min:0.9926964044570923 - rollout_corr/rollout_is_seq_max_deviation:0.007303595542907715 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1091170106083155) - rollout_corr/training_log_ppl:np.float64(0.7282795129576698) - rollout_corr/kl:np.float64(0.0007562485593410884) - rollout_corr/k3_kl:np.float64(0.0006585642515801737) - rollout_corr/rollout_ppl:np.float64(2.107493511401117) - rollout_corr/rollout_log_ppl:np.float64(0.7275232608662918) - rollout_corr/log_ppl_diff:np.float64(0.0007562520913779736) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013768195640295744) - rollout_corr/log_ppl_diff_max:np.float64(0.005323231220245361) - rollout_corr/log_ppl_diff_min:np.float64(-0.0032753348350524902) - rollout_corr/ppl_ratio:np.float64(1.0007577680516988) - rollout_corr/chi2_token:np.float64(0.0011402126401662827) - rollout_corr/chi2_seq:np.float64(0.9514149494934827) - actor/pg_loss:np.float64(-0.00013444386422634125) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002210548966559145) - actor/ppo_kl:np.float64(4.4027712672978225e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2887660190463066) - perf/mfu/actor:np.float64(0.12035666503727542) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(107.33295822143555) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.549404761904762) - val-aux/sciknoweval/reward/std@16:np.float64(0.31608937376624846) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6822666666666667) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.25471779568655717) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.41630476190476196) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.28182671788474917) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5503333333333333) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.3158685881058942) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7810047619047619) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1712884300404119) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.30183333333333334) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.21622513075595678) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.587) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.2597657274415112) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8442666666666666) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.10407977320648662) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.21562857142857142) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.14281142327250002) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.609352380952381) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1968004873596571) - val-aux/sciknoweval/reward/best@16/mean:np.f
(TaskRunner pid=2224264) 
Training Progress:  30%|███       | 30/100 [28:27<2:00:03, 102.91s/it]
(TaskRunner pid=2224264) step:31 - global_seqlen/min:30423 - global_seqlen/max:35721 - global_seqlen/minmax_diff:5298 - global_seqlen/balanced_min:33243 - global_seqlen/balanced_max:33352 - global_seqlen/mean:33261.875 - actor/entropy:0.7014114856719971 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39983245730400085 - training/rollout_probs_diff_mean:0.004444092512130737 - training/rollout_probs_diff_std:0.008645395748317242 - training/rollout_actor_probs_pearson_corr:0.9995979070663452 - rollout_corr/rollout_is_mean:1.0000203847885132 - rollout_corr/rollout_is_ratio_fraction_high:5.1425781748548616e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.1425781748548616e-06 - rollout_corr/rollout_is_max:3.193486452102661 - rollout_corr/rollout_is_min:0.4723663926124573 - rollout_corr/rollout_is_std:0.03425285220146179 - rollout_corr/rollout_is_eff_sample_size:0.998828117093035 - rollout_corr/rollout_is_seq_mean:1.0000666379928589 - rollout_corr/rollout_is_seq_std:0.0014698212035000324 - rollout_corr/rollout_is_seq_max:1.0053932666778564 - rollout_corr/rollout_is_seq_min:0.9955719709396362 - rollout_corr/rollout_is_seq_max_deviation:0.005393266677856445 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.08949265582487) - rollout_corr/training_log_ppl:np.float64(0.7218374547082931) - rollout_corr/kl:np.float64(0.0005589085460151466) - rollout_corr/k3_kl:np.float64(0.0006561674455269895) - rollout_corr/rollout_ppl:np.float64(2.088360784109682) - rollout_corr/rollout_log_ppl:np.float64(0.7212785450974479) - rollout_corr/log_ppl_diff:np.float64(0.0005589096108451486) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012190231354907155) - rollout_corr/log_ppl_diff_max:np.float64(0.004900336265563965) - rollout_corr/log_ppl_diff_min:np.float64(-0.0036498308181762695) - rollout_corr/ppl_ratio:np.float64(1.0005601237062365) - rollout_corr/chi2_token:np.float64(0.0015174625441432) - rollout_corr/chi2_seq:np.float64(2.2517903256230056) - actor/pg_loss:np.float64(-9.307335130870342e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020832591962971492) - actor/ppo_kl:np.float64(1.1285746243316908e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2606422230601311) - perf/mfu/actor:np.float64(0.12484098624463014) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(113.83588361740112) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:0 - critic/score/mean:0.51171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009717543609440327 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009717543609440327 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:759.58984375 - response_length/max:2514.0 - response_length/min:234.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:759.58984375 - response_length_non_aborted/max:2514.0 - response_length_non_aborted/min:234.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.84375 - prompt_length/max:635.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014492124319076538 - timing_s/agent_loop/generate_sequences/min:np.float64(3.011936265975237) - timing_s/agent_loop/generate_sequences/max:np.float64(20.60571327060461) - timing_s/agent_loop/generate_sequences/mean:np.float64(10.018827380772564) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.60571327060461) - timing_
(TaskRunner pid=2224264) 
Training Progress:  31%|███       | 31/100 [29:13<1:38:34, 85.71s/it] 
(TaskRunner pid=2224264) step:32 - global_seqlen/min:25482 - global_seqlen/max:38028 - global_seqlen/minmax_diff:12546 - global_seqlen/balanced_min:31743 - global_seqlen/balanced_max:31780 - global_seqlen/mean:31769.5 - actor/entropy:0.6990305185317993 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2773066759109497 - training/rollout_probs_diff_mean:0.004490962252020836 - training/rollout_probs_diff_std:0.00860312208533287 - training/rollout_actor_probs_pearson_corr:0.9996287226676941 - rollout_corr/rollout_is_mean:1.0000104904174805 - rollout_corr/rollout_is_ratio_fraction_high:1.0702741747081745e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.417339324951172 - rollout_corr/rollout_is_min:0.6783297061920166 - rollout_corr/rollout_is_std:0.03459395840764046 - rollout_corr/rollout_is_eff_sample_size:0.9988045694973763 - rollout_corr/rollout_is_seq_mean:0.9999669790267944 - rollout_corr/rollout_is_seq_std:0.0015490534715354443 - rollout_corr/rollout_is_seq_max:1.004462480545044 - rollout_corr/rollout_is_seq_min:0.9947811365127563 - rollout_corr/rollout_is_seq_max_deviation:0.005218863487243652 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0522580044344068) - rollout_corr/training_log_ppl:np.float64(0.7055484902230091) - rollout_corr/kl:np.float64(0.0007327849301645983) - rollout_corr/k3_kl:np.float64(0.0006532650920689775) - rollout_corr/rollout_ppl:np.float64(2.0508041838184) - rollout_corr/rollout_log_ppl:np.float64(0.7048157050157897) - rollout_corr/log_ppl_diff:np.float64(0.0007327852072194219) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013509184354916215) - rollout_corr/log_ppl_diff_max:np.float64(0.006094217300415039) - rollout_corr/log_ppl_diff_min:np.float64(-0.0032320618629455566) - rollout_corr/ppl_ratio:np.float64(1.0007343096658587) - rollout_corr/chi2_token:np.float64(0.0011649802327156067) - rollout_corr/chi2_seq:np.float64(0.8535998552106321) - actor/pg_loss:np.float64(-7.09133455529809e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00026227998409922293) - actor/ppo_kl:np.float64(7.257984136543882e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.28863443806767464) - perf/mfu/actor:np.float64(0.12142939169953111) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(106.63142013549805) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:0 - critic/score/mean:0.45703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.45703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010431160219013691 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010431160219013691 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:729.953125 - response_length/max:1851.0 - response_length/min:205.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:729.953125 - response_length_non_aborted/max:1851.0 - response_length_non_aborted/min:205.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.84375 - prompt_length/max:404.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010944157838821411 - timing_s/agent_loop/generate_sequences/min:np.float64(2.5309949908405542) - timing_s/agent_loop/generate_sequences/max:np.float64(15.264747710898519) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.378220335514925) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.264747710898519) - timing_s/agent_loop/slowest/t
(TaskRunner pid=2224264) 
Training Progress:  32%|███▏      | 32/100 [29:53<1:21:31, 71.93s/it]
(TaskRunner pid=2224264) step:33 - global_seqlen/min:26534 - global_seqlen/max:38525 - global_seqlen/minmax_diff:11991 - global_seqlen/balanced_min:31307 - global_seqlen/balanced_max:31349 - global_seqlen/mean:31335.875 - actor/entropy:0.7530014514923096 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40599530935287476 - training/rollout_probs_diff_mean:0.004578310996294022 - training/rollout_probs_diff_std:0.00858127698302269 - training/rollout_actor_probs_pearson_corr:0.9995956420898438 - rollout_corr/rollout_is_mean:0.9998608231544495 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.69842768527451e-06 - rollout_corr/rollout_is_max:1.6179550886154175 - rollout_corr/rollout_is_min:0.4878464341163635 - rollout_corr/rollout_is_std:0.03484460711479187 - rollout_corr/rollout_is_eff_sample_size:0.9987870879096347 - rollout_corr/rollout_is_seq_mean:0.9998835921287537 - rollout_corr/rollout_is_seq_std:0.0015449043130502105 - rollout_corr/rollout_is_seq_max:1.003616213798523 - rollout_corr/rollout_is_seq_min:0.9941043257713318 - rollout_corr/rollout_is_seq_max_deviation:0.005895674228668213 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1979672331362963) - rollout_corr/training_log_ppl:np.float64(0.7705774857313372) - rollout_corr/kl:np.float64(0.0007750543829923018) - rollout_corr/k3_kl:np.float64(0.0006546921346171075) - rollout_corr/rollout_ppl:np.float64(2.1962045463733375) - rollout_corr/rollout_log_ppl:np.float64(0.7698024294222705) - rollout_corr/log_ppl_diff:np.float64(0.0007750563090667129) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013818222796544433) - rollout_corr/log_ppl_diff_max:np.float64(0.006634533405303955) - rollout_corr/log_ppl_diff_min:np.float64(-0.004233121871948242) - rollout_corr/ppl_ratio:np.float64(1.0007766336202621) - rollout_corr/chi2_token:np.float64(0.0010767022613435984) - rollout_corr/chi2_seq:np.float64(0.8498944872990251) - actor/pg_loss:np.float64(-0.0003114178543910384) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014040332393960853) - actor/ppo_kl:np.float64(3.355153801543054e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2854798920452595) - perf/mfu/actor:np.float64(0.12016219905694546) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.70193099975586) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:0 - critic/score/mean:0.57421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.57421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013369936496019363 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013369936496019363 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:685.49609375 - response_length/max:1829.0 - response_length/min:195.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:685.49609375 - response_length_non_aborted/max:1829.0 - response_length_non_aborted/min:195.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.75 - prompt_length/max:474.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012979470193386078 - timing_s/agent_loop/generate_sequences/min:np.float64(2.423111606389284) - timing_s/agent_loop/generate_sequences/max:np.float64(15.327023109421134) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.73543391644489) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.327023109421134) - timing_s/agent_loop/slowes
(TaskRunner pid=2224264) 
Training Progress:  33%|███▎      | 33/100 [30:33<1:09:33, 62.29s/it]
(TaskRunner pid=2224264) step:34 - global_seqlen/min:31672 - global_seqlen/max:34621 - global_seqlen/minmax_diff:2949 - global_seqlen/balanced_min:33347 - global_seqlen/balanced_max:33381 - global_seqlen/mean:33365.5 - actor/entropy:0.7147052884101868 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26073455810546875 - training/rollout_probs_diff_mean:0.004393857438117266 - training/rollout_probs_diff_std:0.008422989398241043 - training/rollout_actor_probs_pearson_corr:0.9996627569198608 - rollout_corr/rollout_is_mean:1.0000426769256592 - rollout_corr/rollout_is_ratio_fraction_high:1.0432097042212263e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:5.339309215545654 - rollout_corr/rollout_is_min:0.5086534023284912 - rollout_corr/rollout_is_std:0.03433627635240555 - rollout_corr/rollout_is_eff_sample_size:0.99882252741175 - rollout_corr/rollout_is_seq_mean:1.0001329183578491 - rollout_corr/rollout_is_seq_std:0.001656638691201806 - rollout_corr/rollout_is_seq_max:1.0120186805725098 - rollout_corr/rollout_is_seq_min:0.9953494668006897 - rollout_corr/rollout_is_seq_max_deviation:0.012018680572509766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.108749763574451) - rollout_corr/training_log_ppl:np.float64(0.7289996044710279) - rollout_corr/kl:np.float64(0.0005081289243759057) - rollout_corr/k3_kl:np.float64(0.0006798729952492977) - rollout_corr/rollout_ppl:np.float64(2.107720549684018) - rollout_corr/rollout_log_ppl:np.float64(0.7284914773190394) - rollout_corr/log_ppl_diff:np.float64(0.0005081271519884467) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012719499645754695) - rollout_corr/log_ppl_diff_max:np.float64(0.0048035383224487305) - rollout_corr/log_ppl_diff_min:np.float64(-0.006281495094299316) - rollout_corr/ppl_ratio:np.float64(1.0005095447413623) - rollout_corr/chi2_token:np.float64(0.0017949678003787994) - rollout_corr/chi2_seq:np.float64(4.14957369142212) - actor/pg_loss:np.float64(-5.571101792156696e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020079502564840368) - actor/ppo_kl:np.float64(8.411469075042532e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3010113313794136) - perf/mfu/actor:np.float64(0.12801561879083337) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.21452951431274) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:0 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007644771598279476 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007644771598279476 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:748.890625 - response_length/max:2008.0 - response_length/min:165.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:748.890625 - response_length_non_aborted/max:2008.0 - response_length_non_aborted/min:165.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.78125 - prompt_length/max:621.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014275126159191132 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1188270319253206) - timing_s/agent_loop/generate_sequences/max:np.float64(16.865083750337362) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.53478015192377) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.865083750337362) - timing_s/agent_loop/slowest/tool
(TaskRunner pid=2224264) 
Training Progress:  34%|███▍      | 34/100 [31:14<1:01:32, 55.95s/it]
(TaskRunner pid=2224264) step:35 - global_seqlen/min:28121 - global_seqlen/max:32901 - global_seqlen/minmax_diff:4780 - global_seqlen/balanced_min:30268 - global_seqlen/balanced_max:30287 - global_seqlen/mean:30280.375 - actor/entropy:0.7235326170921326 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21305981278419495 - training/rollout_probs_diff_mean:0.00439543928951025 - training/rollout_probs_diff_std:0.00852300226688385 - training/rollout_actor_probs_pearson_corr:0.9996048808097839 - rollout_corr/rollout_is_mean:1.0000625848770142 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.8470300245971885e-06 - rollout_corr/rollout_is_max:1.7555011510849 - rollout_corr/rollout_is_min:0.4614633619785309 - rollout_corr/rollout_is_std:0.03465937077999115 - rollout_corr/rollout_is_eff_sample_size:0.9988002882356332 - rollout_corr/rollout_is_seq_mean:1.0000625848770142 - rollout_corr/rollout_is_seq_std:0.001464530942030251 - rollout_corr/rollout_is_seq_max:1.0040720701217651 - rollout_corr/rollout_is_seq_min:0.9944659471511841 - rollout_corr/rollout_is_seq_max_deviation:0.005534052848815918 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0981541997753084) - rollout_corr/training_log_ppl:np.float64(0.7273534100968391) - rollout_corr/kl:np.float64(0.0005946585815870264) - rollout_corr/k3_kl:np.float64(0.0006194554455305479) - rollout_corr/rollout_ppl:np.float64(2.0969019741751254) - rollout_corr/rollout_log_ppl:np.float64(0.7267587499227375) - rollout_corr/log_ppl_diff:np.float64(0.0005946601741015911) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012836407404392958) - rollout_corr/log_ppl_diff_max:np.float64(0.005986064672470093) - rollout_corr/log_ppl_diff_min:np.float64(-0.003438234329223633) - rollout_corr/ppl_ratio:np.float64(1.000595951685682) - rollout_corr/chi2_token:np.float64(0.001301520736888051) - rollout_corr/chi2_seq:np.float64(1.3097328948788345) - actor/pg_loss:np.float64(0.00015541678294539452) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001370309196317976) - actor/ppo_kl:np.float64(4.7729094319493015e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2635045759379864) - perf/mfu/actor:np.float64(0.11788960362270617) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.97077560424805) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:0 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0020808118861168623 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0020808118861168623 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:668.07421875 - response_length/max:1568.0 - response_length/min:256.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:668.07421875 - response_length_non_aborted/max:1568.0 - response_length_non_aborted/min:256.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.1875 - prompt_length/max:514.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001488998532295227 - timing_s/agent_loop/generate_sequences/min:np.float64(3.3336482755839825) - timing_s/agent_loop/generate_sequences/max:np.float64(13.82805041782558) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.910963898211776) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.82805041782558) - timing_s/agent_loop/slowest/to
(TaskRunner pid=2224264) 
Training Progress:  35%|███▌      | 35/100 [31:52<54:46, 50.56s/it]  
(TaskRunner pid=2224264) step:36 - global_seqlen/min:23484 - global_seqlen/max:37627 - global_seqlen/minmax_diff:14143 - global_seqlen/balanced_min:32177 - global_seqlen/balanced_max:32196 - global_seqlen/mean:32190.375 - actor/entropy:0.7190540432929993 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9983694553375244 - training/rollout_probs_diff_mean:0.0044546606950461864 - training/rollout_probs_diff_std:0.008799256756901741 - training/rollout_actor_probs_pearson_corr:0.999652624130249 - rollout_corr/rollout_is_mean:0.9999595880508423 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.36610241397284e-06 - rollout_corr/rollout_is_max:1.631447434425354 - rollout_corr/rollout_is_min:0.00012358058302197605 - rollout_corr/rollout_is_std:0.03424588963389397 - rollout_corr/rollout_is_eff_sample_size:0.9988282360231038 - rollout_corr/rollout_is_seq_mean:0.9999497532844543 - rollout_corr/rollout_is_seq_std:0.0013012057170271873 - rollout_corr/rollout_is_seq_max:1.005060076713562 - rollout_corr/rollout_is_seq_min:0.9961336851119995 - rollout_corr/rollout_is_seq_max_deviation:0.005060076713562012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0887983799912035) - rollout_corr/training_log_ppl:np.float64(0.7234805303742178) - rollout_corr/kl:np.float64(0.0007222782085847257) - rollout_corr/k3_kl:np.float64(0.000665171026241751) - rollout_corr/rollout_ppl:np.float64(2.0872840681113303) - rollout_corr/rollout_log_ppl:np.float64(0.7227582528139465) - rollout_corr/log_ppl_diff:np.float64(0.0007222775602713227) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012800676049664617) - rollout_corr/log_ppl_diff_max:np.float64(0.014325737953186035) - rollout_corr/log_ppl_diff_min:np.float64(-0.004422605037689209) - rollout_corr/ppl_ratio:np.float64(1.0007238842081279) - rollout_corr/chi2_token:np.float64(0.001139611704275012) - rollout_corr/chi2_seq:np.float64(2.33724519261159) - actor/pg_loss:np.float64(7.578812073916197e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016148608483490534) - actor/ppo_kl:np.float64(3.428134950134165e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3038923889398575) - perf/mfu/actor:np.float64(0.12476615328489614) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.85765075683594) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:0 - critic/score/mean:0.55859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0017614230746403337 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0017614230746403337 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:727.94921875 - response_length/max:1574.0 - response_length/min:181.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:727.94921875 - response_length_non_aborted/max:1574.0 - response_length_non_aborted/min:181.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.0 - prompt_length/max:704.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010665319859981537 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2873131558299065) - timing_s/agent_loop/generate_sequences/max:np.float64(14.295806081965566) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.435247752218856) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.295806081965566) - timing_s/agent_loop/slowe
(TaskRunner pid=2224264) 
Training Progress:  36%|███▌      | 36/100 [32:30<49:59, 46.86s/it]
(TaskRunner pid=2224264) step:37 - global_seqlen/min:27444 - global_seqlen/max:33696 - global_seqlen/minmax_diff:6252 - global_seqlen/balanced_min:31074 - global_seqlen/balanced_max:31120 - global_seqlen/mean:31108.25 - actor/entropy:0.749553382396698 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1951146274805069 - training/rollout_probs_diff_mean:0.004549640696495771 - training/rollout_probs_diff_std:0.008537894114851952 - training/rollout_actor_probs_pearson_corr:0.9996463060379028 - rollout_corr/rollout_is_mean:0.9998870491981506 - rollout_corr/rollout_is_ratio_fraction_high:5.924381184740923e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0695595741271973 - rollout_corr/rollout_is_min:0.5410090088844299 - rollout_corr/rollout_is_std:0.035030562430620193 - rollout_corr/rollout_is_eff_sample_size:0.9987741257668857 - rollout_corr/rollout_is_seq_mean:0.9999449849128723 - rollout_corr/rollout_is_seq_std:0.001592667424120009 - rollout_corr/rollout_is_seq_max:1.0082616806030273 - rollout_corr/rollout_is_seq_min:0.9951016306877136 - rollout_corr/rollout_is_seq_max_deviation:0.008261680603027344 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2040511020459235) - rollout_corr/training_log_ppl:np.float64(0.7697590510360897) - rollout_corr/kl:np.float64(0.0007292848855877665) - rollout_corr/k3_kl:np.float64(0.0006632836551716537) - rollout_corr/rollout_ppl:np.float64(2.202401583082974) - rollout_corr/rollout_log_ppl:np.float64(0.769029765157029) - rollout_corr/log_ppl_diff:np.float64(0.000729285879060626) - rollout_corr/log_ppl_abs_diff:np.float64(0.001381295034661889) - rollout_corr/log_ppl_diff_max:np.float64(0.005849003791809082) - rollout_corr/log_ppl_diff_min:np.float64(-0.005055129528045654) - rollout_corr/ppl_ratio:np.float64(1.0007309019565582) - rollout_corr/chi2_token:np.float64(0.001207549823448062) - rollout_corr/chi2_seq:np.float64(0.7918412457220256) - actor/pg_loss:np.float64(-0.0002318680053576827) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019654674611047085) - actor/ppo_kl:np.float64(3.777838127683175e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3199927434325218) - perf/mfu/actor:np.float64(0.11945996157211544) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.98107528686523) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:0 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011995390988886356 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011995390988886356 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:659.3515625 - response_length/max:2064.0 - response_length/min:147.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:659.3515625 - response_length_non_aborted/max:2064.0 - response_length_non_aborted/min:147.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.78125 - prompt_length/max:1009.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015878677368164062 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0059274714440107) - timing_s/agent_loop/generate_sequences/max:np.float64(16.902138613164425) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.578634077712195) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.902138613164425) - timing_s/agent_loop/slowest/
(TaskRunner pid=2224264) 
Training Progress:  37%|███▋      | 37/100 [33:11<47:22, 45.12s/it]
(TaskRunner pid=2224264) step:38 - global_seqlen/min:22559 - global_seqlen/max:30637 - global_seqlen/minmax_diff:8078 - global_seqlen/balanced_min:27684 - global_seqlen/balanced_max:27696 - global_seqlen/mean:27690.75 - actor/entropy:0.7535861730575562 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.36917543411254883 - training/rollout_probs_diff_mean:0.004689306020736694 - training/rollout_probs_diff_std:0.008696562610566616 - training/rollout_actor_probs_pearson_corr:0.9996323585510254 - rollout_corr/rollout_is_mean:1.0000420808792114 - rollout_corr/rollout_is_ratio_fraction_high:6.4868513618421275e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0322699546813965 - rollout_corr/rollout_is_min:0.5430662631988525 - rollout_corr/rollout_is_std:0.03519861027598381 - rollout_corr/rollout_is_eff_sample_size:0.9987627098464102 - rollout_corr/rollout_is_seq_mean:1.0000241994857788 - rollout_corr/rollout_is_seq_std:0.0014864748809486628 - rollout_corr/rollout_is_seq_max:1.0049471855163574 - rollout_corr/rollout_is_seq_min:0.995693564414978 - rollout_corr/rollout_is_seq_max_deviation:0.004947185516357422 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1352647584863007) - rollout_corr/training_log_ppl:np.float64(0.7448823321610689) - rollout_corr/kl:np.float64(0.0005967357578882115) - rollout_corr/k3_kl:np.float64(0.0006872201306578063) - rollout_corr/rollout_ppl:np.float64(2.1338966507464647) - rollout_corr/rollout_log_ppl:np.float64(0.7442855968838558) - rollout_corr/log_ppl_diff:np.float64(0.0005967352772131562) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013833154225721955) - rollout_corr/log_ppl_diff_max:np.float64(0.0049326419830322266) - rollout_corr/log_ppl_diff_min:np.float64(-0.004489302635192871) - rollout_corr/ppl_ratio:np.float64(1.0005982399452478) - rollout_corr/chi2_token:np.float64(0.0015692522283643484) - rollout_corr/chi2_seq:np.float64(1.3063512451481074) - actor/pg_loss:np.float64(-0.00012253155000507832) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015133037595660426) - actor/ppo_kl:np.float64(-1.1263918693771302e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.27262789383530617) - perf/mfu/actor:np.float64(0.10989664398239996) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.94292449951172) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:0 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0031607181299477816 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0031607181299477816 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:602.1796875 - response_length/max:1274.0 - response_length/min:154.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:602.1796875 - response_length_non_aborted/max:1274.0 - response_length_non_aborted/min:154.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.15625 - prompt_length/max:500.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.338022649288177e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9933321122080088) - timing_s/agent_loop/generate_sequences/max:np.float64(11.328456040471792) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.766713866491045) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.328456040471792) - timing_s/agent_l
(TaskRunner pid=2224264) 
Training Progress:  38%|███▊      | 38/100 [33:45<43:17, 41.90s/it]
(TaskRunner pid=2224264) step:39 - global_seqlen/min:24673 - global_seqlen/max:36395 - global_seqlen/minmax_diff:11722 - global_seqlen/balanced_min:29976 - global_seqlen/balanced_max:30014 - global_seqlen/mean:30001.0 - actor/entropy:0.7436038255691528 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39769014716148376 - training/rollout_probs_diff_mean:0.004428871441632509 - training/rollout_probs_diff_std:0.008462125435471535 - training/rollout_actor_probs_pearson_corr:0.9996308088302612 - rollout_corr/rollout_is_mean:0.9998753666877747 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.558502197265625 - rollout_corr/rollout_is_min:0.5394280552864075 - rollout_corr/rollout_is_std:0.03459912911057472 - rollout_corr/rollout_is_eff_sample_size:0.9988039748754949 - rollout_corr/rollout_is_seq_mean:0.9998376965522766 - rollout_corr/rollout_is_seq_std:0.0016168290749192238 - rollout_corr/rollout_is_seq_max:1.0040711164474487 - rollout_corr/rollout_is_seq_min:0.9940407276153564 - rollout_corr/rollout_is_seq_max_deviation:0.005959272384643555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.18053255463019) - rollout_corr/training_log_ppl:np.float64(0.7624029302969575) - rollout_corr/kl:np.float64(0.0008376638515699142) - rollout_corr/k3_kl:np.float64(0.0006715801446830483) - rollout_corr/rollout_ppl:np.float64(2.178553344681859) - rollout_corr/rollout_log_ppl:np.float64(0.7615652666427195) - rollout_corr/log_ppl_diff:np.float64(0.0008376636542379856) - rollout_corr/log_ppl_abs_diff:np.float64(0.001551654888316989) - rollout_corr/log_ppl_diff_max:np.float64(0.00657200813293457) - rollout_corr/log_ppl_diff_min:np.float64(-0.004390835762023926) - rollout_corr/ppl_ratio:np.float64(1.0008396471384913) - rollout_corr/chi2_token:np.float64(0.0010091052390635014) - rollout_corr/chi2_seq:np.float64(2.287320557516068) - actor/pg_loss:np.float64(-9.888317435979843e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001992107047499303) - actor/ppo_kl:np.float64(5.7301584355684554e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29527168720960617) - perf/mfu/actor:np.float64(0.11556263551770317) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.95692443847656) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:0 - critic/score/mean:0.47265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.47265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002959122182801366 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002959122182801366 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:655.25 - response_length/max:1746.0 - response_length/min:175.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:655.25 - response_length_non_aborted/max:1746.0 - response_length_non_aborted/min:175.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:282.28125 - prompt_length/max:452.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001086648553609848 - timing_s/agent_loop/generate_sequences/min:np.float64(2.099978694692254) - timing_s/agent_loop/generate_sequences/max:np.float64(14.446734165772796) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.490971767700103) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.446734165772796) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - t
(TaskRunner pid=2224264) 
Training Progress:  39%|███▉      | 39/100 [34:24<41:38, 40.96s/it]
(TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40}
(TaskRunner pid=2224264) validation generation end
(TaskRunner pid=2224264) [prompt] system
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2224264) <reasoning>
(TaskRunner pid=2224264) ...
(TaskRunner pid=2224264) </reasoning>
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) ...
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) A
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) user
(TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2224264) Please reason step by step.
(TaskRunner pid=2224264) assistant
(TaskRunner pid=2224264) <think>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) </think>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) [response] <reasoning>
(TaskRunner pid=2224264) We are given a target product:  
(TaskRunner pid=2224264) **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"**  
(TaskRunner pid=2224264) This is a nitrogen-containing compound with a specific substituent pattern. Let's break down the structure:
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) - The core is a **pyrrole ring** (N connected to a carbon ring).
(TaskRunner pid=2224264) - The nitrogen is attached to a **carbon chain** that leads to:
(TaskRunner pid=2224264)   - A **chloro-substituted benzene ring** (from "Cc2ccc(Cl)cc2C(F)(F)F")
(TaskRunner pid=2224264)   - A **trifluoromethyl group** (C(F)(F)F)
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) So the structure is:  
(TaskRunner pid=2224264) **N–C–(–CH₂–C(Cl)–C(F)(F)F–)**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) Now, we are to identify the **correct reactant** that could be used in the synthesis of this compound. Let's evaluate each option.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Option A:  
(TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This appears to be a **pyrrole derivative** with a **carbonyl group** (O=C) and a **trifluoromethyl group**. The structure seems to match the target product. The **"c1ccn(...)"** part resembles the pyrrole ring.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This is a **strong candidate** for the correct reactant.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Option B:  
(TaskRunner pid=2224264) **O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This has a **chlorine atom** on the ring and a **trifluoromethyl** group? Not directly. The structure is more complex and does not clearly match the target.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Option C:  
(TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This includes a **trifluoromethyl group** and a **chloro-substituted benzene**, but the **N** is not directly attached to the **chloro-substituted benzene**. The structure is more complex than the target.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Option D:  
(TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This has a **trifluoromethyl group** and a **benzene ring**, but the **N** is not directly attached to the **chloro-substituted benzene**. The structure is also more complex than the target.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Final Evaluation:
(TaskRunner pid=2224264) - **Option A** contains a **pyrrole ring** with a **chloro-substituted benzene** and a **trifluoromethyl group**, which matches the structure of the target product.
(TaskRunner pid=2224264) - The other options either lack the correct substituent pattern or have more complex structures that do not align with the target.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### ✅ **Correct Answer: A**
(TaskRunner pid=2224264) </reasoning>
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) A
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) [ground_truth] A
(TaskRunner pid=2224264) [score] 1.0
(TaskRunner pid=2224264) [acc] 1.0
(TaskRunner pid=2224264) [pred] A
(TaskRunner pid=2224264) [incorrect_format] 1
(TaskRunner pid=2224264) [feedback] 
(TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40
(TaskRunner pid=2224264) step:40 - global_seqlen/min:26780 - global_seqlen/max:38560 - global_seqlen/minmax_diff:11780 - global_seqlen/balanced_min:31398 - global_seqlen/balanced_max:31614 - global_seqlen/mean:31430.375 - actor/entropy:0.7148814797401428 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.353511780500412 - training/rollout_probs_diff_mean:0.004250292666256428 - training/rollout_probs_diff_std:0.008334573358297348 - training/rollout_actor_probs_pearson_corr:0.9996918439865112 - rollout_corr/rollout_is_mean:0.9999884963035583 - rollout_corr/rollout_is_ratio_fraction_high:5.834475814481266e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.834475814481266e-06 - rollout_corr/rollout_is_max:2.314491033554077 - rollout_corr/rollout_is_min:0.4158317446708679 - rollout_corr/rollout_is_std:0.034074895083904266 - rollout_corr/rollout_is_eff_sample_size:0.9988401291730077 - rollout_corr/rollout_is_seq_mean:0.9999082088470459 - rollout_corr/rollout_is_seq_std:0.0015310312155634165 - rollout_corr/rollout_is_seq_max:1.0046111345291138 - rollout_corr/rollout_is_seq_min:0.9931868314743042 - rollout_corr/rollout_is_seq_max_deviation:0.006813168525695801 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0881334473378956) - rollout_corr/training_log_ppl:np.float64(0.7168666583602317) - rollout_corr/kl:np.float64(0.0009168515699862212) - rollout_corr/k3_kl:np.float64(0.0007126086813400434) - rollout_corr/rollout_ppl:np.float64(2.0860950271598995) - rollout_corr/rollout_log_ppl:np.float64(0.7159498034743592) - rollout_corr/log_ppl_diff:np.float64(0.0009168548858724535) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013491588761098683) - rollout_corr/log_ppl_diff_max:np.float64(0.006253480911254883) - rollout_corr/log_ppl_diff_min:np.float64(-0.003960609436035156) - rollout_corr/ppl_ratio:np.float64(1.0009184894151986) - rollout_corr/chi2_token:np.float64(0.0010311168152838945) - rollout_corr/chi2_seq:np.float64(0.5281695865560323) - actor/pg_loss:np.float64(0.0002642440376803279) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030195120598364156) - actor/ppo_kl:np.float64(0.00023282111740385858) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.26611069589853287) - perf/mfu/actor:np.float64(0.12153132959867792) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.97843170166016) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6053571428571428) - val-aux/sciknoweval/reward/std@16:np.float64(0.28468084712859354) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7211619047619049) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.22140257031009367) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.4908571428571428) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.25871458688057813) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6054666666666666) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.28424403856079056) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8044809523809524) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.15305264517858658) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.3869142857142857) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.2100138722335839) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6357761904761906) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.22516406707067232) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8634285714285714) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09899214217926788) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.3012904761904762) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.15568733164559595) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6521095238095237) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.15983092148992065) - val-aux/sciknoweval/reward/bes
(TaskRunner pid=2224264) 
Training Progress:  40%|████      | 40/100 [38:25<1:41:00, 101.00s/it]
(TaskRunner pid=2224264) step:41 - global_seqlen/min:25184 - global_seqlen/max:36473 - global_seqlen/minmax_diff:11289 - global_seqlen/balanced_min:30315 - global_seqlen/balanced_max:30362 - global_seqlen/mean:30349.125 - actor/entropy:0.7456910610198975 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1853349208831787 - training/rollout_probs_diff_mean:0.004585789050906897 - training/rollout_probs_diff_std:0.008587997406721115 - training/rollout_actor_probs_pearson_corr:0.9996112585067749 - rollout_corr/rollout_is_mean:1.000074863433838 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1413375432312023e-05 - rollout_corr/rollout_is_max:1.7971372604370117 - rollout_corr/rollout_is_min:0.38024628162384033 - rollout_corr/rollout_is_std:0.0351342037320137 - rollout_corr/rollout_is_eff_sample_size:0.9987673475326293 - rollout_corr/rollout_is_seq_mean:1.000067949295044 - rollout_corr/rollout_is_seq_std:0.0015586130321025848 - rollout_corr/rollout_is_seq_max:1.005883812904358 - rollout_corr/rollout_is_seq_min:0.993992805480957 - rollout_corr/rollout_is_seq_max_deviation:0.006007194519042969 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1489445390179753) - rollout_corr/training_log_ppl:np.float64(0.7491948831593618) - rollout_corr/kl:np.float64(0.0007140677866042111) - rollout_corr/k3_kl:np.float64(0.000693655981081065) - rollout_corr/rollout_ppl:np.float64(2.147350816987455) - rollout_corr/rollout_log_ppl:np.float64(0.7484808203298599) - rollout_corr/log_ppl_diff:np.float64(0.0007140628295019269) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013865955406799912) - rollout_corr/log_ppl_diff_max:np.float64(0.00584721565246582) - rollout_corr/log_ppl_diff_min:np.float64(-0.004841506481170654) - rollout_corr/ppl_ratio:np.float64(1.0007157281506807) - rollout_corr/chi2_token:np.float64(0.0013592832256108522) - rollout_corr/chi2_seq:np.float64(1.1909501447807997) - actor/pg_loss:np.float64(0.000211457721889019) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024292130228786846) - actor/ppo_kl:np.float64(0.00014254798117097067) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2715475298464298) - perf/mfu/actor:np.float64(0.11821237532599987) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(105.00201988220215) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:0 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00029318107408471406 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00029318107408471406 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:684.50390625 - response_length/max:1922.0 - response_length/min:186.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:684.50390625 - response_length_non_aborted/max:1922.0 - response_length_non_aborted/min:186.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.90625 - prompt_length/max:631.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.857250213623047e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0997980404645205) - timing_s/agent_loop/generate_sequences/max:np.float64(15.434562433511019) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.771742777309555) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.434562433511019) - timing_s/agent_loop/slowest/tool_
(TaskRunner pid=2224264) 
Training Progress:  41%|████      | 41/100 [39:05<1:21:09, 82.54s/it] 
(TaskRunner pid=2224264) step:42 - global_seqlen/min:23559 - global_seqlen/max:32248 - global_seqlen/minmax_diff:8689 - global_seqlen/balanced_min:29140 - global_seqlen/balanced_max:29158 - global_seqlen/mean:29150.625 - actor/entropy:0.7418811321258545 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8146259784698486 - training/rollout_probs_diff_mean:0.004636453464627266 - training/rollout_probs_diff_std:0.008857984095811844 - training/rollout_actor_probs_pearson_corr:0.9996300339698792 - rollout_corr/rollout_is_mean:0.9999567270278931 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.9698286349885166e-06 - rollout_corr/rollout_is_max:1.5637112855911255 - rollout_corr/rollout_is_min:0.1537874937057495 - rollout_corr/rollout_is_std:0.03467312455177307 - rollout_corr/rollout_is_eff_sample_size:0.9987990990027735 - rollout_corr/rollout_is_seq_mean:0.9999449849128723 - rollout_corr/rollout_is_seq_std:0.0014816136099398136 - rollout_corr/rollout_is_seq_max:1.006678581237793 - rollout_corr/rollout_is_seq_min:0.9932494759559631 - rollout_corr/rollout_is_seq_max_deviation:0.006750524044036865 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.149658269248903) - rollout_corr/training_log_ppl:np.float64(0.750017037265934) - rollout_corr/kl:np.float64(0.0007465042794017052) - rollout_corr/k3_kl:np.float64(0.0006643551571414719) - rollout_corr/rollout_ppl:np.float64(2.148035450838506) - rollout_corr/rollout_log_ppl:np.float64(0.7492705312324688) - rollout_corr/log_ppl_diff:np.float64(0.0007465060334652662) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012925199698656797) - rollout_corr/log_ppl_diff_max:np.float64(0.005987584590911865) - rollout_corr/log_ppl_diff_min:np.float64(-0.00581049919128418) - rollout_corr/ppl_ratio:np.float64(1.0007479563355446) - rollout_corr/chi2_token:np.float64(0.0011864088010042906) - rollout_corr/chi2_seq:np.float64(0.5256321090273559) - actor/pg_loss:np.float64(-3.93550144508481e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024687639711373777) - actor/ppo_kl:np.float64(6.906396298589357e-05) - actor/pg_clipfrac_lower:np.float64(8.510348379786592e-06) - actor/grad_norm:np.float64(0.31687242165207863) - perf/mfu/actor:np.float64(0.11438361389153799) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.63388013839722) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:0 - critic/score/mean:0.55859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007985391654074192 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007985391654074192 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:654.33203125 - response_length/max:1379.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:654.33203125 - response_length_non_aborted/max:1379.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:256.625 - prompt_length/max:557.0 - prompt_length/min:175.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015127286314964294 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8712534438818693) - timing_s/agent_loop/generate_sequences/max:np.float64(12.603244848549366) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.466689185537689) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.603244848549366) - timing_s/a
(TaskRunner pid=2224264) 
Training Progress:  42%|████▏     | 42/100 [39:42<1:06:43, 69.02s/it]
(TaskRunner pid=2224264) step:43 - global_seqlen/min:28157 - global_seqlen/max:35791 - global_seqlen/minmax_diff:7634 - global_seqlen/balanced_min:32488 - global_seqlen/balanced_max:32514 - global_seqlen/mean:32503.125 - actor/entropy:0.7455630302429199 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102365970611572 - training/rollout_probs_diff_mean:0.0044556111097335815 - training/rollout_probs_diff_std:0.008428295142948627 - training/rollout_actor_probs_pearson_corr:0.9996055960655212 - rollout_corr/rollout_is_mean:1.0000288486480713 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.1921050574746914e-05 - rollout_corr/rollout_is_max:1.6338140964508057 - rollout_corr/rollout_is_min:0.3978634774684906 - rollout_corr/rollout_is_std:0.03456293046474457 - rollout_corr/rollout_is_eff_sample_size:0.9988067101420113 - rollout_corr/rollout_is_seq_mean:1.000025749206543 - rollout_corr/rollout_is_seq_std:0.0014121269341558218 - rollout_corr/rollout_is_seq_max:1.0047554969787598 - rollout_corr/rollout_is_seq_min:0.9933763146400452 - rollout_corr/rollout_is_seq_max_deviation:0.006623685359954834 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1632221047766507) - rollout_corr/training_log_ppl:np.float64(0.7551403474062681) - rollout_corr/kl:np.float64(0.0007046414169096238) - rollout_corr/k3_kl:np.float64(0.0006673236663345961) - rollout_corr/rollout_ppl:np.float64(2.16164415422827) - rollout_corr/rollout_log_ppl:np.float64(0.7544357032165863) - rollout_corr/log_ppl_diff:np.float64(0.0007046441896818578) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013581206439994276) - rollout_corr/log_ppl_diff_max:np.float64(0.013478577136993408) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035314559936523438) - rollout_corr/ppl_ratio:np.float64(1.0007065150421113) - rollout_corr/chi2_token:np.float64(0.0012414606753736734) - rollout_corr/chi2_seq:np.float64(1.941952116554603) - actor/pg_loss:np.float64(6.042851600795984e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017739249074111285) - actor/ppo_kl:np.float64(0.00010932787842676817) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.27033093199133873) - perf/mfu/actor:np.float64(0.12426556577467283) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.3560562133789) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005291193723678589 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005291193723678589 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:712.78515625 - response_length/max:1722.0 - response_length/min:175.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:712.78515625 - response_length_non_aborted/max:1722.0 - response_length_non_aborted/min:175.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.9375 - prompt_length/max:813.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010757148265838623 - timing_s/agent_loop/generate_sequences/min:np.float64(2.216030417010188) - timing_s/agent_loop/generate_sequences/max:np.float64(14.333171149715781) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.09952915172471) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.333171149715781) - timing_s/agent_loop/slowes
(TaskRunner pid=2224264) 
Training Progress:  43%|████▎     | 43/100 [40:21<56:49, 59.81s/it]  
(TaskRunner pid=2224264) step:44 - global_seqlen/min:28261 - global_seqlen/max:39735 - global_seqlen/minmax_diff:11474 - global_seqlen/balanced_min:33071 - global_seqlen/balanced_max:39250 - global_seqlen/mean:33860.375 - actor/entropy:0.7164559364318848 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3860778212547302 - training/rollout_probs_diff_mean:0.0043455152772367 - training/rollout_probs_diff_std:0.008481818251311779 - training/rollout_actor_probs_pearson_corr:0.9996494650840759 - rollout_corr/rollout_is_mean:1.0000731945037842 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.569669802847784e-05 - rollout_corr/rollout_is_max:1.6253478527069092 - rollout_corr/rollout_is_min:0.3264278769493103 - rollout_corr/rollout_is_std:0.03381501883268356 - rollout_corr/rollout_is_eff_sample_size:0.9988579694289216 - rollout_corr/rollout_is_seq_mean:1.0000576972961426 - rollout_corr/rollout_is_seq_std:0.0014711449621245265 - rollout_corr/rollout_is_seq_max:1.0052176713943481 - rollout_corr/rollout_is_seq_min:0.9947458505630493 - rollout_corr/rollout_is_seq_max_deviation:0.005254149436950684 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1722452212125063) - rollout_corr/training_log_ppl:np.float64(0.7589113183676091) - rollout_corr/kl:np.float64(0.0006052850321509595) - rollout_corr/k3_kl:np.float64(0.0006508081007154942) - rollout_corr/rollout_ppl:np.float64(2.170979785732925) - rollout_corr/rollout_log_ppl:np.float64(0.7583060346714774) - rollout_corr/log_ppl_diff:np.float64(0.0006052836961316643) - rollout_corr/log_ppl_abs_diff:np.float64(0.001306259231569129) - rollout_corr/log_ppl_diff_max:np.float64(0.005559146404266357) - rollout_corr/log_ppl_diff_min:np.float64(-0.00449979305267334) - rollout_corr/ppl_ratio:np.float64(1.0006066909991205) - rollout_corr/chi2_token:np.float64(0.001387621508911252) - rollout_corr/chi2_seq:np.float64(1.5838370565325022) - actor/pg_loss:np.float64(8.792639710009098e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022318573451229895) - actor/ppo_kl:np.float64(5.0495547061046864e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.303693313151598) - perf/mfu/actor:np.float64(0.121345258264149) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.35682678222656) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:0 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03973030298948288 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.03973030298948288 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:746.57421875 - response_length/max:8192.0 - response_length/min:178.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:746.57421875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:178.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:311.5625 - prompt_length/max:701.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.71539455652237e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3765512369573116) - timing_s/agent_loop/generate_sequences/max:np.float64(61.594920651987195) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.441452034945542) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(61.594920651987195) - timing_s/agent_loo
(TaskRunner pid=2224264) 
Training Progress:  44%|████▍     | 44/100 [41:47<1:03:24, 67.94s/it]
(TaskRunner pid=2224264) step:45 - global_seqlen/min:30046 - global_seqlen/max:36319 - global_seqlen/minmax_diff:6273 - global_seqlen/balanced_min:33079 - global_seqlen/balanced_max:33119 - global_seqlen/mean:33104.0 - actor/entropy:0.7044548392295837 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.456476628780365 - training/rollout_probs_diff_mean:0.004338444676250219 - training/rollout_probs_diff_std:0.008459679782390594 - training/rollout_actor_probs_pearson_corr:0.999611496925354 - rollout_corr/rollout_is_mean:1.0000234842300415 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.426760253612883e-06 - rollout_corr/rollout_is_max:1.7352091073989868 - rollout_corr/rollout_is_min:0.412738174200058 - rollout_corr/rollout_is_std:0.03374973684549332 - rollout_corr/rollout_is_eff_sample_size:0.9988624890615934 - rollout_corr/rollout_is_seq_mean:0.9999771118164062 - rollout_corr/rollout_is_seq_std:0.0013954972382634878 - rollout_corr/rollout_is_seq_max:1.0044217109680176 - rollout_corr/rollout_is_seq_min:0.9954630732536316 - rollout_corr/rollout_is_seq_max_deviation:0.004536926746368408 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1041139638982713) - rollout_corr/training_log_ppl:np.float64(0.7206308286986314) - rollout_corr/kl:np.float64(0.0006204138266534542) - rollout_corr/k3_kl:np.float64(0.0006203657341075086) - rollout_corr/rollout_ppl:np.float64(2.102761774789542) - rollout_corr/rollout_log_ppl:np.float64(0.7200104069779627) - rollout_corr/log_ppl_diff:np.float64(0.0006204217206686735) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011971264611929655) - rollout_corr/log_ppl_diff_max:np.float64(0.005265474319458008) - rollout_corr/log_ppl_diff_min:np.float64(-0.0036600232124328613) - rollout_corr/ppl_ratio:np.float64(1.0006216508336365) - rollout_corr/chi2_token:np.float64(0.0012450276408344507) - rollout_corr/chi2_seq:np.float64(1.1021782979369164) - actor/pg_loss:np.float64(-7.425120566040277e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015113081622075697) - actor/ppo_kl:np.float64(1.2432504128412347e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29480820521712303) - perf/mfu/actor:np.float64(0.1272612688514589) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.27553939819336) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:0 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005770002957433462 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005770002957433462 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:719.8125 - response_length/max:1962.0 - response_length/min:191.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:719.8125 - response_length_non_aborted/max:1962.0 - response_length_non_aborted/min:191.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.6875 - prompt_length/max:845.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.384138345718384e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.28562774322927) - timing_s/agent_loop/generate_sequences/max:np.float64(16.55846000276506) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.204205383684894) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.55846000276506) - timing_s/agent_loop/slowest/tool_call
(TaskRunner pid=2224264) 
Training Progress:  45%|████▌     | 45/100 [42:28<54:41, 59.66s/it]  
(TaskRunner pid=2224264) step:46 - global_seqlen/min:25951 - global_seqlen/max:37896 - global_seqlen/minmax_diff:11945 - global_seqlen/balanced_min:31356 - global_seqlen/balanced_max:31791 - global_seqlen/mean:31428.0 - actor/entropy:0.7460911870002747 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.456478089094162 - training/rollout_probs_diff_mean:0.004542972426861525 - training/rollout_probs_diff_std:0.008595590479671955 - training/rollout_actor_probs_pearson_corr:0.9995923638343811 - rollout_corr/rollout_is_mean:1.0000001192092896 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1051189176214393e-05 - rollout_corr/rollout_is_max:1.62330961227417 - rollout_corr/rollout_is_min:0.39613932371139526 - rollout_corr/rollout_is_std:0.03452324494719505 - rollout_corr/rollout_is_eff_sample_size:0.9988095643491313 - rollout_corr/rollout_is_seq_mean:0.9999744892120361 - rollout_corr/rollout_is_seq_std:0.0015889799688011408 - rollout_corr/rollout_is_seq_max:1.0049422979354858 - rollout_corr/rollout_is_seq_min:0.9941216111183167 - rollout_corr/rollout_is_seq_max_deviation:0.00587838888168335 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2059524417854846) - rollout_corr/training_log_ppl:np.float64(0.7630574661307037) - rollout_corr/kl:np.float64(0.0006121348877812238) - rollout_corr/k3_kl:np.float64(0.000635629686428274) - rollout_corr/rollout_ppl:np.float64(2.204429803881794) - rollout_corr/rollout_log_ppl:np.float64(0.7624453329481184) - rollout_corr/log_ppl_diff:np.float64(0.0006121331825852394) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012938708532601595) - rollout_corr/log_ppl_diff_max:np.float64(0.006876647472381592) - rollout_corr/log_ppl_diff_min:np.float64(-0.004626870155334473) - rollout_corr/ppl_ratio:np.float64(1.0006135429721326) - rollout_corr/chi2_token:np.float64(0.0013200147077441216) - rollout_corr/chi2_seq:np.float64(1.2292231675237417) - actor/pg_loss:np.float64(-5.2339513786137104e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001717434528245576) - actor/ppo_kl:np.float64(-3.4712943731474155e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.27383991330862045) - perf/mfu/actor:np.float64(0.11895889859672076) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.13016891479492) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:0 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014861086383461952 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014861086383461952 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:706.9375 - response_length/max:2609.0 - response_length/min:176.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:706.9375 - response_length_non_aborted/max:2609.0 - response_length_non_aborted/min:176.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:275.1875 - prompt_length/max:595.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.24672931432724e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7639765553176403) - timing_s/agent_loop/generate_sequences/max:np.float64(19.902556966990232) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.572092931586667) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.902556966990232) - timing_s/agent_loop/slowest/tool
(TaskRunner pid=2224264) 
Training Progress:  46%|████▌     | 46/100 [43:13<49:43, 55.26s/it]
(TaskRunner pid=2224264) step:47 - global_seqlen/min:28796 - global_seqlen/max:40193 - global_seqlen/minmax_diff:11397 - global_seqlen/balanced_min:34771 - global_seqlen/balanced_max:34835 - global_seqlen/mean:34821.25 - actor/entropy:0.7023751139640808 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34761902689933777 - training/rollout_probs_diff_mean:0.004359257873147726 - training/rollout_probs_diff_std:0.008405433967709541 - training/rollout_actor_probs_pearson_corr:0.9996216297149658 - rollout_corr/rollout_is_mean:0.9998831152915955 - rollout_corr/rollout_is_ratio_fraction_high:4.914246346743312e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.914246346743312e-06 - rollout_corr/rollout_is_max:2.393986701965332 - rollout_corr/rollout_is_min:0.404596209526062 - rollout_corr/rollout_is_std:0.03408888727426529 - rollout_corr/rollout_is_eff_sample_size:0.9988389398452722 - rollout_corr/rollout_is_seq_mean:0.9998857975006104 - rollout_corr/rollout_is_seq_std:0.0014544243458658457 - rollout_corr/rollout_is_seq_max:1.0058720111846924 - rollout_corr/rollout_is_seq_min:0.9959080219268799 - rollout_corr/rollout_is_seq_max_deviation:0.005872011184692383 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.088344756513834) - rollout_corr/training_log_ppl:np.float64(0.71887747416622) - rollout_corr/kl:np.float64(0.0006769733809974809) - rollout_corr/k3_kl:np.float64(0.0006363334275647503) - rollout_corr/rollout_ppl:np.float64(2.086956723127514) - rollout_corr/rollout_log_ppl:np.float64(0.7182004956412129) - rollout_corr/log_ppl_diff:np.float64(0.0006769785250071436) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012075564882252365) - rollout_corr/log_ppl_diff_max:np.float64(0.006249547004699707) - rollout_corr/log_ppl_diff_min:np.float64(-0.004652619361877441) - rollout_corr/ppl_ratio:np.float64(1.0006782109849155) - rollout_corr/chi2_token:np.float64(0.0012049346696585417) - rollout_corr/chi2_seq:np.float64(0.8489019055850804) - actor/pg_loss:np.float64(0.00018669036217033863) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019566034120543918) - actor/ppo_kl:np.float64(-4.604769495308858e-05) - actor/pg_clipfrac_lower:np.float64(4.4389203139871825e-06) - actor/grad_norm:np.float64(0.2251530885696411) - perf/mfu/actor:np.float64(0.1320186966648972) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.04901123046875) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:0 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006613961420953274 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006613961420953274 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:794.8828125 - response_length/max:2347.0 - response_length/min:195.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:794.8828125 - response_length_non_aborted/max:2347.0 - response_length_non_aborted/min:195.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.28125 - prompt_length/max:950.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010444037616252899 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4994081798940897) - timing_s/agent_loop/generate_sequences/max:np.float64(18.286521969363093) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.572983121870493) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.286521969363093)
(TaskRunner pid=2224264) 
Training Progress:  47%|████▋     | 47/100 [43:55<45:26, 51.45s/it]
(TaskRunner pid=2224264) step:48 - global_seqlen/min:21182 - global_seqlen/max:34146 - global_seqlen/minmax_diff:12964 - global_seqlen/balanced_min:29830 - global_seqlen/balanced_max:29848 - global_seqlen/mean:29841.125 - actor/entropy:0.7442194819450378 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4606970250606537 - training/rollout_probs_diff_mean:0.004590690601617098 - training/rollout_probs_diff_std:0.008653250522911549 - training/rollout_actor_probs_pearson_corr:0.999643087387085 - rollout_corr/rollout_is_mean:0.9999340772628784 - rollout_corr/rollout_is_ratio_fraction_high:1.1716118024196476e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.565861463546753 - rollout_corr/rollout_is_min:0.6271895170211792 - rollout_corr/rollout_is_std:0.03491979092359543 - rollout_corr/rollout_is_eff_sample_size:0.9987818554445442 - rollout_corr/rollout_is_seq_mean:0.9998950362205505 - rollout_corr/rollout_is_seq_std:0.0015674727037549019 - rollout_corr/rollout_is_seq_max:1.0053834915161133 - rollout_corr/rollout_is_seq_min:0.9940774440765381 - rollout_corr/rollout_is_seq_max_deviation:0.005922555923461914 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.153257429599762) - rollout_corr/training_log_ppl:np.float64(0.7468871437013149) - rollout_corr/kl:np.float64(0.0006051990699234011) - rollout_corr/k3_kl:np.float64(0.0006705579712615872) - rollout_corr/rollout_ppl:np.float64(2.1520667281001806) - rollout_corr/rollout_log_ppl:np.float64(0.7462819424690679) - rollout_corr/log_ppl_diff:np.float64(0.0006052012322470546) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013254067162051797) - rollout_corr/log_ppl_diff_max:np.float64(0.006841838359832764) - rollout_corr/log_ppl_diff_min:np.float64(-0.004772186279296875) - rollout_corr/ppl_ratio:np.float64(1.0006067838985473) - rollout_corr/chi2_token:np.float64(0.0014893666375428438) - rollout_corr/chi2_seq:np.float64(1.7015499353874475) - actor/pg_loss:np.float64(-4.240649286657572e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019128616168018198) - actor/ppo_kl:np.float64(-0.00013246856422000075) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3110559768974781) - perf/mfu/actor:np.float64(0.11562384315159505) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.0516357421875) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:0 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004874637350440025 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004874637350440025 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:666.81640625 - response_length/max:1727.0 - response_length/min:162.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:666.81640625 - response_length_non_aborted/max:1727.0 - response_length_non_aborted/min:162.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.71875 - prompt_length/max:563.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011752359569072723 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0456913970410824) - timing_s/agent_loop/generate_sequences/max:np.float64(14.015774877741933) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.352618356359017) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.015774877741933) - timing_s/agent_loop
(TaskRunner pid=2224264) 
Training Progress:  48%|████▊     | 48/100 [44:34<41:11, 47.54s/it]
(TaskRunner pid=2224264) step:49 - global_seqlen/min:27565 - global_seqlen/max:33967 - global_seqlen/minmax_diff:6402 - global_seqlen/balanced_min:31660 - global_seqlen/balanced_max:31677 - global_seqlen/mean:31668.625 - actor/entropy:0.7485659122467041 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8766915202140808 - training/rollout_probs_diff_mean:0.004531657788902521 - training/rollout_probs_diff_std:0.008790009655058384 - training/rollout_actor_probs_pearson_corr:0.9996523857116699 - rollout_corr/rollout_is_mean:1.0000706911087036 - rollout_corr/rollout_is_ratio_fraction_high:5.495320692716632e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.495320692716632e-06 - rollout_corr/rollout_is_max:8.354974746704102 - rollout_corr/rollout_is_min:0.0006339381216093898 - rollout_corr/rollout_is_std:0.03499140590429306 - rollout_corr/rollout_is_eff_sample_size:0.99877733654154 - rollout_corr/rollout_is_seq_mean:1.0000840425491333 - rollout_corr/rollout_is_seq_std:0.0015765647403895855 - rollout_corr/rollout_is_seq_max:1.0120781660079956 - rollout_corr/rollout_is_seq_min:0.9965959787368774 - rollout_corr/rollout_is_seq_max_deviation:0.012078166007995605 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.135033040307462) - rollout_corr/training_log_ppl:np.float64(0.7435363659169525) - rollout_corr/kl:np.float64(0.000531884074939093) - rollout_corr/k3_kl:np.float64(0.0007136939013889787) - rollout_corr/rollout_ppl:np.float64(2.1338718575425446) - rollout_corr/rollout_log_ppl:np.float64(0.7430044782813638) - rollout_corr/log_ppl_diff:np.float64(0.0005318876355886459) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011775929015129805) - rollout_corr/log_ppl_diff_max:np.float64(0.011757910251617432) - rollout_corr/log_ppl_diff_min:np.float64(-0.0038299560546875) - rollout_corr/ppl_ratio:np.float64(1.0005332818254828) - rollout_corr/chi2_token:np.float64(0.00199099606834352) - rollout_corr/chi2_seq:np.float64(1.2359290518797934) - actor/pg_loss:np.float64(-0.00026974163483828306) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014913909058122954) - actor/ppo_kl:np.float64(-6.670435310951461e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.24813814461231232) - perf/mfu/actor:np.float64(0.12164250497550971) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.1574478149414) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:0 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006872585508972406 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006872585508972406 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:710.83203125 - response_length/max:1348.0 - response_length/min:200.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:710.83203125 - response_length_non_aborted/max:1348.0 - response_length_non_aborted/min:200.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.8125 - prompt_length/max:418.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015538372099399567 - timing_s/agent_loop/generate_sequences/min:np.float64(2.5202769823372364) - timing_s/agent_loop/generate_sequences/max:np.float64(13.002393383532763) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.453353088778385) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.002393383532763) - timing_s/a
(TaskRunner pid=2224264) 
Training Progress:  49%|████▉     | 49/100 [45:11<37:52, 44.56s/it]
(TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50}
(TaskRunner pid=2224264) validation generation end
(TaskRunner pid=2224264) [prompt] system
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2224264) <reasoning>
(TaskRunner pid=2224264) ...
(TaskRunner pid=2224264) </reasoning>
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) ...
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) A
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) user
(TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2224264) Please reason step by step.
(TaskRunner pid=2224264) assistant
(TaskRunner pid=2224264) <think>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) </think>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) [response] <reasoning>
(TaskRunner pid=2224264) We are given a product molecule:  
(TaskRunner pid=2224264) **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"**  
(TaskRunner pid=2224264) This is a **nitrogen-containing heterocyclic compound** (likely a pyrrole or similar), with a substituent on the nitrogen:  
(TaskRunner pid=2224264) - A **benzene ring** with a **chloro group** on one carbon  
(TaskRunner pid=2224264) - A **trifluoromethyl group** (C(F)(F)F) on another carbon  
(TaskRunner pid=2224264) - A **nitrile group** (the N in the ring is bonded to a carbon chain)
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) Let's break it down:
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Product structure:
(TaskRunner pid=2224264) - The core is a **pyrrole-like ring** (N in ring)
(TaskRunner pid=2224264) - Attached to the N is a **benzene ring** with:
(TaskRunner pid=2224264)   - A **chloro group** (Cl) on one carbon
(TaskRunner pid=2224264)   - A **trifluoromethyl group** (C(F)(F)F) on another carbon
(TaskRunner pid=2224264) - The ring also has a **nitrile group** (C≡N)
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) We are to determine which of the four options is the **only correct reactant** in the synthesis of this product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) Let’s analyze each option:
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### **Option A:**
(TaskRunner pid=2224264) `O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1`
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) - Contains a **nitrile group** (C≡N)
(TaskRunner pid=2224264) - Contains a **benzene ring** with Cl and C(F)(F)F
(TaskRunner pid=2224264) - The structure matches the **product** closely
(TaskRunner pid=2224264) - The **nitrogen ring** is connected to the same substituents
(TaskRunner pid=2224264) - Seems to be a **correct structure** for a possible reactant
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### **Option B:**
(TaskRunner pid=2224264) `O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1`
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) - Contains a **Cl group** on the ring
(TaskRunner pid=2224264) - The ring has **nitrogen** and **chlorine**
(TaskRunner pid=2224264) - However, the **trifluoromethyl group** is missing
(TaskRunner pid=2224264) - Structure does **not match** the product
(TaskRunner pid=2224264) - Likely incorrect
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### **Option C:**
(TaskRunner pid=2224264) `O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1`
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) - Contains a **benzene ring** with Cl and C(F)(F)F
(TaskRunner pid=2224264) - Contains **nitrogen and oxygen** in the ring
(TaskRunner pid=2224264) - However, the **nitrile group** is not present
(TaskRunner pid=2224264) - The **structure is not the same** as the product
(TaskRunner pid=2224264) - Likely incorrect
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### **Option D:**
(TaskRunner pid=2224264) `O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F`
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) - Contains a **trifluoromethyl group**
(TaskRunner pid=2224264) - Contains **double bonds and rings**
(TaskRunner pid=2224264) - But **no benzene ring with Cl**
(TaskRunner pid=2224264) - No **nitrogen ring** with the correct substituents
(TaskRunner pid=2224264) - Does not match the product
(TaskRunner pid=2224264) - Incorrect
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### **Conclusion:**
(TaskRunner pid=2224264) Only **Option A** contains all the necessary **functional groups and substituents** to be the correct reactant in the synthesis of the given product. It matches the **structure and substituents** of the product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) </reasoning>
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) A
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) [ground_truth] A
(TaskRunner pid=2224264) [score] 1.0
(TaskRunner pid=2224264) [acc] 1.0
(TaskRunner pid=2224264) [pred] A
(TaskRunner pid=2224264) [incorrect_format] 1
(TaskRunner pid=2224264) [feedback] 
(TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50
(TaskRunner pid=2224264) step:50 - global_seqlen/min:25774 - global_seqlen/max:40010 - global_seqlen/minmax_diff:14236 - global_seqlen/balanced_min:30199 - global_seqlen/balanced_max:30236 - global_seqlen/mean:30221.125 - actor/entropy:0.6831239461898804 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18094828724861145 - training/rollout_probs_diff_mean:0.00435651233419776 - training/rollout_probs_diff_std:0.008434131741523743 - training/rollout_actor_probs_pearson_corr:0.9996002316474915 - rollout_corr/rollout_is_mean:0.9999532699584961 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.877063358639134e-06 - rollout_corr/rollout_is_max:1.8264771699905396 - rollout_corr/rollout_is_min:0.4736720621585846 - rollout_corr/rollout_is_std:0.03403113782405853 - rollout_corr/rollout_is_eff_sample_size:0.998842864637548 - rollout_corr/rollout_is_seq_mean:0.9999036192893982 - rollout_corr/rollout_is_seq_std:0.0016001641051843762 - rollout_corr/rollout_is_seq_max:1.0056898593902588 - rollout_corr/rollout_is_seq_min:0.9943985939025879 - rollout_corr/rollout_is_seq_max_deviation:0.005689859390258789 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0370463258586824) - rollout_corr/training_log_ppl:np.float64(0.6895827453117818) - rollout_corr/kl:np.float64(0.0007133520028830276) - rollout_corr/k3_kl:np.float64(0.0006415439404463541) - rollout_corr/rollout_ppl:np.float64(2.035565957427025) - rollout_corr/rollout_log_ppl:np.float64(0.6888693935470656) - rollout_corr/log_ppl_diff:np.float64(0.000713351764716208) - rollout_corr/log_ppl_abs_diff:np.float64(0.001375065534375608) - rollout_corr/log_ppl_diff_max:np.float64(0.010087192058563232) - rollout_corr/log_ppl_diff_min:np.float64(-0.004953622817993164) - rollout_corr/ppl_ratio:np.float64(1.0007150897290558) - rollout_corr/chi2_token:np.float64(0.0011441940441727638) - rollout_corr/chi2_seq:np.float64(1.4973946234676987) - actor/pg_loss:np.float64(0.00014200282748788595) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030153528336995805) - actor/ppo_kl:np.float64(1.1964239081230232e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3060915023088455) - perf/mfu/actor:np.float64(0.11810264114100524) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.11758041381836) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6223214285714286) - val-aux/sciknoweval/reward/std@16:np.float64(0.2647595134884485) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7291095238095238) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.19877560636838412) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5156142857142857) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.24700525701136392) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6215809523809525) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2629386719576279) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8044857142857144) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.12973275443729954) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4167428571428572) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.20684059335831737) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6544095238095238) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.20967583799172174) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8527809523809524) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07656020696044112) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.3329095238095238) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.16042430484607695) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6712285714285714) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.15275032646284742) - val-aux/sciknoweval/reward/best@16/mean:np.float64(
(TaskRunner pid=2224264) 
Training Progress:  50%|█████     | 50/100 [49:14<1:26:40, 104.01s/it]
(TaskRunner pid=2224264) step:51 - global_seqlen/min:30736 - global_seqlen/max:37371 - global_seqlen/minmax_diff:6635 - global_seqlen/balanced_min:34292 - global_seqlen/balanced_max:34302 - global_seqlen/mean:34297.75 - actor/entropy:0.762580394744873 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2294635772705078 - training/rollout_probs_diff_mean:0.004454846493899822 - training/rollout_probs_diff_std:0.008279886096715927 - training/rollout_actor_probs_pearson_corr:0.9996731281280518 - rollout_corr/rollout_is_mean:1.0000529289245605 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.811339259147644 - rollout_corr/rollout_is_min:0.5012558698654175 - rollout_corr/rollout_is_std:0.03488221764564514 - rollout_corr/rollout_is_eff_sample_size:0.9987848284293472 - rollout_corr/rollout_is_seq_mean:1.0000650882720947 - rollout_corr/rollout_is_seq_std:0.0014846812700852752 - rollout_corr/rollout_is_seq_max:1.0048807859420776 - rollout_corr/rollout_is_seq_min:0.9954461455345154 - rollout_corr/rollout_is_seq_max_deviation:0.004880785942077637 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.208350053522736) - rollout_corr/training_log_ppl:np.float64(0.7750773588195443) - rollout_corr/kl:np.float64(0.0005686312180488429) - rollout_corr/k3_kl:np.float64(0.0006700464483628821) - rollout_corr/rollout_ppl:np.float64(2.207024629227817) - rollout_corr/rollout_log_ppl:np.float64(0.7745087213115767) - rollout_corr/log_ppl_diff:np.float64(0.0005686375079676509) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012317007640376687) - rollout_corr/log_ppl_diff_max:np.float64(0.006791532039642334) - rollout_corr/log_ppl_diff_min:np.float64(-0.006045103073120117) - rollout_corr/ppl_ratio:np.float64(1.000570009695366) - rollout_corr/chi2_token:np.float64(0.0015524374321103096) - rollout_corr/chi2_seq:np.float64(2.341128587955609) - actor/pg_loss:np.float64(-0.00025634258054196835) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016806365397314948) - actor/ppo_kl:np.float64(7.1047582039796e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.24387626722455025) - perf/mfu/actor:np.float64(0.13125870218702868) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.71975231170654) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0033099849242717028 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0033099849242717028 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:759.2734375 - response_length/max:1708.0 - response_length/min:153.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:759.2734375 - response_length_non_aborted/max:1708.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.53125 - prompt_length/max:608.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.316764771938324e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.95208290964365) - timing_s/agent_loop/generate_sequences/max:np.float64(15.170884735882282) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.576293181155052) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.170884735882282) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0
(TaskRunner pid=2224264) 
Training Progress:  51%|█████     | 51/100 [49:54<1:09:09, 84.69s/it] 
(TaskRunner pid=2224264) step:52 - global_seqlen/min:25163 - global_seqlen/max:39354 - global_seqlen/minmax_diff:14191 - global_seqlen/balanced_min:31304 - global_seqlen/balanced_max:31326 - global_seqlen/mean:31315.625 - actor/entropy:0.7275306582450867 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1846359670162201 - training/rollout_probs_diff_mean:0.0044688270427286625 - training/rollout_probs_diff_std:0.008479300886392593 - training/rollout_actor_probs_pearson_corr:0.9996131658554077 - rollout_corr/rollout_is_mean:0.9999827146530151 - rollout_corr/rollout_is_ratio_fraction_high:5.457889528770465e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.091577905754093e-05 - rollout_corr/rollout_is_max:2.513453245162964 - rollout_corr/rollout_is_min:0.2490382194519043 - rollout_corr/rollout_is_std:0.03466796875 - rollout_corr/rollout_is_eff_sample_size:0.9987993368491189 - rollout_corr/rollout_is_seq_mean:0.9999496936798096 - rollout_corr/rollout_is_seq_std:0.0013713843654841185 - rollout_corr/rollout_is_seq_max:1.0041810274124146 - rollout_corr/rollout_is_seq_min:0.9949362277984619 - rollout_corr/rollout_is_seq_max_deviation:0.005063772201538086 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0971961123868823) - rollout_corr/training_log_ppl:np.float64(0.7249851016094908) - rollout_corr/kl:np.float64(0.000624122314107467) - rollout_corr/k3_kl:np.float64(0.0006732126169026742) - rollout_corr/rollout_ppl:np.float64(2.095865353010595) - rollout_corr/rollout_log_ppl:np.float64(0.7243609796278179) - rollout_corr/log_ppl_diff:np.float64(0.0006241219816729426) - rollout_corr/log_ppl_abs_diff:np.float64(0.001236821641214192) - rollout_corr/log_ppl_diff_max:np.float64(0.00571441650390625) - rollout_corr/log_ppl_diff_min:np.float64(-0.005221307277679443) - rollout_corr/ppl_ratio:np.float64(1.0006253765895963) - rollout_corr/chi2_token:np.float64(0.0014572420623153448) - rollout_corr/chi2_seq:np.float64(2.134200472617522) - actor/pg_loss:np.float64(0.00022022065240889788) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021542626041082258) - actor/ppo_kl:np.float64(-4.695473637639225e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.25628167018294334) - perf/mfu/actor:np.float64(0.1214559238638076) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.17374038696289) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:0 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011097254231572151 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011097254231572151 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:715.70703125 - response_length/max:1611.0 - response_length/min:184.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:715.70703125 - response_length_non_aborted/max:1611.0 - response_length_non_aborted/min:184.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.90625 - prompt_length/max:453.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.15229320526123e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3523097801953554) - timing_s/agent_loop/generate_sequences/max:np.float64(14.437729807570577) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.303848396521062) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.437729807570577) - timing_s/agent_loop/
(TaskRunner pid=2224264) 
Training Progress:  52%|█████▏    | 52/100 [50:32<56:38, 70.80s/it]  
(TaskRunner pid=2224264) step:53 - global_seqlen/min:29023 - global_seqlen/max:38170 - global_seqlen/minmax_diff:9147 - global_seqlen/balanced_min:32089 - global_seqlen/balanced_max:32125 - global_seqlen/mean:32112.5 - actor/entropy:0.7673122882843018 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2821834683418274 - training/rollout_probs_diff_mean:0.004418913275003433 - training/rollout_probs_diff_std:0.008327017538249493 - training/rollout_actor_probs_pearson_corr:0.9996277093887329 - rollout_corr/rollout_is_mean:1.0000674724578857 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6528507471084595 - rollout_corr/rollout_is_min:0.6141930818557739 - rollout_corr/rollout_is_std:0.035042472183704376 - rollout_corr/rollout_is_eff_sample_size:0.9987737690154207 - rollout_corr/rollout_is_seq_mean:1.0000889301300049 - rollout_corr/rollout_is_seq_std:0.0013838375452905893 - rollout_corr/rollout_is_seq_max:1.005210518836975 - rollout_corr/rollout_is_seq_min:0.9962599873542786 - rollout_corr/rollout_is_seq_max_deviation:0.005210518836975098 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2401722352951765) - rollout_corr/training_log_ppl:np.float64(0.7886160378111526) - rollout_corr/kl:np.float64(0.0006792797128305494) - rollout_corr/k3_kl:np.float64(0.0006657197256458858) - rollout_corr/rollout_ppl:np.float64(2.238586582709104) - rollout_corr/rollout_log_ppl:np.float64(0.7879367553978227) - rollout_corr/log_ppl_diff:np.float64(0.0006792824133299291) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012466523912735283) - rollout_corr/log_ppl_diff_max:np.float64(0.0049315690994262695) - rollout_corr/log_ppl_diff_min:np.float64(-0.004511535167694092) - rollout_corr/ppl_ratio:np.float64(1.000680540688336) - rollout_corr/chi2_token:np.float64(0.00131767219863832) - rollout_corr/chi2_seq:np.float64(0.9375523473136127) - actor/pg_loss:np.float64(-5.056627560406923e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015928275297483196) - actor/ppo_kl:np.float64(0.00014284484250026708) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2610822059214115) - perf/mfu/actor:np.float64(0.12400191776249886) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.9232177734375) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:0 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009051043540239334 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009051043540239334 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:707.953125 - response_length/max:1922.0 - response_length/min:129.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:707.953125 - response_length_non_aborted/max:1922.0 - response_length_non_aborted/min:129.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.5625 - prompt_length/max:532.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.531714022159576e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6737753115594387) - timing_s/agent_loop/generate_sequences/max:np.float64(16.309401037171483) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.246375748174614) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.309401037171483) - timing_s/agent_loop/slowest/tool_calls:np.float6
(TaskRunner pid=2224264) 
Training Progress:  53%|█████▎    | 53/100 [51:12<48:15, 61.61s/it]
(TaskRunner pid=2224264) step:54 - global_seqlen/min:24046 - global_seqlen/max:37658 - global_seqlen/minmax_diff:13612 - global_seqlen/balanced_min:31667 - global_seqlen/balanced_max:31732 - global_seqlen/mean:31711.0 - actor/entropy:0.7221550345420837 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4564734995365143 - training/rollout_probs_diff_mean:0.0042567262426018715 - training/rollout_probs_diff_std:0.008278296329081059 - training/rollout_actor_probs_pearson_corr:0.9996740221977234 - rollout_corr/rollout_is_mean:0.9999973773956299 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.423934908321826e-06 - rollout_corr/rollout_is_max:1.8484896421432495 - rollout_corr/rollout_is_min:0.4127388894557953 - rollout_corr/rollout_is_std:0.03383616358041763 - rollout_corr/rollout_is_eff_sample_size:0.9988563043114053 - rollout_corr/rollout_is_seq_mean:0.9999293088912964 - rollout_corr/rollout_is_seq_std:0.0013600483071058989 - rollout_corr/rollout_is_seq_max:1.003616213798523 - rollout_corr/rollout_is_seq_min:0.9944389462471008 - rollout_corr/rollout_is_seq_max_deviation:0.00556105375289917 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1455902410671115) - rollout_corr/training_log_ppl:np.float64(0.7426831858465448) - rollout_corr/kl:np.float64(0.0007347915104665148) - rollout_corr/k3_kl:np.float64(0.0006544318975443275) - rollout_corr/rollout_ppl:np.float64(2.14404917974025) - rollout_corr/rollout_log_ppl:np.float64(0.7419483949197456) - rollout_corr/log_ppl_diff:np.float64(0.0007347909267991781) - rollout_corr/log_ppl_abs_diff:np.float64(0.00128568010404706) - rollout_corr/log_ppl_diff_max:np.float64(0.007453739643096924) - rollout_corr/log_ppl_diff_min:np.float64(-0.0039052367210388184) - rollout_corr/ppl_ratio:np.float64(1.000736182089895) - rollout_corr/chi2_token:np.float64(0.0011647003702819347) - rollout_corr/chi2_seq:np.float64(1.3221846474334598) - actor/pg_loss:np.float64(-0.0001485362881794572) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030488177571896813) - actor/ppo_kl:np.float64(6.0147084293804554e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3080219365656376) - perf/mfu/actor:np.float64(0.12202071329811175) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.63494873046875) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.016614189371466637 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.016614189371466637 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:720.1875 - response_length/max:2286.0 - response_length/min:162.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:720.1875 - response_length_non_aborted/max:2286.0 - response_length_non_aborted/min:162.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.78125 - prompt_length/max:597.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.760074317455292e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9534575715661049) - timing_s/agent_loop/generate_sequences/max:np.float64(17.89197232015431) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.935718771463144) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.89197232015431) - timing_s/agent_loop/slowest/tool_calls:np
(TaskRunner pid=2224264) 
Training Progress:  54%|█████▍    | 54/100 [51:55<42:51, 55.91s/it]
(TaskRunner pid=2224264) step:55 - global_seqlen/min:29461 - global_seqlen/max:36701 - global_seqlen/minmax_diff:7240 - global_seqlen/balanced_min:33720 - global_seqlen/balanced_max:34509 - global_seqlen/mean:33827.375 - actor/entropy:0.8024973273277283 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23547464609146118 - training/rollout_probs_diff_mean:0.0043955049477517605 - training/rollout_probs_diff_std:0.008106191642582417 - training/rollout_actor_probs_pearson_corr:0.999698281288147 - rollout_corr/rollout_is_mean:1.0000178813934326 - rollout_corr/rollout_is_ratio_fraction_high:5.190302545088343e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0640006065368652 - rollout_corr/rollout_is_min:0.5909958481788635 - rollout_corr/rollout_is_std:0.03487367555499077 - rollout_corr/rollout_is_eff_sample_size:0.9987854230284318 - rollout_corr/rollout_is_seq_mean:0.999980092048645 - rollout_corr/rollout_is_seq_std:0.0013782148016616702 - rollout_corr/rollout_is_seq_max:1.0047253370285034 - rollout_corr/rollout_is_seq_min:0.9960294961929321 - rollout_corr/rollout_is_seq_max_deviation:0.004725337028503418 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.3131857896223664) - rollout_corr/training_log_ppl:np.float64(0.8200820172205567) - rollout_corr/kl:np.float64(0.0007159894126873567) - rollout_corr/k3_kl:np.float64(0.0006568527235231159) - rollout_corr/rollout_ppl:np.float64(2.311495006084442) - rollout_corr/rollout_log_ppl:np.float64(0.8193660224787891) - rollout_corr/log_ppl_diff:np.float64(0.0007159947417676449) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012078655418008566) - rollout_corr/log_ppl_diff_max:np.float64(0.004928708076477051) - rollout_corr/log_ppl_diff_min:np.float64(-0.0037838220596313477) - rollout_corr/ppl_ratio:np.float64(1.0007172094192356) - rollout_corr/chi2_token:np.float64(0.0011943182907998562) - rollout_corr/chi2_seq:np.float64(2.1422022855840623) - actor/pg_loss:np.float64(6.403832230716944e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014861452268633002) - actor/ppo_kl:np.float64(7.01219851411139e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2634842284023762) - perf/mfu/actor:np.float64(0.12720097629374041) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.60932922363281) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:0 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.000431443884735927 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.000431443884735927 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:752.60546875 - response_length/max:2800.0 - response_length/min:187.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:752.60546875 - response_length_non_aborted/max:2800.0 - response_length_non_aborted/min:187.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:304.5 - prompt_length/max:625.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.310990571975708e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7985909432172775) - timing_s/agent_loop/generate_sequences/max:np.float64(20.643996415659785) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.059685343854653) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.643996415659785) - timing_s/agent_loop/slowest/
(TaskRunner pid=2224264) 
Training Progress:  55%|█████▌    | 55/100 [52:40<39:32, 52.73s/it]
(TaskRunner pid=2224264) step:56 - global_seqlen/min:25792 - global_seqlen/max:39171 - global_seqlen/minmax_diff:13379 - global_seqlen/balanced_min:31985 - global_seqlen/balanced_max:31996 - global_seqlen/mean:31991.875 - actor/entropy:0.8143277764320374 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2929895520210266 - training/rollout_probs_diff_mean:0.004465053323656321 - training/rollout_probs_diff_std:0.008296534419059753 - training/rollout_actor_probs_pearson_corr:0.9996805787086487 - rollout_corr/rollout_is_mean:0.9999870657920837 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.5952778994687833e-05 - rollout_corr/rollout_is_max:1.629705786705017 - rollout_corr/rollout_is_min:0.2926250994205475 - rollout_corr/rollout_is_std:0.034940268844366074 - rollout_corr/rollout_is_eff_sample_size:0.9987806662555786 - rollout_corr/rollout_is_seq_mean:1.0000478029251099 - rollout_corr/rollout_is_seq_std:0.0013941883808001876 - rollout_corr/rollout_is_seq_max:1.0048588514328003 - rollout_corr/rollout_is_seq_min:0.9960804581642151 - rollout_corr/rollout_is_seq_max_deviation:0.004858851432800293 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.281407651025802) - rollout_corr/training_log_ppl:np.float64(0.804402201261837) - rollout_corr/kl:np.float64(0.0005219981214015235) - rollout_corr/k3_kl:np.float64(0.0006556217045954327) - rollout_corr/rollout_ppl:np.float64(2.2802593959495425) - rollout_corr/rollout_log_ppl:np.float64(0.803880195133388) - rollout_corr/log_ppl_diff:np.float64(0.000522006128448993) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012433496885932982) - rollout_corr/log_ppl_diff_max:np.float64(0.0066702961921691895) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035393238067626953) - rollout_corr/ppl_ratio:np.float64(1.0005232789553702) - rollout_corr/chi2_token:np.float64(0.0015740657690912485) - rollout_corr/chi2_seq:np.float64(5.73605796508491) - actor/pg_loss:np.float64(-4.861317574977875e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018069532825393253) - actor/ppo_kl:np.float64(-5.679699522431747e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2360125184059143) - perf/mfu/actor:np.float64(0.12372654611791109) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.54317474365234) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002341070445254445 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002341070445254445 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:734.58984375 - response_length/max:1632.0 - response_length/min:182.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:734.58984375 - response_length_non_aborted/max:1632.0 - response_length_non_aborted/min:182.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.15625 - prompt_length/max:458.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.9824674129486084e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.308333108201623) - timing_s/agent_loop/generate_sequences/max:np.float64(14.604689434170723) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.345092790725175) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.604689434170723) - timing_s/agent_loop/slo
(TaskRunner pid=2224264) 
Training Progress:  56%|█████▌    | 56/100 [53:19<35:31, 48.45s/it]
(TaskRunner pid=2224264) step:57 - global_seqlen/min:27687 - global_seqlen/max:34593 - global_seqlen/minmax_diff:6906 - global_seqlen/balanced_min:31298 - global_seqlen/balanced_max:31311 - global_seqlen/mean:31305.125 - actor/entropy:0.7938913106918335 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.680421769618988 - training/rollout_probs_diff_mean:0.004562178626656532 - training/rollout_probs_diff_std:0.008708939887583256 - training/rollout_actor_probs_pearson_corr:0.9995999932289124 - rollout_corr/rollout_is_mean:0.9999580979347229 - rollout_corr/rollout_is_ratio_fraction_high:5.648919795930851e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.6946758478297852e-05 - rollout_corr/rollout_is_max:2.2787086963653564 - rollout_corr/rollout_is_min:0.24652595818042755 - rollout_corr/rollout_is_std:0.035597678273916245 - rollout_corr/rollout_is_eff_sample_size:0.9987344090023405 - rollout_corr/rollout_is_seq_mean:0.9999576807022095 - rollout_corr/rollout_is_seq_std:0.001491630100645125 - rollout_corr/rollout_is_seq_max:1.0051113367080688 - rollout_corr/rollout_is_seq_min:0.9956648349761963 - rollout_corr/rollout_is_seq_max_deviation:0.005111336708068848 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.283419963438064) - rollout_corr/training_log_ppl:np.float64(0.8058918532915413) - rollout_corr/kl:np.float64(0.0006797224436070337) - rollout_corr/k3_kl:np.float64(0.0007055759461991329) - rollout_corr/rollout_ppl:np.float64(2.281822293996811) - rollout_corr/rollout_log_ppl:np.float64(0.80521212273743) - rollout_corr/log_ppl_diff:np.float64(0.0006797305541113019) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013461158378049731) - rollout_corr/log_ppl_diff_max:np.float64(0.0058408379554748535) - rollout_corr/log_ppl_diff_min:np.float64(-0.004617094993591309) - rollout_corr/ppl_ratio:np.float64(1.0006813008803874) - rollout_corr/chi2_token:np.float64(0.0014507242012768984) - rollout_corr/chi2_seq:np.float64(1.6866442584432662) - actor/pg_loss:np.float64(1.2409058399498463e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002567762589933409) - actor/ppo_kl:np.float64(-1.4622721524304438e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2836088426411152) - perf/mfu/actor:np.float64(0.12280457724250629) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.51215362548828) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:0 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012533540138974786 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0012533540138974786 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:691.50390625 - response_length/max:1421.0 - response_length/min:166.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:691.50390625 - response_length_non_aborted/max:1421.0 - response_length_non_aborted/min:166.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.78125 - prompt_length/max:660.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012706033885478973 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5593209099024534) - timing_s/agent_loop/generate_sequences/max:np.float64(12.716230429708958) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.830025155824842) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.716230429708958) - ti
(TaskRunner pid=2224264) 
Training Progress:  57%|█████▋    | 57/100 [53:56<32:14, 44.98s/it]
(TaskRunner pid=2224264) step:58 - global_seqlen/min:26170 - global_seqlen/max:34255 - global_seqlen/minmax_diff:8085 - global_seqlen/balanced_min:30104 - global_seqlen/balanced_max:30420 - global_seqlen/mean:30161.5 - actor/entropy:0.7656847238540649 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.16260474920272827 - training/rollout_probs_diff_mean:0.00442152377218008 - training/rollout_probs_diff_std:0.00836502481251955 - training/rollout_actor_probs_pearson_corr:0.9997066259384155 - rollout_corr/rollout_is_mean:0.9998526573181152 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.867856089025736e-06 - rollout_corr/rollout_is_max:1.6352856159210205 - rollout_corr/rollout_is_min:0.49183452129364014 - rollout_corr/rollout_is_std:0.034748680889606476 - rollout_corr/rollout_is_eff_sample_size:0.9987937474899505 - rollout_corr/rollout_is_seq_mean:0.9999055862426758 - rollout_corr/rollout_is_seq_std:0.0014970327029004693 - rollout_corr/rollout_is_seq_max:1.0064585208892822 - rollout_corr/rollout_is_seq_min:0.9960758090019226 - rollout_corr/rollout_is_seq_max_deviation:0.0064585208892822266 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.18733112514019) - rollout_corr/training_log_ppl:np.float64(0.7628000770928338) - rollout_corr/kl:np.float64(0.0007255446162326429) - rollout_corr/k3_kl:np.float64(0.0006535338600883733) - rollout_corr/rollout_ppl:np.float64(2.185690295882523) - rollout_corr/rollout_log_ppl:np.float64(0.7620745294261724) - rollout_corr/log_ppl_diff:np.float64(0.0007255476666614413) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013074638554826379) - rollout_corr/log_ppl_diff_max:np.float64(0.005662679672241211) - rollout_corr/log_ppl_diff_min:np.float64(-0.00701594352722168) - rollout_corr/ppl_ratio:np.float64(1.000726986443624) - rollout_corr/chi2_token:np.float64(0.0011695893481373787) - rollout_corr/chi2_seq:np.float64(1.832958000479266) - actor/pg_loss:np.float64(1.464621163904667e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012746554830300738) - actor/ppo_kl:np.float64(9.933237482329105e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2319943569600582) - perf/mfu/actor:np.float64(0.11497644796674522) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.57088470458984) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:0 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002008273731917143 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002008273731917143 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:665.703125 - response_length/max:2537.0 - response_length/min:155.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:665.703125 - response_length_non_aborted/max:2537.0 - response_length_non_aborted/min:155.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.84375 - prompt_length/max:598.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001398194581270218 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0110350400209427) - timing_s/agent_loop/generate_sequences/max:np.float64(19.426406839862466) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.346314342365076) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.426406839862466) - timing_s/agent_loop/slowest/too
(TaskRunner pid=2224264) 
Training Progress:  58%|█████▊    | 58/100 [54:39<31:11, 44.55s/it]
(TaskRunner pid=2224264) step:59 - global_seqlen/min:27515 - global_seqlen/max:33648 - global_seqlen/minmax_diff:6133 - global_seqlen/balanced_min:30606 - global_seqlen/balanced_max:31185 - global_seqlen/mean:30683.625 - actor/entropy:0.7695986032485962 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4610300064086914 - training/rollout_probs_diff_mean:0.004400559701025486 - training/rollout_probs_diff_std:0.008376654237508774 - training/rollout_actor_probs_pearson_corr:0.9996274709701538 - rollout_corr/rollout_is_mean:1.0000059604644775 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.658765076077543e-06 - rollout_corr/rollout_is_max:1.8190242052078247 - rollout_corr/rollout_is_min:0.3681882917881012 - rollout_corr/rollout_is_std:0.03469890356063843 - rollout_corr/rollout_is_eff_sample_size:0.9987974340815278 - rollout_corr/rollout_is_seq_mean:0.9999397397041321 - rollout_corr/rollout_is_seq_std:0.0016430045943707228 - rollout_corr/rollout_is_seq_max:1.0059504508972168 - rollout_corr/rollout_is_seq_min:0.9930161237716675 - rollout_corr/rollout_is_seq_max_deviation:0.0069838762283325195 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2372520426288247) - rollout_corr/training_log_ppl:np.float64(0.7862228258745745) - rollout_corr/kl:np.float64(0.0008618131305837551) - rollout_corr/k3_kl:np.float64(0.0006665370408427407) - rollout_corr/rollout_ppl:np.float64(2.2352824225090444) - rollout_corr/rollout_log_ppl:np.float64(0.7853610130259767) - rollout_corr/log_ppl_diff:np.float64(0.000861812848597765) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014634081162512302) - rollout_corr/log_ppl_diff_max:np.float64(0.008165895938873291) - rollout_corr/log_ppl_diff_min:np.float64(-0.005269348621368408) - rollout_corr/ppl_ratio:np.float64(1.0008637136779726) - rollout_corr/chi2_token:np.float64(0.000947984866797924) - rollout_corr/chi2_seq:np.float64(1.44492331543006) - actor/pg_loss:np.float64(-2.2010761313140392e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016003822292987024) - actor/ppo_kl:np.float64(0.00017181198663673847) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20559313893318176) - perf/mfu/actor:np.float64(0.11867857354144504) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.51495361328125) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:0 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0009747223230078816 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0009747223230078816 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:690.30078125 - response_length/max:2398.0 - response_length/min:212.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:690.30078125 - response_length_non_aborted/max:2398.0 - response_length_non_aborted/min:212.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.5625 - prompt_length/max:615.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012865476310253143 - timing_s/agent_loop/generate_sequences/min:np.float64(2.7066904306411743) - timing_s/agent_loop/generate_sequences/max:np.float64(18.255667094141245) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.844594236710691) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.255667094141245) - timing_s/agent_loop/
(TaskRunner pid=2224264) 
Training Progress:  59%|█████▉    | 59/100 [55:21<29:58, 43.87s/it]
(TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60}
(TaskRunner pid=2224264) validation generation end
(TaskRunner pid=2224264) [prompt] system
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2224264) <reasoning>
(TaskRunner pid=2224264) ...
(TaskRunner pid=2224264) </reasoning>
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) ...
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) A
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) user
(TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2224264) Please reason step by step.
(TaskRunner pid=2224264) assistant
(TaskRunner pid=2224264) <think>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) </think>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) [response] <reasoning>
(TaskRunner pid=2224264) We are given a product molecule:  
(TaskRunner pid=2224264) **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This is a condensed chemical formula. Let's interpret it step by step.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Step 1: Parse the product formula
(TaskRunner pid=2224264) The formula:  
(TaskRunner pid=2224264) **Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This is a shorthand for a benzene ring with substituents. Let's break it down:
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) - **Nc1ccn(...)**: This is a nitrogen-containing ring.
(TaskRunner pid=2224264) - **Cc2ccc(Cl)cc2C(F)(F)F**: This is a side chain attached to the ring via the nitrogen. It contains a benzene ring with a Cl substituent and a trifluoromethyl group (C(F)(F)F).
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) So the product is likely a **nitrogen-containing heterocycle (e.g., pyridine or pyrrole)**, with a **benzene ring bearing a Cl and a CF₃ group** attached to it.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Step 2: Understand what we are looking for
(TaskRunner pid=2224264) We are to find the correct **reactant** that would be used in the synthesis of this product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) The options are all chemical formulas that may be reactants in a synthesis. We must find the one that matches the **structural features of the product**.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Step 3: Analyze each option
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) #### Option A:
(TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This is a **diacyl compound** (two C=O groups), and it includes a **nitrogen ring** with the same side chain as the product:  
(TaskRunner pid=2224264) - `c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1`  
(TaskRunner pid=2224264) This matches the **side chain** of the product.  
(TaskRunner pid=2224264) - The `O=C1c2ccccc2C(=O)N1` part suggests a **diketone ring** (like a benzene ring with two ketone groups), which may be a **starting material** for forming the nitrogen ring.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This **looks like a good candidate** as it contains the same side chain and a structure that could be used to form the product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) #### Option B:
(TaskRunner pid=2224264) **O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This includes a Cl group on the ring and a nitrogen, but it **does not match the side chain** of the product. It also has a different structure and is not consistent with the product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) #### Option C:
(TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This has a **trifluoromethyl group** and a **Cl group**, but the **side chain is different**. It has a **hydroxyl group** that is not in the product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) #### Option D:
(TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This has a **CF₃ group** and a **benzene ring**, but the **side chain is different**. It does not match the structure of the product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Step 4: Final determination
(TaskRunner pid=2224264) Only **Option A** includes the **same side chain** as the product and includes a **ring structure** that could be used in the synthesis of the nitrogen-containing ring in the product. It is the only one that matches the **structural features** of the product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) </reasoning>
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) A
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) [ground_truth] A
(TaskRunner pid=2224264) [score] 1.0
(TaskRunner pid=2224264) [acc] 1.0
(TaskRunner pid=2224264) [pred] A
(TaskRunner pid=2224264) [incorrect_format] 1
(TaskRunner pid=2224264) [feedback] 
(TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60
(TaskRunner pid=2224264) step:60 - global_seqlen/min:28478 - global_seqlen/max:38447 - global_seqlen/minmax_diff:9969 - global_seqlen/balanced_min:32368 - global_seqlen/balanced_max:32372 - global_seqlen/mean:32370.0 - actor/entropy:0.8034039735794067 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4281436800956726 - training/rollout_probs_diff_mean:0.004418870899826288 - training/rollout_probs_diff_std:0.008324537426233292 - training/rollout_actor_probs_pearson_corr:0.9996195435523987 - rollout_corr/rollout_is_mean:1.0001076459884644 - rollout_corr/rollout_is_ratio_fraction_high:1.047778732754523e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.238893663772615e-06 - rollout_corr/rollout_is_max:2.1436619758605957 - rollout_corr/rollout_is_min:0.026059480383992195 - rollout_corr/rollout_is_std:0.0351596400141716 - rollout_corr/rollout_is_eff_sample_size:0.9987655638020635 - rollout_corr/rollout_is_seq_mean:1.0000665187835693 - rollout_corr/rollout_is_seq_std:0.0014459878439083695 - rollout_corr/rollout_is_seq_max:1.005103349685669 - rollout_corr/rollout_is_seq_min:0.9960857629776001 - rollout_corr/rollout_is_seq_max_deviation:0.005103349685668945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2919784113764763) - rollout_corr/training_log_ppl:np.float64(0.806903613905888) - rollout_corr/kl:np.float64(0.0006409502570816628) - rollout_corr/k3_kl:np.float64(0.0006509173431368254) - rollout_corr/rollout_ppl:np.float64(2.290479706134647) - rollout_corr/rollout_log_ppl:np.float64(0.8062626618193462) - rollout_corr/log_ppl_diff:np.float64(0.0006409520865418017) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012293330510146916) - rollout_corr/log_ppl_diff_max:np.float64(0.005213141441345215) - rollout_corr/log_ppl_diff_min:np.float64(-0.004137933254241943) - rollout_corr/ppl_ratio:np.float64(1.0006422218866646) - rollout_corr/chi2_token:np.float64(0.0013303321320563555) - rollout_corr/chi2_seq:np.float64(2.3223254224285483) - actor/pg_loss:np.float64(-0.0001239327248185873) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011024115792679368) - actor/ppo_kl:np.float64(6.963595266995526e-05) - actor/pg_clipfrac_lower:np.float64(2.784212483675219e-06) - actor/grad_norm:np.float64(0.24239107966423035) - perf/mfu/actor:np.float64(0.12592590460533137) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.14717817306519) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6589285714285714) - val-aux/sciknoweval/reward/std@16:np.float64(0.26147699146732906) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7634047619047619) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.19852720557117806) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5552285714285714) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2397040092225637) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6592000000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.26073502970265483) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8421285714285713) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.13617544486794253) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.46132857142857137) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.20013008279957237) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6862619047619049) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.20619876382539531) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8954952380952381) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08197180786429441) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.3817190476190476) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.1582460634427849) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6993809523809524) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.14750004177458542) - val-aux/sciknow
(TaskRunner pid=2224264) 
Training Progress:  60%|██████    | 60/100 [59:23<1:08:47, 103.19s/it]
(TaskRunner pid=2224264) step:61 - global_seqlen/min:26866 - global_seqlen/max:38557 - global_seqlen/minmax_diff:11691 - global_seqlen/balanced_min:32591 - global_seqlen/balanced_max:32615 - global_seqlen/mean:32607.625 - actor/entropy:0.7484701871871948 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.32022327184677124 - training/rollout_probs_diff_mean:0.004327286966145039 - training/rollout_probs_diff_std:0.008308346383273602 - training/rollout_actor_probs_pearson_corr:0.9996346235275269 - rollout_corr/rollout_is_mean:1.0000005960464478 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.5481393347727135e-05 - rollout_corr/rollout_is_max:1.9516299962997437 - rollout_corr/rollout_is_min:0.3159535825252533 - rollout_corr/rollout_is_std:0.03432586044073105 - rollout_corr/rollout_is_eff_sample_size:0.9988231220557214 - rollout_corr/rollout_is_seq_mean:1.0000786781311035 - rollout_corr/rollout_is_seq_std:0.001447982620447874 - rollout_corr/rollout_is_seq_max:1.0040830373764038 - rollout_corr/rollout_is_seq_min:0.9964959025382996 - rollout_corr/rollout_is_seq_max_deviation:0.004083037376403809 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.167487114202231) - rollout_corr/training_log_ppl:np.float64(0.7570779783418402) - rollout_corr/kl:np.float64(0.0007175489161515003) - rollout_corr/k3_kl:np.float64(0.0006208742135527245) - rollout_corr/rollout_ppl:np.float64(2.1659105462022126) - rollout_corr/rollout_log_ppl:np.float64(0.7563604272436351) - rollout_corr/log_ppl_diff:np.float64(0.0007175510982051492) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012418738333508372) - rollout_corr/log_ppl_diff_max:np.float64(0.007266104221343994) - rollout_corr/log_ppl_diff_min:np.float64(-0.003547847270965576) - rollout_corr/ppl_ratio:np.float64(1.0007189356256276) - rollout_corr/chi2_token:np.float64(0.0010508422274142504) - rollout_corr/chi2_seq:np.float64(1.2128563108853996) - actor/pg_loss:np.float64(7.9984194599092e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012339052045717835) - actor/ppo_kl:np.float64(0.0001892128098228696) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21570049226284027) - perf/mfu/actor:np.float64(0.1265814073644025) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.56413173675537) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0024512207601219416 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0024512207601219416 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:756.95703125 - response_length/max:1627.0 - response_length/min:211.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:756.95703125 - response_length_non_aborted/max:1627.0 - response_length_non_aborted/min:211.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.03125 - prompt_length/max:462.0 - prompt_length/min:177.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018923170864582062 - timing_s/agent_loop/generate_sequences/min:np.float64(2.607853963971138) - timing_s/agent_loop/generate_sequences/max:np.float64(14.494692988693714) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.566406984020432) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.494692988693714) - timing_s/agent_loop/sl
(TaskRunner pid=2224264) 
Training Progress:  61%|██████    | 61/100 [1:00:01<54:21, 83.62s/it] 
(TaskRunner pid=2224264) step:62 - global_seqlen/min:27396 - global_seqlen/max:35660 - global_seqlen/minmax_diff:8264 - global_seqlen/balanced_min:31880 - global_seqlen/balanced_max:31902 - global_seqlen/mean:31893.125 - actor/entropy:0.8587515950202942 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2571102976799011 - training/rollout_probs_diff_mean:0.004673279356211424 - training/rollout_probs_diff_std:0.008435208350419998 - training/rollout_actor_probs_pearson_corr:0.9996413588523865 - rollout_corr/rollout_is_mean:0.9998844861984253 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9304524660110474 - rollout_corr/rollout_is_min:0.5585559010505676 - rollout_corr/rollout_is_std:0.03600391373038292 - rollout_corr/rollout_is_eff_sample_size:0.9987051585549779 - rollout_corr/rollout_is_seq_mean:0.9999316930770874 - rollout_corr/rollout_is_seq_std:0.0016182069666683674 - rollout_corr/rollout_is_seq_max:1.0045205354690552 - rollout_corr/rollout_is_seq_min:0.9909459948539734 - rollout_corr/rollout_is_seq_max_deviation:0.009054005146026611 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.4564432627521455) - rollout_corr/training_log_ppl:np.float64(0.8728480438003317) - rollout_corr/kl:np.float64(0.000761049760463095) - rollout_corr/k3_kl:np.float64(0.000696839270858618) - rollout_corr/rollout_ppl:np.float64(2.454547442961484) - rollout_corr/rollout_log_ppl:np.float64(0.8720869926037267) - rollout_corr/log_ppl_diff:np.float64(0.0007610511966049671) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013679289259016514) - rollout_corr/log_ppl_diff_max:np.float64(0.009840637445449829) - rollout_corr/log_ppl_diff_min:np.float64(-0.0033975839614868164) - rollout_corr/ppl_ratio:np.float64(1.0007626954466105) - rollout_corr/chi2_token:np.float64(0.001287585822865367) - rollout_corr/chi2_seq:np.float64(1.209890850353986) - actor/pg_loss:np.float64(-3.718864172697067e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001223060548909416) - actor/ppo_kl:np.float64(2.9713928825003677e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29783353582024574) - perf/mfu/actor:np.float64(0.12234924411478575) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.59233093261719) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:1 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006781507283449173 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006781507283449173 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:697.41015625 - response_length/max:1847.0 - response_length/min:168.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:697.41015625 - response_length_non_aborted/max:1847.0 - response_length_non_aborted/min:168.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.25 - prompt_length/max:662.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017420388758182526 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4971247743815184) - timing_s/agent_loop/generate_sequences/max:np.float64(15.592459442093968) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.807667781009513) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.592459442093968) - timing_s/agent_loop/slowest/tool_calls:np.f
(TaskRunner pid=2224264) 
Training Progress:  62%|██████▏   | 62/100 [1:00:41<44:37, 70.45s/it]
(TaskRunner pid=2224264) step:63 - global_seqlen/min:26919 - global_seqlen/max:40692 - global_seqlen/minmax_diff:13773 - global_seqlen/balanced_min:33436 - global_seqlen/balanced_max:33474 - global_seqlen/mean:33459.75 - actor/entropy:0.7692099809646606 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39975038170814514 - training/rollout_probs_diff_mean:0.0043107387609779835 - training/rollout_probs_diff_std:0.008506524376571178 - training/rollout_actor_probs_pearson_corr:0.9996923804283142 - rollout_corr/rollout_is_mean:0.999955415725708 - rollout_corr/rollout_is_ratio_fraction_high:5.143874204804888e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.143874204804888e-06 - rollout_corr/rollout_is_max:2.5493264198303223 - rollout_corr/rollout_is_min:0.3577776551246643 - rollout_corr/rollout_is_std:0.03481551632285118 - rollout_corr/rollout_is_eff_sample_size:0.9987891095585573 - rollout_corr/rollout_is_seq_mean:1.000030755996704 - rollout_corr/rollout_is_seq_std:0.0014692960539832711 - rollout_corr/rollout_is_seq_max:1.0053269863128662 - rollout_corr/rollout_is_seq_min:0.9961836934089661 - rollout_corr/rollout_is_seq_max_deviation:0.005326986312866211 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.23876611655578) - rollout_corr/training_log_ppl:np.float64(0.7817316737491637) - rollout_corr/kl:np.float64(0.0004539784086303733) - rollout_corr/k3_kl:np.float64(0.0006596663926075053) - rollout_corr/rollout_ppl:np.float64(2.2376751606352627) - rollout_corr/rollout_log_ppl:np.float64(0.781277688452974) - rollout_corr/log_ppl_diff:np.float64(0.000453985296189785) - rollout_corr/log_ppl_abs_diff:np.float64(0.001180113758891821) - rollout_corr/log_ppl_diff_max:np.float64(0.005360126495361328) - rollout_corr/log_ppl_diff_min:np.float64(-0.004343867301940918) - rollout_corr/ppl_ratio:np.float64(1.0004551277961582) - rollout_corr/chi2_token:np.float64(0.001740993233397603) - rollout_corr/chi2_seq:np.float64(1.6487001434434205) - actor/pg_loss:np.float64(-0.00019339460413902998) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018608193454383581) - actor/ppo_kl:np.float64(-0.00013630927301377938) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.25826945155858994) - perf/mfu/actor:np.float64(0.12769393570296111) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.44427490234375) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:1 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002252373844385147 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002252373844385147 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:759.3984375 - response_length/max:1951.0 - response_length/min:194.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:759.3984375 - response_length_non_aborted/max:1951.0 - response_length_non_aborted/min:194.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.21875 - prompt_length/max:532.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010393001139163971 - timing_s/agent_loop/generate_sequences/min:np.float64(2.39320345595479) - timing_s/agent_loop/generate_sequences/max:np.float64(15.86801179125905) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.712352399430529) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.86801179125905) - timing_s/
(TaskRunner pid=2224264) 
Training Progress:  63%|██████▎   | 63/100 [1:01:21<37:53, 61.44s/it]
(TaskRunner pid=2224264) step:64 - global_seqlen/min:26306 - global_seqlen/max:34933 - global_seqlen/minmax_diff:8627 - global_seqlen/balanced_min:30904 - global_seqlen/balanced_max:30933 - global_seqlen/mean:30923.0 - actor/entropy:0.7835981845855713 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3175749182701111 - training/rollout_probs_diff_mean:0.004405513871461153 - training/rollout_probs_diff_std:0.008360867388546467 - training/rollout_actor_probs_pearson_corr:0.9996961355209351 - rollout_corr/rollout_is_mean:0.9999273419380188 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1598237506404985e-05 - rollout_corr/rollout_is_max:1.7908501625061035 - rollout_corr/rollout_is_min:0.2933122217655182 - rollout_corr/rollout_is_std:0.03481380268931389 - rollout_corr/rollout_is_eff_sample_size:0.9987892284793369 - rollout_corr/rollout_is_seq_mean:0.9998888373374939 - rollout_corr/rollout_is_seq_std:0.0014235756825655699 - rollout_corr/rollout_is_seq_max:1.0034641027450562 - rollout_corr/rollout_is_seq_min:0.9946786165237427 - rollout_corr/rollout_is_seq_max_deviation:0.005321383476257324 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.228744763880968) - rollout_corr/training_log_ppl:np.float64(0.7778443787246943) - rollout_corr/kl:np.float64(0.000828439747124321) - rollout_corr/k3_kl:np.float64(0.0006999520201134146) - rollout_corr/rollout_ppl:np.float64(2.2268494172021747) - rollout_corr/rollout_log_ppl:np.float64(0.7770159327192232) - rollout_corr/log_ppl_diff:np.float64(0.0008284460054710507) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013873394345864654) - rollout_corr/log_ppl_diff_max:np.float64(0.007387757301330566) - rollout_corr/log_ppl_diff_min:np.float64(-0.0033279061317443848) - rollout_corr/ppl_ratio:np.float64(1.0008300342597067) - rollout_corr/chi2_token:np.float64(0.0011283555068075657) - rollout_corr/chi2_seq:np.float64(1.5243452661670744) - actor/pg_loss:np.float64(-7.703946903347969e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002916122386977804) - actor/ppo_kl:np.float64(8.220684872384254e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.241146732121706) - perf/mfu/actor:np.float64(0.12080489767203606) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.29822540283203) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:1 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005282997153699398 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005282997153699398 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:673.59375 - response_length/max:1796.0 - response_length/min:176.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:673.59375 - response_length_non_aborted/max:1796.0 - response_length_non_aborted/min:176.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:292.75 - prompt_length/max:495.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012784451246261597 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1317031513899565) - timing_s/agent_loop/generate_sequences/max:np.float64(14.859888169914484) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.550355381361442) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.859888169914484) - timing_s/agent_loop/slowest/tool
(TaskRunner pid=2224264) 
Training Progress:  64%|██████▍   | 64/100 [1:01:59<32:43, 54.53s/it]
(TaskRunner pid=2224264) step:65 - global_seqlen/min:24032 - global_seqlen/max:34111 - global_seqlen/minmax_diff:10079 - global_seqlen/balanced_min:29951 - global_seqlen/balanced_max:30559 - global_seqlen/mean:30032.625 - actor/entropy:0.7714234590530396 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28636491298675537 - training/rollout_probs_diff_mean:0.004574572201818228 - training/rollout_probs_diff_std:0.008538489229977131 - training/rollout_actor_probs_pearson_corr:0.9996505379676819 - rollout_corr/rollout_is_mean:0.9998939633369446 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6290990114212036 - rollout_corr/rollout_is_min:0.5353134870529175 - rollout_corr/rollout_is_std:0.03545675054192543 - rollout_corr/rollout_is_eff_sample_size:0.9987440406221719 - rollout_corr/rollout_is_seq_mean:0.9999128580093384 - rollout_corr/rollout_is_seq_std:0.0015794334467500448 - rollout_corr/rollout_is_seq_max:1.0056021213531494 - rollout_corr/rollout_is_seq_min:0.9935817122459412 - rollout_corr/rollout_is_seq_max_deviation:0.006418287754058838 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.22273992234841) - rollout_corr/training_log_ppl:np.float64(0.7840376188978553) - rollout_corr/kl:np.float64(0.0006947023282570797) - rollout_corr/k3_kl:np.float64(0.0007119409705182989) - rollout_corr/rollout_ppl:np.float64(2.221208442468196) - rollout_corr/rollout_log_ppl:np.float64(0.7833429137244821) - rollout_corr/log_ppl_diff:np.float64(0.0006947051733732224) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013338688295334578) - rollout_corr/log_ppl_diff_max:np.float64(0.0072593092918396) - rollout_corr/log_ppl_diff_min:np.float64(-0.004377126693725586) - rollout_corr/ppl_ratio:np.float64(1.0006962057668716) - rollout_corr/chi2_token:np.float64(0.0014604993630200624) - rollout_corr/chi2_seq:np.float64(0.6011870829388499) - actor/pg_loss:np.float64(0.0001900325296446681) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003155937018846089) - actor/ppo_kl:np.float64(-3.965757715462814e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.28370653092861176) - perf/mfu/actor:np.float64(0.11707659877392018) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.31705093383789) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:1 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018750634044408798 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0018750634044408798 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:654.92578125 - response_length/max:2451.0 - response_length/min:231.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:654.92578125 - response_length_non_aborted/max:2451.0 - response_length_non_aborted/min:231.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.59375 - prompt_length/max:818.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011583603918552399 - timing_s/agent_loop/generate_sequences/min:np.float64(2.9298688657581806) - timing_s/agent_loop/generate_sequences/max:np.float64(18.389138467609882) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.613169322678004) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.389138467609882) - timing_s/agent_loop/slowest/tool_calls
(TaskRunner pid=2224264) 
Training Progress:  65%|██████▌   | 65/100 [1:02:42<29:46, 51.03s/it]
(TaskRunner pid=2224264) step:66 - global_seqlen/min:28697 - global_seqlen/max:35089 - global_seqlen/minmax_diff:6392 - global_seqlen/balanced_min:30450 - global_seqlen/balanced_max:30469 - global_seqlen/mean:30460.5 - actor/entropy:0.7473804354667664 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3424738645553589 - training/rollout_probs_diff_mean:0.004387002903968096 - training/rollout_probs_diff_std:0.008388126268982887 - training/rollout_actor_probs_pearson_corr:0.9996775388717651 - rollout_corr/rollout_is_mean:0.9999525547027588 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1807769624283537e-05 - rollout_corr/rollout_is_max:1.7755447626113892 - rollout_corr/rollout_is_min:0.3809736371040344 - rollout_corr/rollout_is_std:0.0348651260137558 - rollout_corr/rollout_is_eff_sample_size:0.9987857797882222 - rollout_corr/rollout_is_seq_mean:0.999984622001648 - rollout_corr/rollout_is_seq_std:0.001510056434199214 - rollout_corr/rollout_is_seq_max:1.0043302774429321 - rollout_corr/rollout_is_seq_min:0.9965217709541321 - rollout_corr/rollout_is_seq_max_deviation:0.004330277442932129 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1832842687144876) - rollout_corr/training_log_ppl:np.float64(0.7604161555645987) - rollout_corr/kl:np.float64(0.0006697535459316839) - rollout_corr/k3_kl:np.float64(0.0006770667089313065) - rollout_corr/rollout_ppl:np.float64(2.18177503393963) - rollout_corr/rollout_log_ppl:np.float64(0.7597463995916769) - rollout_corr/log_ppl_diff:np.float64(0.0006697559729218483) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012636978644877672) - rollout_corr/log_ppl_diff_max:np.float64(0.005221843719482422) - rollout_corr/log_ppl_diff_min:np.float64(-0.0034908056259155273) - rollout_corr/ppl_ratio:np.float64(1.0006710432935506) - rollout_corr/chi2_token:np.float64(0.001379573019221425) - rollout_corr/chi2_seq:np.float64(0.7963614012114704) - actor/pg_loss:np.float64(-9.32483235374093e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021860310857846343) - actor/ppo_kl:np.float64(3.254798675556003e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2768869362771511) - perf/mfu/actor:np.float64(0.11916675434662388) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.34953784942627) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:1 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0009460975416004658 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0009460975416004658 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:661.640625 - response_length/max:1563.0 - response_length/min:181.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:661.640625 - response_length_non_aborted/max:1563.0 - response_length_non_aborted/min:181.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.25 - prompt_length/max:617.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000106053426861763 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3329896610230207) - timing_s/agent_loop/generate_sequences/max:np.float64(13.444917742162943) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.728698585859092) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.444917742162943) - timing_s/agent_loop/slowest/tool_ca
(TaskRunner pid=2224264) 
Training Progress:  66%|██████▌   | 66/100 [1:03:20<26:39, 47.05s/it]
(TaskRunner pid=2224264) step:67 - global_seqlen/min:21976 - global_seqlen/max:31921 - global_seqlen/minmax_diff:9945 - global_seqlen/balanced_min:28720 - global_seqlen/balanced_max:28745 - global_seqlen/mean:28734.25 - actor/entropy:0.7726788520812988 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3997594714164734 - training/rollout_probs_diff_mean:0.004599802196025848 - training/rollout_probs_diff_std:0.008748593740165234 - training/rollout_actor_probs_pearson_corr:0.9996384978294373 - rollout_corr/rollout_is_mean:1.0000046491622925 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9334631360834464e-05 - rollout_corr/rollout_is_max:1.7132740020751953 - rollout_corr/rollout_is_min:0.35777419805526733 - rollout_corr/rollout_is_std:0.03550714626908302 - rollout_corr/rollout_is_eff_sample_size:0.9987409489708702 - rollout_corr/rollout_is_seq_mean:1.0000040531158447 - rollout_corr/rollout_is_seq_std:0.0016247251769527793 - rollout_corr/rollout_is_seq_max:1.005348563194275 - rollout_corr/rollout_is_seq_min:0.9943971037864685 - rollout_corr/rollout_is_seq_max_deviation:0.005602896213531494 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.254453372210264) - rollout_corr/training_log_ppl:np.float64(0.7888535652309656) - rollout_corr/kl:np.float64(0.0006227922752195525) - rollout_corr/k3_kl:np.float64(0.0006862304701940047) - rollout_corr/rollout_ppl:np.float64(2.2529089520685375) - rollout_corr/rollout_log_ppl:np.float64(0.7882307717809454) - rollout_corr/log_ppl_diff:np.float64(0.0006227934500202537) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014135913224890828) - rollout_corr/log_ppl_diff_max:np.float64(0.007272124290466309) - rollout_corr/log_ppl_diff_min:np.float64(-0.004362225532531738) - rollout_corr/ppl_ratio:np.float64(1.000624400563538) - rollout_corr/chi2_token:np.float64(0.001499387202784419) - rollout_corr/chi2_seq:np.float64(1.2466026986949146) - actor/pg_loss:np.float64(-2.0926236175000668e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024252940147562185) - actor/ppo_kl:np.float64(-2.8959387931237757e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29370710253715515) - perf/mfu/actor:np.float64(0.11386412359551698) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.32535934448242) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:1 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009934777393937111 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009934777393937111 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:606.1015625 - response_length/max:1545.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:606.1015625 - response_length_non_aborted/max:1545.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.84375 - prompt_length/max:576.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00022127665579319 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8514165561646223) - timing_s/agent_loop/generate_sequences/max:np.float64(13.32896644063294) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.998300799677963) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.32896644063294) - timing_s/agent_loop/slowest/t
(TaskRunner pid=2224264) 
Training Progress:  67%|██████▋   | 67/100 [1:03:57<24:16, 44.12s/it]
(TaskRunner pid=2224264) step:68 - global_seqlen/min:23879 - global_seqlen/max:35240 - global_seqlen/minmax_diff:11361 - global_seqlen/balanced_min:29992 - global_seqlen/balanced_max:30029 - global_seqlen/mean:30017.25 - actor/entropy:0.7595304250717163 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7168671488761902 - training/rollout_probs_diff_mean:0.004404507111757994 - training/rollout_probs_diff_std:0.008633471094071865 - training/rollout_actor_probs_pearson_corr:0.9996846318244934 - rollout_corr/rollout_is_mean:0.999897301197052 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.8815619114320725e-05 - rollout_corr/rollout_is_max:1.5993539094924927 - rollout_corr/rollout_is_min:0.019239548593759537 - rollout_corr/rollout_is_std:0.03493344411253929 - rollout_corr/rollout_is_eff_sample_size:0.9987809040931451 - rollout_corr/rollout_is_seq_mean:0.9998229742050171 - rollout_corr/rollout_is_seq_std:0.0016775608528405428 - rollout_corr/rollout_is_seq_max:1.005442500114441 - rollout_corr/rollout_is_seq_min:0.9936239123344421 - rollout_corr/rollout_is_seq_max_deviation:0.006376087665557861 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.246089845895767) - rollout_corr/training_log_ppl:np.float64(0.7848675699206069) - rollout_corr/kl:np.float64(0.0009016538949424557) - rollout_corr/k3_kl:np.float64(0.0007493318355500378) - rollout_corr/rollout_ppl:np.float64(2.243979589547962) - rollout_corr/rollout_log_ppl:np.float64(0.7839659146266058) - rollout_corr/log_ppl_diff:np.float64(0.0009016552940011024) - rollout_corr/log_ppl_abs_diff:np.float64(0.001556948758661747) - rollout_corr/log_ppl_diff_max:np.float64(0.012223243713378906) - rollout_corr/log_ppl_diff_min:np.float64(-0.004822075366973877) - rollout_corr/ppl_ratio:np.float64(1.0009038699790835) - rollout_corr/chi2_token:np.float64(0.0011472513433545828) - rollout_corr/chi2_seq:np.float64(2.487598191248253) - actor/pg_loss:np.float64(-4.085572436451912e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027236666869612236) - actor/ppo_kl:np.float64(5.479786341200388e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2867882438004017) - perf/mfu/actor:np.float64(0.11808746991936632) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.29296493530273) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:1 - critic/score/mean:0.41796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006193474982865155 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006193474982865155 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:622.8203125 - response_length/max:1786.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:622.8203125 - response_length_non_aborted/max:1786.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.21875 - prompt_length/max:704.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014729611575603485 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0367041416466236) - timing_s/agent_loop/generate_sequences/max:np.float64(14.751183833926916) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.172361494318466) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.751183833926916) - timing_s/agent_loop/sl
(TaskRunner pid=2224264) 
Training Progress:  68%|██████▊   | 68/100 [1:04:36<22:42, 42.59s/it]
(TaskRunner pid=2224264) step:69 - global_seqlen/min:24189 - global_seqlen/max:45366 - global_seqlen/minmax_diff:21177 - global_seqlen/balanced_min:33420 - global_seqlen/balanced_max:39649 - global_seqlen/mean:34239.875 - actor/entropy:0.7071983814239502 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.38365769386291504 - training/rollout_probs_diff_mean:0.0040749297477304935 - training/rollout_probs_diff_std:0.00811900943517685 - training/rollout_actor_probs_pearson_corr:0.9996503591537476 - rollout_corr/rollout_is_mean:1.000025749206543 - rollout_corr/rollout_is_ratio_fraction_high:5.1385613915044814e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.1385613915044814e-06 - rollout_corr/rollout_is_max:2.4045958518981934 - rollout_corr/rollout_is_min:0.4110872149467468 - rollout_corr/rollout_is_std:0.03386257588863373 - rollout_corr/rollout_is_eff_sample_size:0.9988547581358254 - rollout_corr/rollout_is_seq_mean:1.0000370740890503 - rollout_corr/rollout_is_seq_std:0.0013557960046455264 - rollout_corr/rollout_is_seq_max:1.003797173500061 - rollout_corr/rollout_is_seq_min:0.9964458346366882 - rollout_corr/rollout_is_seq_max_deviation:0.003797173500061035 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.177241276949644) - rollout_corr/training_log_ppl:np.float64(0.7569285913486965) - rollout_corr/kl:np.float64(0.0004947985197105709) - rollout_corr/k3_kl:np.float64(0.0006430674019384242) - rollout_corr/rollout_ppl:np.float64(2.1761399055831134) - rollout_corr/rollout_log_ppl:np.float64(0.7564337924704887) - rollout_corr/log_ppl_diff:np.float64(0.0004947988782078028) - rollout_corr/log_ppl_abs_diff:np.float64(0.001128945965319872) - rollout_corr/log_ppl_diff_max:np.float64(0.004031240940093994) - rollout_corr/log_ppl_diff_min:np.float64(-0.004697918891906738) - rollout_corr/ppl_ratio:np.float64(1.00049584871158) - rollout_corr/chi2_token:np.float64(0.001590353436768055) - rollout_corr/chi2_seq:np.float64(0.8412239120807499) - actor/pg_loss:np.float64(-7.094477768987417e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019783686673235934) - actor/ppo_kl:np.float64(-9.313821129719457e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.25627975165843964) - perf/mfu/actor:np.float64(0.12214039553244277) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.25387191772461) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:1 - critic/score/mean:0.5703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010957982391119003 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010957982391119003 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:760.18359375 - response_length/max:8192.0 - response_length/min:169.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:760.18359375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:169.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:309.8125 - prompt_length/max:597.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012904033064842224 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1227447520941496) - timing_s/agent_loop/generate_sequences/max:np.float64(60.78352793678641) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.487850829478703) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(60.7835279367
(TaskRunner pid=2224264) 
Training Progress:  69%|██████▉   | 69/100 [1:06:02<28:43, 55.58s/it]
(TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70}
(TaskRunner pid=2224264) validation generation end
(TaskRunner pid=2224264) [prompt] system
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2224264) <reasoning>
(TaskRunner pid=2224264) ...
(TaskRunner pid=2224264) </reasoning>
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) ...
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) A
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) user
(TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2224264) Please reason step by step.
(TaskRunner pid=2224264) assistant
(TaskRunner pid=2224264) <think>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) </think>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) [response] <reasoning>
(TaskRunner pid=2224264) We are given a product:  
(TaskRunner pid=2224264) **Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This is a molecular structure with the following key features:
(TaskRunner pid=2224264) - A nitrogen atom at the end of a chain (Nc1...n1)
(TaskRunner pid=2224264) - A benzene ring with a substituent: `Cc2ccc(Cl)cc2C(F)(F)F` (i.e., a substituted benzene ring with a chloro group and a trifluoromethyl group)
(TaskRunner pid=2224264) - The overall structure appears to be a **pyrrole-like ring** (Nc1ccn...n1) with a substituent group on the nitrogen.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) We are to identify which of the four options is the **only correct reactant** that would be used in the synthesis of this product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) Let’s analyze each option:
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) **Option A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) - This contains the core structure: `c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1` — which matches the product.
(TaskRunner pid=2224264) - It also has a ketone group `O=C1` and another ketone group `C(=O)N1` — which may be a part of a synthesis pathway (e.g., a diketone or acylated structure).
(TaskRunner pid=2224264) - This seems like a plausible reactant that could be used to form the product via a reaction like a **condensation** or **cyclization**.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) **Option B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) - This contains a `C(Cl)` group, not the `C(F)(F)F` group we see in the product.
(TaskRunner pid=2224264) - The rest of the structure is not matching the product's ring system or substituents.
(TaskRunner pid=2224264) - Therefore, this is **not a correct reactant**.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) **Option C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) - Contains a `C(F)(F)F` group (trifluoromethyl), which matches the product.
(TaskRunner pid=2224264) - It also has a `Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1` — which includes a benzene ring with a chloro and trifluoromethyl group.
(TaskRunner pid=2224264) - However, this structure is more of a **complex aromatic ring system**, not a direct match for the pyrrole-like structure in the product.
(TaskRunner pid=2224264) - It may be a reactant in a more complex multi-step synthesis, but it is **not the only correct one**.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) **Option D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) - Contains the `C(F)(F)F` group and a benzene ring.
(TaskRunner pid=2224264) - However, it does **not** match the nitrogen-containing ring structure of the product.
(TaskRunner pid=2224264) - The presence of a double bond in the side chain (CC=C) is not matching the pyrrole-like ring in the product.
(TaskRunner pid=2224264) - So this is **not the correct reactant**.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Conclusion:
(TaskRunner pid=2224264) Only **Option A** contains the **exact ring structure** and **substituents** needed to form the product. It includes the pyrrole-like ring (`c1ccn(...n1`), the chloro-substituted benzene ring with a trifluoromethyl group, and has functional groups that are consistent with a synthetic pathway leading to the product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) </reasoning>
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) A
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) [ground_truth] A
(TaskRunner pid=2224264) [score] 1.0
(TaskRunner pid=2224264) [acc] 1.0
(TaskRunner pid=2224264) [pred] A
(TaskRunner pid=2224264) [incorrect_format] 1
(TaskRunner pid=2224264) [feedback] 
(TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70
(TaskRunner pid=2224264) step:70 - global_seqlen/min:22032 - global_seqlen/max:35660 - global_seqlen/minmax_diff:13628 - global_seqlen/balanced_min:30368 - global_seqlen/balanced_max:30409 - global_seqlen/mean:30397.25 - actor/entropy:0.8035950064659119 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22990918159484863 - training/rollout_probs_diff_mean:0.004655633121728897 - training/rollout_probs_diff_std:0.008662798441946507 - training/rollout_actor_probs_pearson_corr:0.9996584057807922 - rollout_corr/rollout_is_mean:1.0000817775726318 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.9939820857835e-06 - rollout_corr/rollout_is_max:1.8103430271148682 - rollout_corr/rollout_is_min:0.4562520384788513 - rollout_corr/rollout_is_std:0.035914406180381775 - rollout_corr/rollout_is_eff_sample_size:0.9987120548475574 - rollout_corr/rollout_is_seq_mean:1.0000861883163452 - rollout_corr/rollout_is_seq_std:0.0016128499992191792 - rollout_corr/rollout_is_seq_max:1.00638747215271 - rollout_corr/rollout_is_seq_min:0.9938380718231201 - rollout_corr/rollout_is_seq_max_deviation:0.006387472152709961 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.279810177627951) - rollout_corr/training_log_ppl:np.float64(0.8013737860601395) - rollout_corr/kl:np.float64(0.0007208169924197705) - rollout_corr/k3_kl:np.float64(0.0007489672824476656) - rollout_corr/rollout_ppl:np.float64(2.278083552606404) - rollout_corr/rollout_log_ppl:np.float64(0.8006529699778184) - rollout_corr/log_ppl_diff:np.float64(0.0007208160823211074) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014672434190288186) - rollout_corr/log_ppl_diff_max:np.float64(0.008777737617492676) - rollout_corr/log_ppl_diff_min:np.float64(-0.005332350730895996) - rollout_corr/ppl_ratio:np.float64(1.000722888391465) - rollout_corr/chi2_token:np.float64(0.0015351453330367804) - rollout_corr/chi2_seq:np.float64(2.378258903743699) - actor/pg_loss:np.float64(-0.000118100899271667) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018792178138937743) - actor/ppo_kl:np.float64(0.00014457460659045296) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2343890555202961) - perf/mfu/actor:np.float64(0.11821289785724132) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.17000198364258) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6714285714285714) - val-aux/sciknoweval/reward/std@16:np.float64(0.2317466189895333) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7653619047619048) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1782119464541067) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5797619047619048) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2136060282647753) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6726619047619047) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.23202999913590713) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8335190476190476) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11938512568929552) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.49611428571428573) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1773846797198801) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6981047619047619) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.184153742497439) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8789999999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0730800489542387) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.42429047619047616) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.13809200842124467) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7115333333333334) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13558200983257696) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9054
(TaskRunner pid=2224264) 
Training Progress:  70%|███████   | 70/100 [1:10:04<55:39, 111.31s/it]
(TaskRunner pid=2224264) step:71 - global_seqlen/min:26528 - global_seqlen/max:38219 - global_seqlen/minmax_diff:11691 - global_seqlen/balanced_min:31790 - global_seqlen/balanced_max:32177 - global_seqlen/mean:31863.5 - actor/entropy:0.7618645429611206 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21302002668380737 - training/rollout_probs_diff_mean:0.004489706829190254 - training/rollout_probs_diff_std:0.008394083939492702 - training/rollout_actor_probs_pearson_corr:0.9996015429496765 - rollout_corr/rollout_is_mean:0.9997878670692444 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.690140925347805e-05 - rollout_corr/rollout_is_max:1.6874182224273682 - rollout_corr/rollout_is_min:0.4357602596282959 - rollout_corr/rollout_is_std:0.034910403192043304 - rollout_corr/rollout_is_eff_sample_size:0.9987822122017861 - rollout_corr/rollout_is_seq_mean:0.9997093677520752 - rollout_corr/rollout_is_seq_std:0.0016728744376450777 - rollout_corr/rollout_is_seq_max:1.005326747894287 - rollout_corr/rollout_is_seq_min:0.9935998916625977 - rollout_corr/rollout_is_seq_max_deviation:0.006400108337402344 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.216639978811145) - rollout_corr/training_log_ppl:np.float64(0.7745629880810156) - rollout_corr/kl:np.float64(0.000979817594189747) - rollout_corr/k3_kl:np.float64(0.0007290477265087247) - rollout_corr/rollout_ppl:np.float64(2.2143593542277813) - rollout_corr/rollout_log_ppl:np.float64(0.7735831652535126) - rollout_corr/log_ppl_diff:np.float64(0.0009798228275030851) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014419290237128735) - rollout_corr/log_ppl_diff_max:np.float64(0.008339285850524902) - rollout_corr/log_ppl_diff_min:np.float64(-0.0046994686126708984) - rollout_corr/ppl_ratio:np.float64(1.0009817637037486) - rollout_corr/chi2_token:np.float64(0.0009503676556050777) - rollout_corr/chi2_seq:np.float64(0.568886311724782) - actor/pg_loss:np.float64(5.502370186150074e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019256132691225503) - actor/ppo_kl:np.float64(4.591261260600277e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20790256932377815) - perf/mfu/actor:np.float64(0.12276829314252474) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(109.04192352294922) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:1 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0034556337632238865 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0034556337632238865 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:693.359375 - response_length/max:2588.0 - response_length/min:171.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:693.359375 - response_length_non_aborted/max:2588.0 - response_length_non_aborted/min:171.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.375 - prompt_length/max:666.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.34839928150177e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.169468689709902) - timing_s/agent_loop/generate_sequences/max:np.float64(19.67856295593083) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.767557479441166) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.67856295593083) - timing_s/agent_loop/slowest/too
(TaskRunner pid=2224264) 
Training Progress:  71%|███████   | 71/100 [1:10:47<43:58, 90.98s/it] 
(TaskRunner pid=2224264) step:72 - global_seqlen/min:26255 - global_seqlen/max:35299 - global_seqlen/minmax_diff:9044 - global_seqlen/balanced_min:30687 - global_seqlen/balanced_max:30704 - global_seqlen/mean:30699.25 - actor/entropy:0.7660148739814758 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2309356927871704 - training/rollout_probs_diff_mean:0.0045791687443852425 - training/rollout_probs_diff_std:0.008554509840905666 - training/rollout_actor_probs_pearson_corr:0.999636709690094 - rollout_corr/rollout_is_mean:0.9999493360519409 - rollout_corr/rollout_is_ratio_fraction_high:5.8084829106519464e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0826847553253174 - rollout_corr/rollout_is_min:0.5040711760520935 - rollout_corr/rollout_is_std:0.03556250035762787 - rollout_corr/rollout_is_eff_sample_size:0.9987369060711254 - rollout_corr/rollout_is_seq_mean:0.9999403953552246 - rollout_corr/rollout_is_seq_std:0.0016658001113682985 - rollout_corr/rollout_is_seq_max:1.006662368774414 - rollout_corr/rollout_is_seq_min:0.99498051404953 - rollout_corr/rollout_is_seq_max_deviation:0.0066623687744140625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.172296098433435) - rollout_corr/training_log_ppl:np.float64(0.7561264133546501) - rollout_corr/kl:np.float64(0.0008874626383423845) - rollout_corr/k3_kl:np.float64(0.0007379499492117247) - rollout_corr/rollout_ppl:np.float64(2.170314473565668) - rollout_corr/rollout_log_ppl:np.float64(0.7552389547927305) - rollout_corr/log_ppl_diff:np.float64(0.0008874585619196296) - rollout_corr/log_ppl_abs_diff:np.float64(0.001500150072388351) - rollout_corr/log_ppl_diff_max:np.float64(0.00498625636100769) - rollout_corr/log_ppl_diff_min:np.float64(-0.0046173930168151855) - rollout_corr/ppl_ratio:np.float64(1.0008892305195332) - rollout_corr/chi2_token:np.float64(0.0011937415692955256) - rollout_corr/chi2_seq:np.float64(0.66025594715029) - actor/pg_loss:np.float64(3.53293726220727e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002114741264449549) - actor/ppo_kl:np.float64(0.00018003375489444862) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2194691300392151) - perf/mfu/actor:np.float64(0.1192681330197729) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.72951126098633) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:1 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007243903819471598 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007243903819471598 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:672.5078125 - response_length/max:1734.0 - response_length/min:186.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:672.5078125 - response_length_non_aborted/max:1734.0 - response_length_non_aborted/min:186.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.84375 - prompt_length/max:620.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.6632100343704224e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.31886431761086) - timing_s/agent_loop/generate_sequences/max:np.float64(14.253411900252104) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.534527560528659) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.253411900252104) - timing_s/agent_loop/slowest/tool_c
(TaskRunner pid=2224264) 
Training Progress:  72%|███████▏  | 72/100 [1:11:26<35:08, 75.31s/it]
(TaskRunner pid=2224264) step:73 - global_seqlen/min:32171 - global_seqlen/max:38705 - global_seqlen/minmax_diff:6534 - global_seqlen/balanced_min:33645 - global_seqlen/balanced_max:39741 - global_seqlen/mean:34425.125 - actor/entropy:0.763630211353302 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30194756388664246 - training/rollout_probs_diff_mean:0.004307533614337444 - training/rollout_probs_diff_std:0.008280006237328053 - training/rollout_actor_probs_pearson_corr:0.999650776386261 - rollout_corr/rollout_is_mean:0.9999016523361206 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.910024017590331e-06 - rollout_corr/rollout_is_max:1.6241782903671265 - rollout_corr/rollout_is_min:0.4921497702598572 - rollout_corr/rollout_is_std:0.034490425139665604 - rollout_corr/rollout_is_eff_sample_size:0.9988115860890442 - rollout_corr/rollout_is_seq_mean:0.9998833537101746 - rollout_corr/rollout_is_seq_std:0.0013762600719928741 - rollout_corr/rollout_is_seq_max:1.0045596361160278 - rollout_corr/rollout_is_seq_min:0.9957677125930786 - rollout_corr/rollout_is_seq_max_deviation:0.004559636116027832 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2688772692345083) - rollout_corr/training_log_ppl:np.float64(0.8037596073554596) - rollout_corr/kl:np.float64(0.0007620832186034932) - rollout_corr/k3_kl:np.float64(0.0006744360291293106) - rollout_corr/rollout_ppl:np.float64(2.267069778870791) - rollout_corr/rollout_log_ppl:np.float64(0.8029975256649777) - rollout_corr/log_ppl_diff:np.float64(0.0007620816904818639) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012150325201218948) - rollout_corr/log_ppl_diff_max:np.float64(0.006102800369262695) - rollout_corr/log_ppl_diff_min:np.float64(-0.003519415855407715) - rollout_corr/ppl_ratio:np.float64(1.0007633492350578) - rollout_corr/chi2_token:np.float64(0.0011786054819822311) - rollout_corr/chi2_seq:np.float64(0.6752500396687537) - actor/pg_loss:np.float64(7.905624806880951e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014205676188794314) - actor/ppo_kl:np.float64(1.4047067626421494e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2183140367269516) - perf/mfu/actor:np.float64(0.12278116839994453) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.55410766601562) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:1 - critic/score/mean:0.578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005568580701947212 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005568580701947212 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:795.56640625 - response_length/max:8192.0 - response_length/min:236.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:795.56640625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:236.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:280.21875 - prompt_length/max:557.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014955177903175354 - timing_s/agent_loop/generate_sequences/min:np.float64(2.818276045843959) - timing_s/agent_loop/generate_sequences/max:np.float64(61.54518798738718) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.143103411246557) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(61.54518798738718) - timing_s/age
(TaskRunner pid=2224264) 
Training Progress:  73%|███████▎  | 73/100 [1:12:54<35:32, 78.99s/it]
(TaskRunner pid=2224264) step:74 - global_seqlen/min:29986 - global_seqlen/max:43985 - global_seqlen/minmax_diff:13999 - global_seqlen/balanced_min:33665 - global_seqlen/balanced_max:33695 - global_seqlen/mean:33681.375 - actor/entropy:0.761839747428894 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19873207807540894 - training/rollout_probs_diff_mean:0.004497237037867308 - training/rollout_probs_diff_std:0.008453729562461376 - training/rollout_actor_probs_pearson_corr:0.999616265296936 - rollout_corr/rollout_is_mean:0.9999936819076538 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6013860702514648 - rollout_corr/rollout_is_min:0.606749951839447 - rollout_corr/rollout_is_std:0.03470577299594879 - rollout_corr/rollout_is_eff_sample_size:0.9987970773134115 - rollout_corr/rollout_is_seq_mean:0.9999818801879883 - rollout_corr/rollout_is_seq_std:0.0015479137655347586 - rollout_corr/rollout_is_seq_max:1.0074827671051025 - rollout_corr/rollout_is_seq_min:0.9962047338485718 - rollout_corr/rollout_is_seq_max_deviation:0.007482767105102539 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2138938223943114) - rollout_corr/training_log_ppl:np.float64(0.7749610797036439) - rollout_corr/kl:np.float64(0.0006632767860637601) - rollout_corr/k3_kl:np.float64(0.0006568010118712664) - rollout_corr/rollout_ppl:np.float64(2.212330491282046) - rollout_corr/rollout_log_ppl:np.float64(0.7742978031164967) - rollout_corr/log_ppl_diff:np.float64(0.0006632765871472657) - rollout_corr/log_ppl_abs_diff:np.float64(0.001370829122606665) - rollout_corr/log_ppl_diff_max:np.float64(0.004554629325866699) - rollout_corr/log_ppl_diff_min:np.float64(-0.006599485874176025) - rollout_corr/ppl_ratio:np.float64(1.0006648083217442) - rollout_corr/chi2_token:np.float64(0.0013132649473845959) - rollout_corr/chi2_seq:np.float64(1.9479922475293279) - actor/pg_loss:np.float64(-5.3111580200493336e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014415476721296727) - actor/ppo_kl:np.float64(1.6327075574196215e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20973877236247063) - perf/mfu/actor:np.float64(0.12778629328477945) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.42087173461914) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:1 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005659057758748531 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005659057758748531 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:779.13671875 - response_length/max:2249.0 - response_length/min:207.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:779.13671875 - response_length_non_aborted/max:2249.0 - response_length_non_aborted/min:207.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.40625 - prompt_length/max:665.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010413303971290588 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6374386623501778) - timing_s/agent_loop/generate_sequences/max:np.float64(18.268491968512535) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.814450513171323) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.268491968512535) - timing_s/agent_loop/slowest/tool_calls
(TaskRunner pid=2224264) 
Training Progress:  74%|███████▍  | 74/100 [1:13:36<29:31, 68.13s/it]
(TaskRunner pid=2224264) step:75 - global_seqlen/min:24487 - global_seqlen/max:40274 - global_seqlen/minmax_diff:15787 - global_seqlen/balanced_min:34105 - global_seqlen/balanced_max:34135 - global_seqlen/mean:34124.625 - actor/entropy:0.8144033551216125 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17917940020561218 - training/rollout_probs_diff_mean:0.00457530515268445 - training/rollout_probs_diff_std:0.008413956500589848 - training/rollout_actor_probs_pearson_corr:0.9996695518493652 - rollout_corr/rollout_is_mean:1.0000370740890503 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.187879196455469e-06 - rollout_corr/rollout_is_max:1.7824959754943848 - rollout_corr/rollout_is_min:0.4731596112251282 - rollout_corr/rollout_is_std:0.035698000341653824 - rollout_corr/rollout_is_eff_sample_size:0.9987273934949555 - rollout_corr/rollout_is_seq_mean:1.0000509023666382 - rollout_corr/rollout_is_seq_std:0.0015275044133886695 - rollout_corr/rollout_is_seq_max:1.004511833190918 - rollout_corr/rollout_is_seq_min:0.9952264428138733 - rollout_corr/rollout_is_seq_max_deviation:0.004773557186126709 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2711348473094404) - rollout_corr/training_log_ppl:np.float64(0.803941803984344) - rollout_corr/kl:np.float64(0.0005384247337580916) - rollout_corr/k3_kl:np.float64(0.000659077487966897) - rollout_corr/rollout_ppl:np.float64(2.2699187486432493) - rollout_corr/rollout_log_ppl:np.float64(0.803403376834467) - rollout_corr/log_ppl_diff:np.float64(0.0005384271498769522) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012698217760771513) - rollout_corr/log_ppl_diff_max:np.float64(0.0053713321685791016) - rollout_corr/log_ppl_diff_min:np.float64(-0.003410249948501587) - rollout_corr/ppl_ratio:np.float64(1.0005397242493927) - rollout_corr/chi2_token:np.float64(0.0015677101910114288) - rollout_corr/chi2_seq:np.float64(1.5271832724101841) - actor/pg_loss:np.float64(-4.3374369852244854e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.082515417598188e-05) - actor/ppo_kl:np.float64(-5.413583712154946e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17228768952190876) - perf/mfu/actor:np.float64(0.13115778980210233) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.54808044433594) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:1 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018546667415648699 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0018546667415648699 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:752.95703125 - response_length/max:1483.0 - response_length/min:199.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:752.95703125 - response_length_non_aborted/max:1483.0 - response_length_non_aborted/min:199.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.4375 - prompt_length/max:608.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011150166392326355 - timing_s/agent_loop/generate_sequences/min:np.float64(2.5929522551596165) - timing_s/agent_loop/generate_sequences/max:np.float64(14.319374568760395) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.897164289628563) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.319374568760395) - timing_s/agent_l
(TaskRunner pid=2224264) 
Training Progress:  75%|███████▌  | 75/100 [1:14:15<24:40, 59.22s/it]
(TaskRunner pid=2224264) step:76 - global_seqlen/min:31915 - global_seqlen/max:42520 - global_seqlen/minmax_diff:10605 - global_seqlen/balanced_min:37284 - global_seqlen/balanced_max:37334 - global_seqlen/mean:37307.375 - actor/entropy:0.746766984462738 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48574504256248474 - training/rollout_probs_diff_mean:0.004121627192944288 - training/rollout_probs_diff_std:0.008096432313323021 - training/rollout_actor_probs_pearson_corr:0.999716579914093 - rollout_corr/rollout_is_mean:1.0002018213272095 - rollout_corr/rollout_is_ratio_fraction_high:4.438073119672481e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.438073119672481e-06 - rollout_corr/rollout_is_max:2.379199504852295 - rollout_corr/rollout_is_min:0.4145638346672058 - rollout_corr/rollout_is_std:0.03402938321232796 - rollout_corr/rollout_is_eff_sample_size:0.9988438161069871 - rollout_corr/rollout_is_seq_mean:1.000188946723938 - rollout_corr/rollout_is_seq_std:0.0011295655276626348 - rollout_corr/rollout_is_seq_max:1.0031288862228394 - rollout_corr/rollout_is_seq_min:0.9967604875564575 - rollout_corr/rollout_is_seq_max_deviation:0.0032395124435424805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.173908702097833) - rollout_corr/training_log_ppl:np.float64(0.7561200592317618) - rollout_corr/kl:np.float64(0.0004855001487325261) - rollout_corr/k3_kl:np.float64(0.0005892427249136745) - rollout_corr/rollout_ppl:np.float64(2.1727346065454185) - rollout_corr/rollout_log_ppl:np.float64(0.755634555243887) - rollout_corr/log_ppl_diff:np.float64(0.00048550398787483573) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009925910853780806) - rollout_corr/log_ppl_diff_max:np.float64(0.0035272836685180664) - rollout_corr/log_ppl_diff_min:np.float64(-0.002400696277618408) - rollout_corr/ppl_ratio:np.float64(1.0004863012582064) - rollout_corr/chi2_token:np.float64(0.0013905672822147608) - rollout_corr/chi2_seq:np.float64(0.9606744900811464) - actor/pg_loss:np.float64(-2.3183762095868587e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.338781410406227e-05) - actor/ppo_kl:np.float64(8.314622859639798e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18520821258425713) - perf/mfu/actor:np.float64(0.13935883033564134) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.53511428833008) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:1 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00035671013756655157 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00035671013756655157 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:880.16796875 - response_length/max:2522.0 - response_length/min:204.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:880.16796875 - response_length_non_aborted/max:2522.0 - response_length_non_aborted/min:204.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:285.6875 - prompt_length/max:526.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001479722559452057 - timing_s/agent_loop/generate_sequences/min:np.float64(2.7096176706254482) - timing_s/agent_loop/generate_sequences/max:np.float64(20.600152047351003) - timing_s/agent_loop/generate_sequences/mean:np.float64(10.33266173733864) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.600152047351003) 
(TaskRunner pid=2224264) 
Training Progress:  76%|███████▌  | 76/100 [1:15:00<21:59, 54.96s/it]
(TaskRunner pid=2224264) step:77 - global_seqlen/min:27604 - global_seqlen/max:39041 - global_seqlen/minmax_diff:11437 - global_seqlen/balanced_min:34288 - global_seqlen/balanced_max:34332 - global_seqlen/mean:34317.375 - actor/entropy:0.8058213591575623 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28673720359802246 - training/rollout_probs_diff_mean:0.004518347326666117 - training/rollout_probs_diff_std:0.008365931920707226 - training/rollout_actor_probs_pearson_corr:0.999616265296936 - rollout_corr/rollout_is_mean:0.9999557733535767 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6204079389572144 - rollout_corr/rollout_is_min:0.6416282057762146 - rollout_corr/rollout_is_std:0.03523077070713043 - rollout_corr/rollout_is_eff_sample_size:0.9987602126485932 - rollout_corr/rollout_is_seq_mean:0.9999356269836426 - rollout_corr/rollout_is_seq_std:0.0013528199633583426 - rollout_corr/rollout_is_seq_max:1.0050373077392578 - rollout_corr/rollout_is_seq_min:0.9947447180747986 - rollout_corr/rollout_is_seq_max_deviation:0.005255281925201416 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.28120540920645) - rollout_corr/training_log_ppl:np.float64(0.8050968096358702) - rollout_corr/kl:np.float64(0.0006265352302827765) - rollout_corr/k3_kl:np.float64(0.0006796217078317568) - rollout_corr/rollout_ppl:np.float64(2.279738489538431) - rollout_corr/rollout_log_ppl:np.float64(0.8044702711631544) - rollout_corr/log_ppl_diff:np.float64(0.0006265384727157652) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010923692607320845) - rollout_corr/log_ppl_diff_max:np.float64(0.005904078483581543) - rollout_corr/log_ppl_diff_min:np.float64(-0.0028671622276306152) - rollout_corr/ppl_ratio:np.float64(1.0006276031490415) - rollout_corr/chi2_token:np.float64(0.0014646041672676802) - rollout_corr/chi2_seq:np.float64(0.7338430834934115) - actor/pg_loss:np.float64(0.00012833683285862207) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017851591564976843) - actor/ppo_kl:np.float64(-6.952557651018765e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2424900233745575) - perf/mfu/actor:np.float64(0.13009512944939702) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.56500244140625) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:1 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007992738857865334 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007992738857865334 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:755.32421875 - response_length/max:2071.0 - response_length/min:186.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:755.32421875 - response_length_non_aborted/max:2071.0 - response_length_non_aborted/min:186.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:317.09375 - prompt_length/max:716.0 - prompt_length/min:173.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013962015509605408 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1678621973842382) - timing_s/agent_loop/generate_sequences/max:np.float64(17.2276831548661) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.9278243403023225) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.2276831548661) - timing_s/agent_loop/slowest/tool_calls:np.flo
(TaskRunner pid=2224264) 
Training Progress:  77%|███████▋  | 77/100 [1:15:42<19:35, 51.12s/it]
(TaskRunner pid=2224264) step:78 - global_seqlen/min:28938 - global_seqlen/max:37974 - global_seqlen/minmax_diff:9036 - global_seqlen/balanced_min:34229 - global_seqlen/balanced_max:34251 - global_seqlen/mean:34245.0 - actor/entropy:0.7469825148582458 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48943376541137695 - training/rollout_probs_diff_mean:0.004324706271290779 - training/rollout_probs_diff_std:0.008407223038375378 - training/rollout_actor_probs_pearson_corr:0.9996141195297241 - rollout_corr/rollout_is_mean:1.0000590085983276 - rollout_corr/rollout_is_ratio_fraction_high:4.9409068196837325e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.881813639367465e-06 - rollout_corr/rollout_is_max:4.681254863739014 - rollout_corr/rollout_is_min:0.4222910404205322 - rollout_corr/rollout_is_std:0.03437097743153572 - rollout_corr/rollout_is_eff_sample_size:0.9988201488429446 - rollout_corr/rollout_is_seq_mean:1.00004243850708 - rollout_corr/rollout_is_seq_std:0.001362150302156806 - rollout_corr/rollout_is_seq_max:1.0039869546890259 - rollout_corr/rollout_is_seq_min:0.9943282008171082 - rollout_corr/rollout_is_seq_max_deviation:0.005671799182891846 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1253380407579243) - rollout_corr/training_log_ppl:np.float64(0.7371736962813884) - rollout_corr/kl:np.float64(0.0005827662733111083) - rollout_corr/k3_kl:np.float64(0.0006460418212554941) - rollout_corr/rollout_ppl:np.float64(2.1241166265681386) - rollout_corr/rollout_log_ppl:np.float64(0.7365909329382703) - rollout_corr/log_ppl_diff:np.float64(0.0005827633431181312) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012038786662742496) - rollout_corr/log_ppl_diff_max:np.float64(0.006325364112854004) - rollout_corr/log_ppl_diff_min:np.float64(-0.003412604331970215) - rollout_corr/ppl_ratio:np.float64(1.0005839837249368) - rollout_corr/chi2_token:np.float64(0.0014792552683502436) - rollout_corr/chi2_seq:np.float64(2.1139756981283426) - actor/pg_loss:np.float64(0.0001295657129958272) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001085545256955811) - actor/ppo_kl:np.float64(1.3940017076219391e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20791660994291306) - perf/mfu/actor:np.float64(0.1296498704894837) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.49341583251953) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:1 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0018497519195079803 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0018497519195079803 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:790.59375 - response_length/max:1840.0 - response_length/min:181.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:790.59375 - response_length_non_aborted/max:1840.0 - response_length_non_aborted/min:181.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.5625 - prompt_length/max:449.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010417215526103973 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2816403061151505) - timing_s/agent_loop/generate_sequences/max:np.float64(16.65826540067792) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.189224266308884) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.65826540067792) - timing_s/agent_loo
(TaskRunner pid=2224264) 
Training Progress:  78%|███████▊  | 78/100 [1:16:23<17:38, 48.10s/it]
(TaskRunner pid=2224264) step:79 - global_seqlen/min:31052 - global_seqlen/max:40489 - global_seqlen/minmax_diff:9437 - global_seqlen/balanced_min:35267 - global_seqlen/balanced_max:35291 - global_seqlen/mean:35282.375 - actor/entropy:0.7419886589050293 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3267946243286133 - training/rollout_probs_diff_mean:0.004350726492702961 - training/rollout_probs_diff_std:0.008377747610211372 - training/rollout_actor_probs_pearson_corr:0.9996976256370544 - rollout_corr/rollout_is_mean:1.0000663995742798 - rollout_corr/rollout_is_ratio_fraction_high:4.6575100896006916e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.315020179201383e-06 - rollout_corr/rollout_is_max:2.486323833465576 - rollout_corr/rollout_is_min:0.3386184871196747 - rollout_corr/rollout_is_std:0.034230221062898636 - rollout_corr/rollout_is_eff_sample_size:0.9988299010470403 - rollout_corr/rollout_is_seq_mean:1.0000535249710083 - rollout_corr/rollout_is_seq_std:0.001242220401763916 - rollout_corr/rollout_is_seq_max:1.0033190250396729 - rollout_corr/rollout_is_seq_min:0.9950276017189026 - rollout_corr/rollout_is_seq_max_deviation:0.004972398281097412 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1713191107846797) - rollout_corr/training_log_ppl:np.float64(0.7553618648089468) - rollout_corr/kl:np.float64(0.0005726922157927028) - rollout_corr/k3_kl:np.float64(0.0006373734220233018) - rollout_corr/rollout_ppl:np.float64(2.170036384370178) - rollout_corr/rollout_log_ppl:np.float64(0.754789168946445) - rollout_corr/log_ppl_diff:np.float64(0.0005726958625018597) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010784545447677374) - rollout_corr/log_ppl_diff_max:np.float64(0.004006564617156982) - rollout_corr/log_ppl_diff_min:np.float64(-0.00302046537399292) - rollout_corr/ppl_ratio:np.float64(1.0005736567545682) - rollout_corr/chi2_token:np.float64(0.001405506394803524) - rollout_corr/chi2_seq:np.float64(1.6988238336052746) - actor/pg_loss:np.float64(6.845139432698488e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017011942372846534) - actor/ppo_kl:np.float64(1.6240256268673647e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.24053851515054703) - perf/mfu/actor:np.float64(0.13349716649984694) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.54805374145508) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:1 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011310762725770473 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011310762725770473 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:838.69921875 - response_length/max:2007.0 - response_length/min:283.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:838.69921875 - response_length_non_aborted/max:2007.0 - response_length_non_aborted/min:283.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.875 - prompt_length/max:766.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010603666305541992 - timing_s/agent_loop/generate_sequences/min:np.float64(3.655642168596387) - timing_s/agent_loop/generate_sequences/max:np.float64(17.38879804685712) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.424919297925953) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.38879804685712) - timing_s
(TaskRunner pid=2224264) 
Training Progress:  79%|███████▉  | 79/100 [1:17:05<16:10, 46.19s/it]
(TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80}
(TaskRunner pid=2224264) validation generation end
(TaskRunner pid=2224264) [prompt] system
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2224264) <reasoning>
(TaskRunner pid=2224264) ...
(TaskRunner pid=2224264) </reasoning>
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) ...
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) A
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) user
(TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2224264) Please reason step by step.
(TaskRunner pid=2224264) assistant
(TaskRunner pid=2224264) <think>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) </think>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) [response] <reasoning>
(TaskRunner pid=2224264) We are given a target product:  
(TaskRunner pid=2224264) **Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1**  
(TaskRunner pid=2224264) This is a nitrogen-containing compound with a ring structure. Let's analyze it:
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) - The formula appears to represent a **pyrrole ring** (a 5-membered ring with one nitrogen atom).
(TaskRunner pid=2224264) - The nitrogen is at the first position (Nc1...), and there's a substituent at the third carbon: **Cc2ccc(Cl)cc2C(F)(F)F**, which implies a **benzene ring** with a **chlorine atom** and a **trifluoromethyl group (C(F)(F)F)** on the same carbon.
(TaskRunner pid=2224264) - So the full structure is: a **pyrrole ring** with a **benzene ring** substituted with **Cl** and **CF₃** at the same position, attached to the third carbon of the pyrrole.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) Now we examine the options to find which one is the **correct reactant** that would lead to this product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Option A:  
(TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1**  
(TaskRunner pid=2224264) - This has a **pyrrole ring** (from the N1 and the 5-membered ring).
(TaskRunner pid=2224264) - The ring is substituted with a **benzene ring** (c2ccccc2) with a **Cl** and **CF₃** at the same position.
(TaskRunner pid=2224264) - The structure matches the **target product** exactly.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Option B:  
(TaskRunner pid=2224264) **O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1**  
(TaskRunner pid=2224264) - This contains a **Cl** on the ring, but the ring structure is different (not a pyrrole, and the Cl is not in the correct position).
(TaskRunner pid=2224264) - The structure is not matching the target.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Option C:  
(TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1**  
(TaskRunner pid=2224264) - Contains a **benzene ring** with Cl and CF₃, but it's attached to an **oxygen** (Oc2...), not directly to the pyrrole.
(TaskRunner pid=2224264) - The structure is different from the target.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Option D:  
(TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F**  
(TaskRunner pid=2224264) - This has a **trifluoromethyl group** (C(F)(F)F), but it is attached to a **methylene group** (CC=C) and not directly to the ring.
(TaskRunner pid=2224264) - The structure is not matching the target.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Conclusion:
(TaskRunner pid=2224264) Only **Option A** has the correct **pyrrole ring** with the correct **benzene substituent** (Cl and CF₃) in the correct position, matching the structure of the given product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) </reasoning>
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) A
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) [ground_truth] A
(TaskRunner pid=2224264) [score] 1.0
(TaskRunner pid=2224264) [acc] 1.0
(TaskRunner pid=2224264) [pred] A
(TaskRunner pid=2224264) [incorrect_format] 1
(TaskRunner pid=2224264) [feedback] 
(TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80
(TaskRunner pid=2224264) step:80 - global_seqlen/min:28862 - global_seqlen/max:47477 - global_seqlen/minmax_diff:18615 - global_seqlen/balanced_min:36267 - global_seqlen/balanced_max:36835 - global_seqlen/mean:36375.375 - actor/entropy:0.7752931714057922 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967353940010071 - training/rollout_probs_diff_mean:0.004443757236003876 - training/rollout_probs_diff_std:0.008464205078780651 - training/rollout_actor_probs_pearson_corr:0.9995999336242676 - rollout_corr/rollout_is_mean:1.0000269412994385 - rollout_corr/rollout_is_ratio_fraction_high:9.492214303463697e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.492214303463697e-06 - rollout_corr/rollout_is_max:2.5489871501922607 - rollout_corr/rollout_is_min:0.2599738538265228 - rollout_corr/rollout_is_std:0.03524937480688095 - rollout_corr/rollout_is_eff_sample_size:0.99875914242478 - rollout_corr/rollout_is_seq_mean:0.9999639391899109 - rollout_corr/rollout_is_seq_std:0.0014692033873870969 - rollout_corr/rollout_is_seq_max:1.0044381618499756 - rollout_corr/rollout_is_seq_min:0.9949309825897217 - rollout_corr/rollout_is_seq_max_deviation:0.00506901741027832 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2062102765776217) - rollout_corr/training_log_ppl:np.float64(0.7724762546131387) - rollout_corr/kl:np.float64(0.0008447698922706692) - rollout_corr/k3_kl:np.float64(0.0006712018573580281) - rollout_corr/rollout_ppl:np.float64(2.204339082352817) - rollout_corr/rollout_log_ppl:np.float64(0.7716314849676564) - rollout_corr/log_ppl_diff:np.float64(0.0008447696454823017) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013305179309099913) - rollout_corr/log_ppl_diff_max:np.float64(0.005142807960510254) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035692453384399414) - rollout_corr/ppl_ratio:np.float64(1.0008462956175208) - rollout_corr/chi2_token:np.float64(0.0010051389690488577) - rollout_corr/chi2_seq:np.float64(3.9995412554126233) - actor/pg_loss:np.float64(6.170920096337795e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.559897284816543e-05) - actor/ppo_kl:np.float64(0.00016689712089679176) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20688721537590027) - perf/mfu/actor:np.float64(0.1360888320089338) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.50061416625977) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6782738095238096) - val-aux/sciknoweval/reward/std@16:np.float64(0.2194428960706272) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7662047619047618) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1707809803907824) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5911047619047619) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.19703442080325248) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6786285714285715) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2187494224175815) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8316666666666666) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1211692975832172) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.513647619047619) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.16233574997526817) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7021999999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.17154812598360286) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8789666666666667) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08054198286562807) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.44790952380952387) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.12433382532071413) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7142047619047619) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.12568969808131827) - val-aux/sciknoweval/reward/best@16/
(TaskRunner pid=2224264) 
Training Progress:  80%|████████  | 80/100 [1:21:33<37:37, 112.89s/it]
(TaskRunner pid=2224264) step:81 - global_seqlen/min:23435 - global_seqlen/max:40414 - global_seqlen/minmax_diff:16979 - global_seqlen/balanced_min:31718 - global_seqlen/balanced_max:31758 - global_seqlen/mean:31742.875 - actor/entropy:0.728011965751648 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25490427017211914 - training/rollout_probs_diff_mean:0.00432547414675355 - training/rollout_probs_diff_std:0.00843879021704197 - training/rollout_actor_probs_pearson_corr:0.9996644258499146 - rollout_corr/rollout_is_mean:0.9999246001243591 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.7127294995589182e-05 - rollout_corr/rollout_is_max:1.6446915864944458 - rollout_corr/rollout_is_min:0.3955734968185425 - rollout_corr/rollout_is_std:0.03461290895938873 - rollout_corr/rollout_is_eff_sample_size:0.9988031424060505 - rollout_corr/rollout_is_seq_mean:0.9999666810035706 - rollout_corr/rollout_is_seq_std:0.001528046908788383 - rollout_corr/rollout_is_seq_max:1.0054874420166016 - rollout_corr/rollout_is_seq_min:0.9947968125343323 - rollout_corr/rollout_is_seq_max_deviation:0.0054874420166015625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.084522245451808) - rollout_corr/training_log_ppl:np.float64(0.7124685312155634) - rollout_corr/kl:np.float64(0.0007158686560133631) - rollout_corr/k3_kl:np.float64(0.000656772480510881) - rollout_corr/rollout_ppl:np.float64(2.083047821652144) - rollout_corr/rollout_log_ppl:np.float64(0.7117526578949764) - rollout_corr/log_ppl_diff:np.float64(0.0007158733205869794) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012598995817825198) - rollout_corr/log_ppl_diff_max:np.float64(0.005965232849121094) - rollout_corr/log_ppl_diff_min:np.float64(-0.0034725069999694824) - rollout_corr/ppl_ratio:np.float64(1.000717243179679) - rollout_corr/chi2_token:np.float64(0.0012002624571323395) - rollout_corr/chi2_seq:np.float64(0.9090896914713085) - actor/pg_loss:np.float64(-3.0948780477046967e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011802908329627826) - actor/ppo_kl:np.float64(6.148918862525221e-05) - actor/pg_clipfrac_lower:np.float64(2.239822151750559e-06) - actor/grad_norm:np.float64(0.2090231031179428) - perf/mfu/actor:np.float64(0.12132654821351155) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(109.81432342529297) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:1 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0020438572391867638 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0020438572391867638 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:684.21484375 - response_length/max:1807.0 - response_length/min:189.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:684.21484375 - response_length_non_aborted/max:1807.0 - response_length_non_aborted/min:189.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:307.75 - prompt_length/max:735.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001064930111169815 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4930436480790377) - timing_s/agent_loop/generate_sequences/max:np.float64(14.782667718827724) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.092499443388078) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.782667718827724) - timing_s/a
(TaskRunner pid=2224264) 
Training Progress:  81%|████████  | 81/100 [1:22:12<28:42, 90.64s/it] 
(TaskRunner pid=2224264) step:82 - global_seqlen/min:30553 - global_seqlen/max:40093 - global_seqlen/minmax_diff:9540 - global_seqlen/balanced_min:34588 - global_seqlen/balanced_max:34791 - global_seqlen/mean:34631.0 - actor/entropy:0.7073292136192322 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48784658312797546 - training/rollout_probs_diff_mean:0.004207909107208252 - training/rollout_probs_diff_std:0.008302947506308556 - training/rollout_actor_probs_pearson_corr:0.9996032118797302 - rollout_corr/rollout_is_mean:0.9999532103538513 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4358463886310346e-05 - rollout_corr/rollout_is_max:1.861779808998108 - rollout_corr/rollout_is_min:0.3777696490287781 - rollout_corr/rollout_is_std:0.033776216208934784 - rollout_corr/rollout_is_eff_sample_size:0.998860348177861 - rollout_corr/rollout_is_seq_mean:0.9999899864196777 - rollout_corr/rollout_is_seq_std:0.0011922791600227356 - rollout_corr/rollout_is_seq_max:1.004441738128662 - rollout_corr/rollout_is_seq_min:0.9973618984222412 - rollout_corr/rollout_is_seq_max_deviation:0.004441738128662109 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0682692220434546) - rollout_corr/training_log_ppl:np.float64(0.7130721984431148) - rollout_corr/kl:np.float64(0.0005954480367265091) - rollout_corr/k3_kl:np.float64(0.0006088201273541927) - rollout_corr/rollout_ppl:np.float64(2.067028284072876) - rollout_corr/rollout_log_ppl:np.float64(0.712476747808978) - rollout_corr/log_ppl_diff:np.float64(0.000595450634136796) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011131800711154938) - rollout_corr/log_ppl_diff_max:np.float64(0.0041391849517822266) - rollout_corr/log_ppl_diff_min:np.float64(-0.00474703311920166) - rollout_corr/ppl_ratio:np.float64(1.0005964431911707) - rollout_corr/chi2_token:np.float64(0.0012475282419472933) - rollout_corr/chi2_seq:np.float64(2.9451562664471567) - actor/pg_loss:np.float64(0.00018601585179567337) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019039797803088732) - actor/ppo_kl:np.float64(-5.924717907390686e-07) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.25703465938568115) - perf/mfu/actor:np.float64(0.1307738033068337) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(109.3072280883789) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:1 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006862149457447231 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006862149457447231 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:816.15625 - response_length/max:2571.0 - response_length/min:362.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:816.15625 - response_length_non_aborted/max:2571.0 - response_length_non_aborted/min:362.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.0625 - prompt_length/max:408.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.565187454223633e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(4.610755139961839) - timing_s/agent_loop/generate_sequences/max:np.float64(20.74920883215964) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.369914709837758) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.74920883215964) - timing_s/agent_loop/slowest/tool_c
(TaskRunner pid=2224264) 
Training Progress:  82%|████████▏ | 82/100 [1:22:57<23:05, 76.96s/it]
(TaskRunner pid=2224264) step:83 - global_seqlen/min:24830 - global_seqlen/max:36777 - global_seqlen/minmax_diff:11947 - global_seqlen/balanced_min:31744 - global_seqlen/balanced_max:31974 - global_seqlen/mean:31783.0 - actor/entropy:0.7531368136405945 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553977966308594 - training/rollout_probs_diff_mean:0.0044290670193731785 - training/rollout_probs_diff_std:0.008538827300071716 - training/rollout_actor_probs_pearson_corr:0.999602735042572 - rollout_corr/rollout_is_mean:1.0000147819519043 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.6260687415488064e-06 - rollout_corr/rollout_is_max:1.5903000831604004 - rollout_corr/rollout_is_min:0.4172116219997406 - rollout_corr/rollout_is_std:0.03477611765265465 - rollout_corr/rollout_is_eff_sample_size:0.9987920825865458 - rollout_corr/rollout_is_seq_mean:0.9999656677246094 - rollout_corr/rollout_is_seq_std:0.0014977967366576195 - rollout_corr/rollout_is_seq_max:1.0056312084197998 - rollout_corr/rollout_is_seq_min:0.9933651685714722 - rollout_corr/rollout_is_seq_max_deviation:0.006634831428527832 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.162785124965012) - rollout_corr/training_log_ppl:np.float64(0.7481051302747801) - rollout_corr/kl:np.float64(0.0006437221610795518) - rollout_corr/k3_kl:np.float64(0.0006854262796878174) - rollout_corr/rollout_ppl:np.float64(2.161293275654316) - rollout_corr/rollout_log_ppl:np.float64(0.7474614095408469) - rollout_corr/log_ppl_diff:np.float64(0.0006437207339331508) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012794457143172622) - rollout_corr/log_ppl_diff_max:np.float64(0.0074920654296875) - rollout_corr/log_ppl_diff_min:np.float64(-0.004778653383255005) - rollout_corr/ppl_ratio:np.float64(1.000645110849291) - rollout_corr/chi2_token:np.float64(0.0014334383886307478) - rollout_corr/chi2_seq:np.float64(0.4927413531113416) - actor/pg_loss:np.float64(-0.00020378001499921083) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024108612069539959) - actor/ppo_kl:np.float64(-1.7973163554674265e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.23665034770965576) - perf/mfu/actor:np.float64(0.12241472211925314) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.91914367675781) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:1 - critic/score/mean:0.55078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008029807358980179 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008029807358980179 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:694.3125 - response_length/max:2176.0 - response_length/min:179.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:694.3125 - response_length_non_aborted/max:2176.0 - response_length_non_aborted/min:179.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.90625 - prompt_length/max:563.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010167807340621948 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2234518099576235) - timing_s/agent_loop/generate_sequences/max:np.float64(17.420780858024955) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.779617345870065) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.420780858024955) - timing_s/agent_loop/slowest/t
(TaskRunner pid=2224264) 
Training Progress:  83%|████████▎ | 83/100 [1:23:38<18:45, 66.22s/it]
(TaskRunner pid=2224264) step:84 - global_seqlen/min:20654 - global_seqlen/max:34303 - global_seqlen/minmax_diff:13649 - global_seqlen/balanced_min:29574 - global_seqlen/balanced_max:29701 - global_seqlen/mean:29614.0 - actor/entropy:0.6809946894645691 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43606680631637573 - training/rollout_probs_diff_mean:0.004484572447836399 - training/rollout_probs_diff_std:0.008784463629126549 - training/rollout_actor_probs_pearson_corr:0.999607264995575 - rollout_corr/rollout_is_mean:0.9998617172241211 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1636025192274246e-05 - rollout_corr/rollout_is_max:1.5042952299118042 - rollout_corr/rollout_is_min:0.4855743646621704 - rollout_corr/rollout_is_std:0.0344783253967762 - rollout_corr/rollout_is_eff_sample_size:0.9988122996462617 - rollout_corr/rollout_is_seq_mean:0.9998700618743896 - rollout_corr/rollout_is_seq_std:0.0014390589203685522 - rollout_corr/rollout_is_seq_max:1.0044506788253784 - rollout_corr/rollout_is_seq_min:0.9937525391578674 - rollout_corr/rollout_is_seq_max_deviation:0.006247460842132568 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9907695450820029) - rollout_corr/training_log_ppl:np.float64(0.670324468635954) - rollout_corr/kl:np.float64(0.0006556954003826831) - rollout_corr/k3_kl:np.float64(0.000665820000790518) - rollout_corr/rollout_ppl:np.float64(1.9894441780634224) - rollout_corr/rollout_log_ppl:np.float64(0.6696687689982355) - rollout_corr/log_ppl_diff:np.float64(0.0006556996377184987) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012462410377338529) - rollout_corr/log_ppl_diff_max:np.float64(0.007571697235107422) - rollout_corr/log_ppl_diff_min:np.float64(-0.004120081663131714) - rollout_corr/ppl_ratio:np.float64(1.000657090684399) - rollout_corr/chi2_token:np.float64(0.0013479453045874834) - rollout_corr/chi2_seq:np.float64(0.807735110167414) - actor/pg_loss:np.float64(-0.00014557677786797285) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001528950670035556) - actor/ppo_kl:np.float64(-9.21530613155197e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18504177778959274) - perf/mfu/actor:np.float64(0.1154202224364208) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.7387924194336) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:1 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01133712474256754 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01133712474256754 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:671.40625 - response_length/max:2226.0 - response_length/min:177.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:671.40625 - response_length_non_aborted/max:2226.0 - response_length_non_aborted/min:177.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.03125 - prompt_length/max:466.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017830543220043182 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6220184583216906) - timing_s/agent_loop/generate_sequences/max:np.float64(16.999597314745188) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.399542242870666) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.999597314745188) - timing_s/agent_loop/slowest/tool
(TaskRunner pid=2224264) 
Training Progress:  84%|████████▍ | 84/100 [1:24:20<15:41, 58.82s/it]
(TaskRunner pid=2224264) step:85 - global_seqlen/min:27367 - global_seqlen/max:37161 - global_seqlen/minmax_diff:9794 - global_seqlen/balanced_min:32947 - global_seqlen/balanced_max:32956 - global_seqlen/mean:32953.5 - actor/entropy:0.735228419303894 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5948740243911743 - training/rollout_probs_diff_mean:0.004468359053134918 - training/rollout_probs_diff_std:0.008735636249184608 - training/rollout_actor_probs_pearson_corr:0.9996408224105835 - rollout_corr/rollout_is_mean:0.9999686479568481 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0702284271246754e-05 - rollout_corr/rollout_is_max:1.7922857999801636 - rollout_corr/rollout_is_min:0.23469160497188568 - rollout_corr/rollout_is_std:0.03531694784760475 - rollout_corr/rollout_is_eff_sample_size:0.9987541480773114 - rollout_corr/rollout_is_seq_mean:1.0000033378601074 - rollout_corr/rollout_is_seq_std:0.0015298437792807817 - rollout_corr/rollout_is_seq_max:1.0047695636749268 - rollout_corr/rollout_is_seq_min:0.9963509440422058 - rollout_corr/rollout_is_seq_max_deviation:0.004769563674926758 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1323386910371482) - rollout_corr/training_log_ppl:np.float64(0.7435957858106121) - rollout_corr/kl:np.float64(0.0005658202565279602) - rollout_corr/k3_kl:np.float64(0.0006766391076098444) - rollout_corr/rollout_ppl:np.float64(2.1311398232355714) - rollout_corr/rollout_log_ppl:np.float64(0.7430299640400335) - rollout_corr/log_ppl_diff:np.float64(0.0005658217705786228) - rollout_corr/log_ppl_abs_diff:np.float64(0.00123508065007627) - rollout_corr/log_ppl_diff_max:np.float64(0.004446566104888916) - rollout_corr/log_ppl_diff_min:np.float64(-0.0032745003700256348) - rollout_corr/ppl_ratio:np.float64(1.000567066250369) - rollout_corr/chi2_token:np.float64(0.0015738557558506727) - rollout_corr/chi2_seq:np.float64(1.2473218482919037) - actor/pg_loss:np.float64(-0.00010735273826867342) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.374866226607992e-05) - actor/ppo_kl:np.float64(-8.093407468590108e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18677664920687675) - perf/mfu/actor:np.float64(0.1273840230570504) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.69750595092773) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:1 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00654712226241827 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00654712226241827 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:729.984375 - response_length/max:1647.0 - response_length/min:212.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:729.984375 - response_length_non_aborted/max:1647.0 - response_length_non_aborted/min:212.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.8125 - prompt_length/max:706.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014764443039894104 - timing_s/agent_loop/generate_sequences/min:np.float64(2.8548108004033566) - timing_s/agent_loop/generate_sequences/max:np.float64(14.612083107233047) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.586479216020962) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.612083107233047) - timing_s/agent_loop/slowest/tool_ca
(TaskRunner pid=2224264) 
Training Progress:  85%|████████▌ | 85/100 [1:24:58<13:11, 52.74s/it]
(TaskRunner pid=2224264) step:86 - global_seqlen/min:24044 - global_seqlen/max:35362 - global_seqlen/minmax_diff:11318 - global_seqlen/balanced_min:30989 - global_seqlen/balanced_max:31001 - global_seqlen/mean:30998.375 - actor/entropy:0.6628086566925049 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4241427183151245 - training/rollout_probs_diff_mean:0.004253617953509092 - training/rollout_probs_diff_std:0.008793003857135773 - training/rollout_actor_probs_pearson_corr:0.9995636940002441 - rollout_corr/rollout_is_mean:1.00003182888031 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.650387720379513e-06 - rollout_corr/rollout_is_max:1.8955367803573608 - rollout_corr/rollout_is_min:0.47840172052383423 - rollout_corr/rollout_is_std:0.03403814136981964 - rollout_corr/rollout_is_eff_sample_size:0.998842864637548 - rollout_corr/rollout_is_seq_mean:0.999988317489624 - rollout_corr/rollout_is_seq_std:0.001431034179404378 - rollout_corr/rollout_is_seq_max:1.0039067268371582 - rollout_corr/rollout_is_seq_min:0.9941695332527161 - rollout_corr/rollout_is_seq_max_deviation:0.0058304667472839355 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.980902411043644) - rollout_corr/training_log_ppl:np.float64(0.6618329326156527) - rollout_corr/kl:np.float64(0.0007042251129403265) - rollout_corr/k3_kl:np.float64(0.0006341270048437764) - rollout_corr/rollout_ppl:np.float64(1.979446857701987) - rollout_corr/rollout_log_ppl:np.float64(0.6611287064151838) - rollout_corr/log_ppl_diff:np.float64(0.0007042262004688382) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012508378131315112) - rollout_corr/log_ppl_diff_max:np.float64(0.006446421146392822) - rollout_corr/log_ppl_diff_min:np.float64(-0.0046353936195373535) - rollout_corr/ppl_ratio:np.float64(1.0007055627647787) - rollout_corr/chi2_token:np.float64(0.0011301424819976091) - rollout_corr/chi2_seq:np.float64(0.7622500413563102) - actor/pg_loss:np.float64(0.00021916942205280066) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00026571475882519735) - actor/ppo_kl:np.float64(8.79584136299627e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2480062022805214) - perf/mfu/actor:np.float64(0.12020480007027705) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.7301025390625) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:1 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0002775753091555089 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0002775753091555089 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:691.32421875 - response_length/max:1574.0 - response_length/min:164.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:691.32421875 - response_length_non_aborted/max:1574.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.375 - prompt_length/max:503.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017972663044929504 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9018322117626667) - timing_s/agent_loop/generate_sequences/max:np.float64(14.014412458986044) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.677346075157402) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.014412458986044) - timing_s/agent_loop/slowest
(TaskRunner pid=2224264) 
Training Progress:  86%|████████▌ | 86/100 [1:25:36<11:15, 48.26s/it]
(TaskRunner pid=2224264) step:87 - global_seqlen/min:27376 - global_seqlen/max:34559 - global_seqlen/minmax_diff:7183 - global_seqlen/balanced_min:30988 - global_seqlen/balanced_max:31014 - global_seqlen/mean:31004.375 - actor/entropy:0.6844121217727661 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45646268129348755 - training/rollout_probs_diff_mean:0.004233994521200657 - training/rollout_probs_diff_std:0.008540390990674496 - training/rollout_actor_probs_pearson_corr:0.9996417760848999 - rollout_corr/rollout_is_mean:1.000025749206543 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1601264304772485e-05 - rollout_corr/rollout_is_max:1.5985411405563354 - rollout_corr/rollout_is_min:0.412740021944046 - rollout_corr/rollout_is_std:0.033871378749608994 - rollout_corr/rollout_is_eff_sample_size:0.998854282390456 - rollout_corr/rollout_is_seq_mean:1.0000381469726562 - rollout_corr/rollout_is_seq_std:0.0015245388494804502 - rollout_corr/rollout_is_seq_max:1.0053566694259644 - rollout_corr/rollout_is_seq_min:0.9960179328918457 - rollout_corr/rollout_is_seq_max_deviation:0.0053566694259643555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0139922401867807) - rollout_corr/training_log_ppl:np.float64(0.6768635550397448) - rollout_corr/kl:np.float64(0.0006538237156501481) - rollout_corr/k3_kl:np.float64(0.0006354092881792894) - rollout_corr/rollout_ppl:np.float64(2.0125835882499814) - rollout_corr/rollout_log_ppl:np.float64(0.6762097296887077) - rollout_corr/log_ppl_diff:np.float64(0.0006538253510370851) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013526437105610967) - rollout_corr/log_ppl_diff_max:np.float64(0.00598597526550293) - rollout_corr/log_ppl_diff_min:np.float64(-0.005899667739868164) - rollout_corr/ppl_ratio:np.float64(1.0006554168649018) - rollout_corr/chi2_token:np.float64(0.0012414318043738604) - rollout_corr/chi2_seq:np.float64(0.7429793914780021) - actor/pg_loss:np.float64(-8.32361401990056e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030786197271481797) - actor/ppo_kl:np.float64(9.98133720919725e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2665778249502182) - perf/mfu/actor:np.float64(0.12025084389863877) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.70088958740234) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:1 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005423591006547213 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005423591006547213 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:673.41796875 - response_length/max:1719.0 - response_length/min:153.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:673.41796875 - response_length_non_aborted/max:1719.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.46875 - prompt_length/max:845.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.614036202430725e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9507676176726818) - timing_s/agent_loop/generate_sequences/max:np.float64(14.615875907242298) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.613317727045796) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.615875907242298) - timing_s/agent_loop/slowest
(TaskRunner pid=2224264) 
Training Progress:  87%|████████▋ | 87/100 [1:26:14<09:48, 45.29s/it]
(TaskRunner pid=2224264) step:88 - global_seqlen/min:25663 - global_seqlen/max:37476 - global_seqlen/minmax_diff:11813 - global_seqlen/balanced_min:30723 - global_seqlen/balanced_max:30761 - global_seqlen/mean:30750.875 - actor/entropy:0.6911125183105469 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.729590892791748 - training/rollout_probs_diff_mean:0.004324982408434153 - training/rollout_probs_diff_std:0.0088273286819458 - training/rollout_actor_probs_pearson_corr:0.9995751976966858 - rollout_corr/rollout_is_mean:0.9999687671661377 - rollout_corr/rollout_is_ratio_fraction_high:5.643627446261235e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.128725489252247e-05 - rollout_corr/rollout_is_max:3.966233491897583 - rollout_corr/rollout_is_min:0.06101220101118088 - rollout_corr/rollout_is_std:0.03435536473989487 - rollout_corr/rollout_is_eff_sample_size:0.9988211002691073 - rollout_corr/rollout_is_seq_mean:0.9998989105224609 - rollout_corr/rollout_is_seq_std:0.0017780093476176262 - rollout_corr/rollout_is_seq_max:1.012873888015747 - rollout_corr/rollout_is_seq_min:0.9941136837005615 - rollout_corr/rollout_is_seq_max_deviation:0.01287388801574707 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0185752864927053) - rollout_corr/training_log_ppl:np.float64(0.6845818213769235) - rollout_corr/kl:np.float64(0.0008537062007487606) - rollout_corr/k3_kl:np.float64(0.0007372877449256521) - rollout_corr/rollout_ppl:np.float64(2.0169089869596064) - rollout_corr/rollout_log_ppl:np.float64(0.6837281162734143) - rollout_corr/log_ppl_diff:np.float64(0.0008537051035091281) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014192980015650392) - rollout_corr/log_ppl_diff_max:np.float64(0.015641063451766968) - rollout_corr/log_ppl_diff_min:np.float64(-0.0030567049980163574) - rollout_corr/ppl_ratio:np.float64(1.0008557450491935) - rollout_corr/chi2_token:np.float64(0.0012827848549932241) - rollout_corr/chi2_seq:np.float64(1.2393478201702237) - actor/pg_loss:np.float64(-2.0556850358843803e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00026034799202534487) - actor/ppo_kl:np.float64(7.98297634219125e-05) - actor/pg_clipfrac_lower:np.float64(1.187310044770129e-05) - actor/grad_norm:np.float64(0.25147897377610207) - perf/mfu/actor:np.float64(0.11886404252645519) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.61107635498047) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:1 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0010137365898117423 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0010137365898117423 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:692.15234375 - response_length/max:1935.0 - response_length/min:145.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:692.15234375 - response_length_non_aborted/max:1935.0 - response_length_non_aborted/min:145.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.8125 - prompt_length/max:482.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.903677523136139e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8212964609265327) - timing_s/agent_loop/generate_sequences/max:np.float64(16.11359540745616) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.880805795379274) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.11359540
(TaskRunner pid=2224264) 
Training Progress:  88%|████████▊ | 88/100 [1:26:54<08:44, 43.68s/it]
(TaskRunner pid=2224264) step:89 - global_seqlen/min:30256 - global_seqlen/max:40801 - global_seqlen/minmax_diff:10545 - global_seqlen/balanced_min:34912 - global_seqlen/balanced_max:34937 - global_seqlen/mean:34929.25 - actor/entropy:0.7014224529266357 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5826672315597534 - training/rollout_probs_diff_mean:0.004312381613999605 - training/rollout_probs_diff_std:0.008647686801850796 - training/rollout_actor_probs_pearson_corr:0.9996581673622131 - rollout_corr/rollout_is_mean:1.0000931024551392 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.952648199221585e-05 - rollout_corr/rollout_is_max:1.9627834558486938 - rollout_corr/rollout_is_min:0.3244985342025757 - rollout_corr/rollout_is_std:0.03435710072517395 - rollout_corr/rollout_is_eff_sample_size:0.9988213381259312 - rollout_corr/rollout_is_seq_mean:1.000161051750183 - rollout_corr/rollout_is_seq_std:0.0014398518251255155 - rollout_corr/rollout_is_seq_max:1.0050846338272095 - rollout_corr/rollout_is_seq_min:0.9958447813987732 - rollout_corr/rollout_is_seq_max_deviation:0.005084633827209473 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.095038112718612) - rollout_corr/training_log_ppl:np.float64(0.7223394099273719) - rollout_corr/kl:np.float64(0.0004870060852217861) - rollout_corr/k3_kl:np.float64(0.0006450895384659816) - rollout_corr/rollout_ppl:np.float64(2.094029687810689) - rollout_corr/rollout_log_ppl:np.float64(0.7218523986521177) - rollout_corr/log_ppl_diff:np.float64(0.00048701127525418997) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011224675690755248) - rollout_corr/log_ppl_diff_max:np.float64(0.004818141460418701) - rollout_corr/log_ppl_diff_min:np.float64(-0.003881394863128662) - rollout_corr/ppl_ratio:np.float64(1.0004880586639047) - rollout_corr/chi2_token:np.float64(0.0016097698826342821) - rollout_corr/chi2_seq:np.float64(1.3918255360331386) - actor/pg_loss:np.float64(7.724866736680269e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001636164154206199) - actor/ppo_kl:np.float64(3.664902180844365e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.23648764193058014) - perf/mfu/actor:np.float64(0.1338068099945285) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.57841873168945) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:1 - critic/score/mean:0.546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012478643329814076 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0012478643329814076 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:800.1953125 - response_length/max:1701.0 - response_length/min:290.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:800.1953125 - response_length_non_aborted/max:1701.0 - response_length_non_aborted/min:290.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.34375 - prompt_length/max:720.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.920712232589722e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(3.7672608997672796) - timing_s/agent_loop/generate_sequences/max:np.float64(15.327744403854012) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.27638334871881) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.327744403854012) - timing_s/agent_loop/slowest/tool
(TaskRunner pid=2224264) 
Training Progress:  89%|████████▉ | 89/100 [1:27:34<07:45, 42.36s/it]
(TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90}
(TaskRunner pid=2224264) validation generation end
(TaskRunner pid=2224264) [prompt] system
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2224264) <reasoning>
(TaskRunner pid=2224264) ...
(TaskRunner pid=2224264) </reasoning>
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) ...
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) A
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) user
(TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2224264) Please reason step by step.
(TaskRunner pid=2224264) assistant
(TaskRunner pid=2224264) <think>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) </think>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) [response] <reasoning>
(TaskRunner pid=2224264) We are given a product molecule:  
(TaskRunner pid=2224264) **Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This is a nitrogen-containing heterocyclic compound with the following structural features:
(TaskRunner pid=2224264) - A ring with the formula Nc1ccn(...)n1, suggesting a pyrrole-like ring (a five-membered ring with a nitrogen).
(TaskRunner pid=2224264) - A substituent on the nitrogen: **Cc2ccc(Cl)cc2C(F)(F)F**, which is a benzene ring with a chlorine and a trifluoromethyl group attached to the same carbon.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) We are to determine which of the four options is the **only correct reactant** used in the synthesis of this product. Let's analyze the options one by one.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Option A:
(TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This molecule has a **benzene ring** (c2ccccc2), a **carbonyl group** (O=C), and a **trifluoromethyl group** (C(F)(F)F) attached to a carbon that is also connected to a **chlorine-substituted benzene ring**. Additionally, it contains a **pyrrole-like ring** with a nitrogen (N1) and a substituent **Cc2ccc(Cl)cc2C(F)(F)F**, which is **identical to the substituent in the product**.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This matches the **product** almost exactly, and it appears to be a **precursor** or **reactant** that could form the product via a reaction such as a **condensation or cyclization**.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Option B:
(TaskRunner pid=2224264) **O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This molecule has a **benzene ring** and a **trifluoromethyl group is missing**, and the nitrogen is attached to a **chlorine** (C(Cl)N1), not a **trifluoromethyl** group. It also has a **different structure** (e.g., a different substituent on the nitrogen), and the ring structure doesn't match the product's pyrrole-like ring.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) **Not a match.**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Option C:
(TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This has a **benzene ring**, a **trifluoromethyl group** (C(F)(F)F), and a **chlorine** (on a benzene ring). However, the **substituent on the nitrogen is different** (it has an **ether oxygen** and a different aromatic ring), and the **structure of the ring is different** from the product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) **Not a match.**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Option D:
(TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) This has a **trifluoromethyl group** and a **benzene ring**, but the **substituent on the nitrogen is not the same** as in the product (it has an ethyl group, not the same benzene ring with a chlorine). The **ring structure is also different**.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) **Not a match.**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Final Analysis:
(TaskRunner pid=2224264) Only **Option A** has:
(TaskRunner pid=2224264) - A **pyrrole-like ring** with a nitrogen.
(TaskRunner pid=2224264) - A **trifluoromethyl group** (C(F)(F)F).
(TaskRunner pid=2224264) - A **chlorine-substituted benzene ring** (Cc2ccc(Cl)cc2C(F)(F)F).
(TaskRunner pid=2224264) - The **same substituent as in the product**.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) Therefore, **Option A** is the only molecule that is a **valid reactant** for the synthesis of the given product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) </reasoning>
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) A
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) [ground_truth] A
(TaskRunner pid=2224264) [score] 1.0
(TaskRunner pid=2224264) [acc] 1.0
(TaskRunner pid=2224264) [pred] A
(TaskRunner pid=2224264) [incorrect_format] 1
(TaskRunner pid=2224264) [feedback] 
(TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90
(TaskRunner pid=2224264) step:90 - global_seqlen/min:26503 - global_seqlen/max:35349 - global_seqlen/minmax_diff:8846 - global_seqlen/balanced_min:32189 - global_seqlen/balanced_max:32202 - global_seqlen/mean:32197.5 - actor/entropy:0.6767091751098633 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24459987878799438 - training/rollout_probs_diff_mean:0.004204816184937954 - training/rollout_probs_diff_std:0.008432798087596893 - training/rollout_actor_probs_pearson_corr:0.999668538570404 - rollout_corr/rollout_is_mean:1.000195026397705 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.363540822145296e-06 - rollout_corr/rollout_is_max:1.6443707942962646 - rollout_corr/rollout_is_min:0.3406277298927307 - rollout_corr/rollout_is_std:0.033881932497024536 - rollout_corr/rollout_is_eff_sample_size:0.9988539255817266 - rollout_corr/rollout_is_seq_mean:1.0002124309539795 - rollout_corr/rollout_is_seq_std:0.0013589308364316821 - rollout_corr/rollout_is_seq_max:1.0058317184448242 - rollout_corr/rollout_is_seq_min:0.9959684610366821 - rollout_corr/rollout_is_seq_max_deviation:0.005831718444824219 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.031145083718002) - rollout_corr/training_log_ppl:np.float64(0.6887705723638646) - rollout_corr/kl:np.float64(0.0006024212293984088) - rollout_corr/k3_kl:np.float64(0.0006607583559343766) - rollout_corr/rollout_ppl:np.float64(2.0299026197753847) - rollout_corr/rollout_log_ppl:np.float64(0.6881681511295028) - rollout_corr/log_ppl_diff:np.float64(0.0006024212343618274) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012092863908037543) - rollout_corr/log_ppl_diff_max:np.float64(0.008667349815368652) - rollout_corr/log_ppl_diff_min:np.float64(-0.0036773681640625) - rollout_corr/ppl_ratio:np.float64(1.000603698194027) - rollout_corr/chi2_token:np.float64(0.0014522264245897532) - rollout_corr/chi2_seq:np.float64(0.6241840105503798) - actor/pg_loss:np.float64(-5.9257843531668186e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015337120976255392) - actor/ppo_kl:np.float64(0.00021126134707571964) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18526915088295937) - perf/mfu/actor:np.float64(0.11914403018889852) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.8291244506836) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6815476190476191) - val-aux/sciknoweval/reward/std@16:np.float64(0.2156221846747797) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7688476190476189) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.18144705579138373) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5926619047619047) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1824273460834292) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6801857142857143) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2154753727607372) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8427380952380952) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.13509620467575745) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5238857142857143) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.13671434851677694) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7026523809523809) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1762779201364988) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.895695238095238) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08638656437772123) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.47133333333333327) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09762932212980469) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7126714285714286) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13634398316845442) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9
(TaskRunner pid=2224264) 
Training Progress:  90%|█████████ | 90/100 [1:31:52<17:51, 107.11s/it]
(TaskRunner pid=2224264) step:91 - global_seqlen/min:27956 - global_seqlen/max:38739 - global_seqlen/minmax_diff:10783 - global_seqlen/balanced_min:31665 - global_seqlen/balanced_max:31695 - global_seqlen/mean:31683.5 - actor/entropy:0.7154297232627869 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9744127988815308 - training/rollout_probs_diff_mean:0.004367751069366932 - training/rollout_probs_diff_std:0.00891902670264244 - training/rollout_actor_probs_pearson_corr:0.9995865821838379 - rollout_corr/rollout_is_mean:0.9999821186065674 - rollout_corr/rollout_is_ratio_fraction_high:5.548773515329231e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.1097547030658461e-05 - rollout_corr/rollout_is_max:2.104722261428833 - rollout_corr/rollout_is_min:0.00197325786575675 - rollout_corr/rollout_is_std:0.03477783128619194 - rollout_corr/rollout_is_eff_sample_size:0.9987919636650865 - rollout_corr/rollout_is_seq_mean:0.9999819993972778 - rollout_corr/rollout_is_seq_std:0.0012956903083249927 - rollout_corr/rollout_is_seq_max:1.00333833694458 - rollout_corr/rollout_is_seq_min:0.9953454732894897 - rollout_corr/rollout_is_seq_max_deviation:0.004654526710510254 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.104408088605851) - rollout_corr/training_log_ppl:np.float64(0.7243244127603248) - rollout_corr/kl:np.float64(0.0006612393046498966) - rollout_corr/k3_kl:np.float64(0.0006583722558843874) - rollout_corr/rollout_ppl:np.float64(2.102920087054372) - rollout_corr/rollout_log_ppl:np.float64(0.7236631693085656) - rollout_corr/log_ppl_diff:np.float64(0.0006612434517592192) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012168984394520521) - rollout_corr/log_ppl_diff_max:np.float64(0.00693821907043457) - rollout_corr/log_ppl_diff_min:np.float64(-0.0027844905853271484) - rollout_corr/ppl_ratio:np.float64(1.0006624751258641) - rollout_corr/chi2_token:np.float64(0.001317617017775774) - rollout_corr/chi2_seq:np.float64(1.2634925008751452) - actor/pg_loss:np.float64(0.0001394905848428607) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015355198502220446) - actor/ppo_kl:np.float64(-4.81430525312021e-06) - actor/pg_clipfrac_lower:np.float64(4.3841191654792055e-06) - actor/grad_norm:np.float64(0.22441749274730682) - perf/mfu/actor:np.float64(0.1235221491000339) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(107.04840850830078) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006852735532447696 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006852735532447696 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:703.984375 - response_length/max:1848.0 - response_length/min:173.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:703.984375 - response_length_non_aborted/max:1848.0 - response_length_non_aborted/min:173.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.125 - prompt_length/max:1009.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.789582014083862e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.103301614522934) - timing_s/agent_loop/generate_sequences/max:np.float64(15.148089375346899) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.013559678431193) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.14808937534689
(TaskRunner pid=2224264) 
Training Progress:  91%|█████████ | 91/100 [1:32:30<12:59, 86.56s/it] 
(TaskRunner pid=2224264) step:92 - global_seqlen/min:28907 - global_seqlen/max:37134 - global_seqlen/minmax_diff:8227 - global_seqlen/balanced_min:32432 - global_seqlen/balanced_max:32477 - global_seqlen/mean:32465.625 - actor/entropy:0.7178817987442017 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2975451946258545 - training/rollout_probs_diff_mean:0.004235805943608284 - training/rollout_probs_diff_std:0.008502726443111897 - training/rollout_actor_probs_pearson_corr:0.9996208548545837 - rollout_corr/rollout_is_mean:0.9999630451202393 - rollout_corr/rollout_is_ratio_fraction_high:5.372733539843466e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.1497116088867188 - rollout_corr/rollout_is_min:0.550010621547699 - rollout_corr/rollout_is_std:0.03401536867022514 - rollout_corr/rollout_is_eff_sample_size:0.9988440539746302 - rollout_corr/rollout_is_seq_mean:0.9999423623085022 - rollout_corr/rollout_is_seq_std:0.0013055680319666862 - rollout_corr/rollout_is_seq_max:1.004227876663208 - rollout_corr/rollout_is_seq_min:0.9956101775169373 - rollout_corr/rollout_is_seq_max_deviation:0.004389822483062744 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.064156681764871) - rollout_corr/training_log_ppl:np.float64(0.7071324991993606) - rollout_corr/kl:np.float64(0.0008166503134958703) - rollout_corr/k3_kl:np.float64(0.0006099401913388647) - rollout_corr/rollout_ppl:np.float64(2.0623925644904375) - rollout_corr/rollout_log_ppl:np.float64(0.7063158513628878) - rollout_corr/log_ppl_diff:np.float64(0.0008166478364728391) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012665537069551647) - rollout_corr/log_ppl_diff_max:np.float64(0.007030129432678223) - rollout_corr/log_ppl_diff_min:np.float64(-0.0036208629608154297) - rollout_corr/ppl_ratio:np.float64(1.0008179140277207) - rollout_corr/chi2_token:np.float64(0.0008041651453822851) - rollout_corr/chi2_seq:np.float64(0.6329832167830318) - actor/pg_loss:np.float64(6.987596862018108e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012492035489231057) - actor/ppo_kl:np.float64(0.0001699282564437965) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.19231200218200684) - perf/mfu/actor:np.float64(0.12487351766630518) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(106.44847011566162) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:1 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004429147113114595 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004429147113114595 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:727.05078125 - response_length/max:1984.0 - response_length/min:182.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:727.05078125 - response_length_non_aborted/max:1984.0 - response_length_non_aborted/min:182.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:287.5 - prompt_length/max:436.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00019785761833190918 - timing_s/agent_loop/generate_sequences/min:np.float64(2.329558404162526) - timing_s/agent_loop/generate_sequences/max:np.float64(16.53738614730537) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.280053526119445) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.53738614730537) - timing_s/agent_loop/slowest
(TaskRunner pid=2224264) 
Training Progress:  92%|█████████▏| 92/100 [1:33:11<09:41, 72.72s/it]
(TaskRunner pid=2224264) step:93 - global_seqlen/min:23013 - global_seqlen/max:34946 - global_seqlen/minmax_diff:11933 - global_seqlen/balanced_min:27908 - global_seqlen/balanced_max:27913 - global_seqlen/mean:27910.875 - actor/entropy:0.7500928044319153 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1930849552154541 - training/rollout_probs_diff_mean:0.004663003142923117 - training/rollout_probs_diff_std:0.008796103298664093 - training/rollout_actor_probs_pearson_corr:0.9996201992034912 - rollout_corr/rollout_is_mean:0.999956488609314 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6296987533569336 - rollout_corr/rollout_is_min:0.6042289137840271 - rollout_corr/rollout_is_std:0.035632822662591934 - rollout_corr/rollout_is_eff_sample_size:0.9987316741318075 - rollout_corr/rollout_is_seq_mean:0.9998897910118103 - rollout_corr/rollout_is_seq_std:0.0015105620259419084 - rollout_corr/rollout_is_seq_max:1.0058465003967285 - rollout_corr/rollout_is_seq_min:0.9952189922332764 - rollout_corr/rollout_is_seq_max_deviation:0.005846500396728516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1263672169297934) - rollout_corr/training_log_ppl:np.float64(0.7347684952546842) - rollout_corr/kl:np.float64(0.0007506872085336624) - rollout_corr/k3_kl:np.float64(0.0006735208150985272) - rollout_corr/rollout_ppl:np.float64(2.124785815831274) - rollout_corr/rollout_log_ppl:np.float64(0.7340178099111654) - rollout_corr/log_ppl_diff:np.float64(0.0007506853435188532) - rollout_corr/log_ppl_abs_diff:np.float64(0.001359031768515706) - rollout_corr/log_ppl_diff_max:np.float64(0.005597174167633057) - rollout_corr/log_ppl_diff_min:np.float64(-0.003313422203063965) - rollout_corr/ppl_ratio:np.float64(1.000752320047468) - rollout_corr/chi2_token:np.float64(0.0011984859593212605) - rollout_corr/chi2_seq:np.float64(0.6925807157531381) - actor/pg_loss:np.float64(-4.779419396072626e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.594229848313262e-05) - actor/ppo_kl:np.float64(-7.275951552143667e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18249434977769852) - perf/mfu/actor:np.float64(0.11041731672624022) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(106.24386596679688) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:1 - critic/score/mean:0.8515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007838345132768154 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007838345132768154 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:605.74609375 - response_length/max:1369.0 - response_length/min:190.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:605.74609375 - response_length_non_aborted/max:1369.0 - response_length_non_aborted/min:190.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:709.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018016062676906586 - timing_s/agent_loop/generate_sequences/min:np.float64(2.355575904250145) - timing_s/agent_loop/generate_sequences/max:np.float64(11.529610631987453) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.815270844308543) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.529610631987453) - timing_s/agent_loop/slowest/tool_calls:n
(TaskRunner pid=2224264) 
Training Progress:  93%|█████████▎| 93/100 [1:33:46<07:09, 61.36s/it]
(TaskRunner pid=2224264) step:94 - global_seqlen/min:24960 - global_seqlen/max:37338 - global_seqlen/minmax_diff:12378 - global_seqlen/balanced_min:30359 - global_seqlen/balanced_max:30372 - global_seqlen/mean:30366.25 - actor/entropy:0.7063392400741577 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3128166198730469 - training/rollout_probs_diff_mean:0.004224883858114481 - training/rollout_probs_diff_std:0.008449923247098923 - training/rollout_actor_probs_pearson_corr:0.9996349811553955 - rollout_corr/rollout_is_mean:0.9998794198036194 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.6113572100002784e-06 - rollout_corr/rollout_is_max:1.5920956134796143 - rollout_corr/rollout_is_min:0.19120460748672485 - rollout_corr/rollout_is_std:0.03371969982981682 - rollout_corr/rollout_is_eff_sample_size:0.99886415419973 - rollout_corr/rollout_is_seq_mean:0.9998659491539001 - rollout_corr/rollout_is_seq_std:0.0013268847251310945 - rollout_corr/rollout_is_seq_max:1.0026715993881226 - rollout_corr/rollout_is_seq_min:0.9956828355789185 - rollout_corr/rollout_is_seq_max_deviation:0.004317164421081543 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.097409463953227) - rollout_corr/training_log_ppl:np.float64(0.7194324630545452) - rollout_corr/kl:np.float64(0.0006619715687250505) - rollout_corr/k3_kl:np.float64(0.0006161111765550231) - rollout_corr/rollout_ppl:np.float64(2.0959334070794284) - rollout_corr/rollout_log_ppl:np.float64(0.7187704919488169) - rollout_corr/log_ppl_diff:np.float64(0.0006619711057282984) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012567761004902422) - rollout_corr/log_ppl_diff_max:np.float64(0.004305779933929443) - rollout_corr/log_ppl_diff_min:np.float64(-0.004627227783203125) - rollout_corr/ppl_ratio:np.float64(1.0006632735021412) - rollout_corr/chi2_token:np.float64(0.00113762472756207) - rollout_corr/chi2_seq:np.float64(2.2418907159008086) - actor/pg_loss:np.float64(9.933649562299252e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.07081788580399e-05) - actor/ppo_kl:np.float64(-6.808040846806307e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1753209587186575) - perf/mfu/actor:np.float64(0.11873190434052154) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(106.19572830200195) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:1 - critic/score/mean:0.72265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.72265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006172493100166321 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006172493100166321 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:696.1328125 - response_length/max:1462.0 - response_length/min:227.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:696.1328125 - response_length_non_aborted/max:1462.0 - response_length_non_aborted/min:227.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:252.8125 - prompt_length/max:384.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.6177377700805664e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.743150135502219) - timing_s/agent_loop/generate_sequences/max:np.float64(13.47325548902154) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.079402086157643) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.47325548902154) - timing_s/agent_loop/slowest/t
(TaskRunner pid=2224264) 
Training Progress:  94%|█████████▍| 94/100 [1:34:23<05:24, 54.02s/it]
(TaskRunner pid=2224264) step:95 - global_seqlen/min:20976 - global_seqlen/max:34422 - global_seqlen/minmax_diff:13446 - global_seqlen/balanced_min:29028 - global_seqlen/balanced_max:29035 - global_seqlen/mean:29031.75 - actor/entropy:0.712871789932251 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2599616050720215 - training/rollout_probs_diff_mean:0.004302844405174255 - training/rollout_probs_diff_std:0.008476506918668747 - training/rollout_actor_probs_pearson_corr:0.9996457099914551 - rollout_corr/rollout_is_mean:1.0001041889190674 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.065090474294266e-06 - rollout_corr/rollout_is_max:1.6687859296798706 - rollout_corr/rollout_is_min:0.4844895303249359 - rollout_corr/rollout_is_std:0.03429459035396576 - rollout_corr/rollout_is_eff_sample_size:0.9988256195681333 - rollout_corr/rollout_is_seq_mean:1.0001366138458252 - rollout_corr/rollout_is_seq_std:0.0015432714717462659 - rollout_corr/rollout_is_seq_max:1.005467414855957 - rollout_corr/rollout_is_seq_min:0.9945797324180603 - rollout_corr/rollout_is_seq_max_deviation:0.005467414855957031 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0434681265614927) - rollout_corr/training_log_ppl:np.float64(0.6912423392059281) - rollout_corr/kl:np.float64(0.0005482688078317643) - rollout_corr/k3_kl:np.float64(0.0006431571834468741) - rollout_corr/rollout_ppl:np.float64(2.0423742262646556) - rollout_corr/rollout_log_ppl:np.float64(0.6906940678600222) - rollout_corr/log_ppl_diff:np.float64(0.0005482713459059596) - rollout_corr/log_ppl_abs_diff:np.float64(0.001258363830856979) - rollout_corr/log_ppl_diff_max:np.float64(0.005878806114196777) - rollout_corr/log_ppl_diff_min:np.float64(-0.005048781633377075) - rollout_corr/ppl_ratio:np.float64(1.000549679622054) - rollout_corr/chi2_token:np.float64(0.0014886993449181318) - rollout_corr/chi2_seq:np.float64(3.0285099220927805) - actor/pg_loss:np.float64(-0.000160741969011724) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010068776555272052) - actor/ppo_kl:np.float64(7.83663859635908e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.23990549892187119) - perf/mfu/actor:np.float64(0.11424140158128934) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(106.15832138061523) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:1 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.014393975958228111 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.014393975958228111 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:644.0546875 - response_length/max:1596.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:644.0546875 - response_length_non_aborted/max:1596.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.1875 - prompt_length/max:646.0 - prompt_length/min:159.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.495019912719727e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.937621222808957) - timing_s/agent_loop/generate_sequences/max:np.float64(13.714711025357246) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.356954876297095) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.714711025357246) - timing_s/agent_loop/slowest/tool
(TaskRunner pid=2224264) 
Training Progress:  95%|█████████▌| 95/100 [1:35:00<04:05, 49.08s/it]
(TaskRunner pid=2224264) step:96 - global_seqlen/min:27198 - global_seqlen/max:35708 - global_seqlen/minmax_diff:8510 - global_seqlen/balanced_min:32177 - global_seqlen/balanced_max:32213 - global_seqlen/mean:32201.875 - actor/entropy:0.7203529477119446 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6764640808105469 - training/rollout_probs_diff_mean:0.004217594396322966 - training/rollout_probs_diff_std:0.008475503884255886 - training/rollout_actor_probs_pearson_corr:0.999649703502655 - rollout_corr/rollout_is_mean:1.0000306367874146 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.28119744558353e-06 - rollout_corr/rollout_is_max:1.7523001432418823 - rollout_corr/rollout_is_min:0.0011872303439304233 - rollout_corr/rollout_is_std:0.03390303626656532 - rollout_corr/rollout_is_eff_sample_size:0.9988520226061409 - rollout_corr/rollout_is_seq_mean:0.9999527931213379 - rollout_corr/rollout_is_seq_std:0.0014494654024019837 - rollout_corr/rollout_is_seq_max:1.0033957958221436 - rollout_corr/rollout_is_seq_min:0.9943961501121521 - rollout_corr/rollout_is_seq_max_deviation:0.0056038498878479 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0765154217369854) - rollout_corr/training_log_ppl:np.float64(0.7100228805211373) - rollout_corr/kl:np.float64(0.0007996692028076069) - rollout_corr/k3_kl:np.float64(0.0006997747711352531) - rollout_corr/rollout_ppl:np.float64(2.074850105214864) - rollout_corr/rollout_log_ppl:np.float64(0.7092232063878328) - rollout_corr/log_ppl_diff:np.float64(0.0007996741333045065) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013392329565249383) - rollout_corr/log_ppl_diff_max:np.float64(0.006226241588592529) - rollout_corr/log_ppl_diff_min:np.float64(-0.002697765827178955) - rollout_corr/ppl_ratio:np.float64(1.0008012598846108) - rollout_corr/chi2_token:np.float64(0.0011850390583276749) - rollout_corr/chi2_seq:np.float64(0.8988333453889936) - actor/pg_loss:np.float64(-6.152375135570765e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024159732652151433) - actor/ppo_kl:np.float64(0.00012709150854451137) - actor/pg_clipfrac_lower:np.float64(4.822531082027126e-06) - actor/grad_norm:np.float64(0.21260051056742668) - perf/mfu/actor:np.float64(0.12350445771061563) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(106.11673736572266) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01730252243578434 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01730252243578434 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:739.65234375 - response_length/max:1942.0 - response_length/min:151.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:739.65234375 - response_length_non_aborted/max:1942.0 - response_length_non_aborted/min:151.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.65625 - prompt_length/max:456.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010816939175128937 - timing_s/agent_loop/generate_sequences/min:np.float64(1.915278973057866) - timing_s/agent_loop/generate_sequences/max:np.float64(16.20021486841142) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.129882725828793) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.20021486841142) - timing_s/ag
(TaskRunner pid=2224264) 
Training Progress:  96%|█████████▌| 96/100 [1:35:40<03:05, 46.34s/it]
(TaskRunner pid=2224264) step:97 - global_seqlen/min:31313 - global_seqlen/max:43694 - global_seqlen/minmax_diff:12381 - global_seqlen/balanced_min:35934 - global_seqlen/balanced_max:35960 - global_seqlen/mean:35950.125 - actor/entropy:0.7323668003082275 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2915082573890686 - training/rollout_probs_diff_mean:0.004103551618754864 - training/rollout_probs_diff_std:0.008156946860253811 - training/rollout_actor_probs_pearson_corr:0.9996232986450195 - rollout_corr/rollout_is_mean:0.9999871850013733 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.7118023758230265e-06 - rollout_corr/rollout_is_max:1.6210092306137085 - rollout_corr/rollout_is_min:0.4609464108943939 - rollout_corr/rollout_is_std:0.03369848057627678 - rollout_corr/rollout_is_eff_sample_size:0.9988657004043994 - rollout_corr/rollout_is_seq_mean:1.000003695487976 - rollout_corr/rollout_is_seq_std:0.0014436065685003996 - rollout_corr/rollout_is_seq_max:1.0053772926330566 - rollout_corr/rollout_is_seq_min:0.9946680665016174 - rollout_corr/rollout_is_seq_max_deviation:0.005377292633056641 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1260625598952174) - rollout_corr/training_log_ppl:np.float64(0.7336573095526546) - rollout_corr/kl:np.float64(0.0006526818591758854) - rollout_corr/k3_kl:np.float64(0.0006329746193500796) - rollout_corr/rollout_ppl:np.float64(2.124627975281328) - rollout_corr/rollout_log_ppl:np.float64(0.7330046314746141) - rollout_corr/log_ppl_diff:np.float64(0.0006526780780404806) - rollout_corr/log_ppl_abs_diff:np.float64(0.001235815929248929) - rollout_corr/log_ppl_diff_max:np.float64(0.00612562894821167) - rollout_corr/log_ppl_diff_min:np.float64(-0.00426250696182251) - rollout_corr/ppl_ratio:np.float64(1.0006539451424032) - rollout_corr/chi2_token:np.float64(0.0012377453967928886) - rollout_corr/chi2_seq:np.float64(0.6915090307593346) - actor/pg_loss:np.float64(-3.366696182638407e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016491950168529002) - actor/ppo_kl:np.float64(6.479799488445792e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20394239574670792) - perf/mfu/actor:np.float64(0.135031184027903) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(106.04956817626953) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:1 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012476264499127865 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012476264499127865 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:829.03515625 - response_length/max:2245.0 - response_length/min:215.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:829.03515625 - response_length_non_aborted/max:2245.0 - response_length_non_aborted/min:215.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:294.40625 - prompt_length/max:900.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010967254638671875 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4484062399715185) - timing_s/agent_loop/generate_sequences/max:np.float64(18.13408470340073) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.178321859493735) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.13408470340073) - timing_s/agent_loop/slowes
(TaskRunner pid=2224264) 
Training Progress:  97%|█████████▋| 97/100 [1:36:22<02:15, 45.13s/it]
(TaskRunner pid=2224264) step:98 - global_seqlen/min:25644 - global_seqlen/max:42510 - global_seqlen/minmax_diff:16866 - global_seqlen/balanced_min:34661 - global_seqlen/balanced_max:34705 - global_seqlen/mean:34687.75 - actor/entropy:0.7501354813575745 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19508248567581177 - training/rollout_probs_diff_mean:0.004144626669585705 - training/rollout_probs_diff_std:0.008178996853530407 - training/rollout_actor_probs_pearson_corr:0.9996928572654724 - rollout_corr/rollout_is_mean:1.0000689029693604 - rollout_corr/rollout_is_ratio_fraction_high:4.974382136424538e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.2251296043395996 - rollout_corr/rollout_is_min:0.6142621636390686 - rollout_corr/rollout_is_std:0.03403989225625992 - rollout_corr/rollout_is_eff_sample_size:0.9988425078369757 - rollout_corr/rollout_is_seq_mean:1.000027060508728 - rollout_corr/rollout_is_seq_std:0.001423987210728228 - rollout_corr/rollout_is_seq_max:1.0042808055877686 - rollout_corr/rollout_is_seq_min:0.9920931458473206 - rollout_corr/rollout_is_seq_max_deviation:0.007906854152679443 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1749815698713064) - rollout_corr/training_log_ppl:np.float64(0.7591774334432557) - rollout_corr/kl:np.float64(0.0006937429601876488) - rollout_corr/k3_kl:np.float64(0.0006346612378251848) - rollout_corr/rollout_ppl:np.float64(2.17338493373245) - rollout_corr/rollout_log_ppl:np.float64(0.7584836900932714) - rollout_corr/log_ppl_diff:np.float64(0.0006937433499842882) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011937296949326992) - rollout_corr/log_ppl_diff_max:np.float64(0.004764825105667114) - rollout_corr/log_ppl_diff_min:np.float64(-0.003519713878631592) - rollout_corr/ppl_ratio:np.float64(1.000694976421073) - rollout_corr/chi2_token:np.float64(0.0011504706926643848) - rollout_corr/chi2_seq:np.float64(0.7677441604901105) - actor/pg_loss:np.float64(-0.00023154309019446373) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014778738523091306) - actor/ppo_kl:np.float64(0.00011894125558598034) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.22566131502389908) - perf/mfu/actor:np.float64(0.13134675089113737) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.8719253540039) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:1 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0074124508537352085 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0074124508537352085 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:785.2734375 - response_length/max:2121.0 - response_length/min:178.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:785.2734375 - response_length_non_aborted/max:2121.0 - response_length_non_aborted/min:178.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:631.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.526918590068817e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.268034750595689) - timing_s/agent_loop/generate_sequences/max:np.float64(17.475637735798955) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.994100491654535) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.475637735798955) - timing_s/agent_loop/slow
(TaskRunner pid=2224264) 
Training Progress:  98%|█████████▊| 98/100 [1:37:04<01:28, 44.09s/it]
(TaskRunner pid=2224264) step:99 - global_seqlen/min:31172 - global_seqlen/max:42762 - global_seqlen/minmax_diff:11590 - global_seqlen/balanced_min:36579 - global_seqlen/balanced_max:36620 - global_seqlen/mean:36605.125 - actor/entropy:0.7388753890991211 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2583175301551819 - training/rollout_probs_diff_mean:0.004119738936424255 - training/rollout_probs_diff_std:0.008252698928117752 - training/rollout_actor_probs_pearson_corr:0.9996559023857117 - rollout_corr/rollout_is_mean:1.0000630617141724 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.395273082191125e-06 - rollout_corr/rollout_is_max:1.7297682762145996 - rollout_corr/rollout_is_min:0.46209102869033813 - rollout_corr/rollout_is_std:0.03351224586367607 - rollout_corr/rollout_is_eff_sample_size:0.998878308098044 - rollout_corr/rollout_is_seq_mean:1.0000331401824951 - rollout_corr/rollout_is_seq_std:0.0012539547169581056 - rollout_corr/rollout_is_seq_max:1.0045143365859985 - rollout_corr/rollout_is_seq_min:0.9951211214065552 - rollout_corr/rollout_is_seq_max_deviation:0.004878878593444824 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.149727716576308) - rollout_corr/training_log_ppl:np.float64(0.7441099616116844) - rollout_corr/kl:np.float64(0.0006301436051412423) - rollout_corr/k3_kl:np.float64(0.0006160447873639896) - rollout_corr/rollout_ppl:np.float64(2.1482933391816914) - rollout_corr/rollout_log_ppl:np.float64(0.7434798121103086) - rollout_corr/log_ppl_diff:np.float64(0.000630149501375854) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011455799685791135) - rollout_corr/log_ppl_diff_max:np.float64(0.0055887699127197266) - rollout_corr/log_ppl_diff_min:np.float64(-0.00408819317817688) - rollout_corr/ppl_ratio:np.float64(1.0006313105113804) - rollout_corr/chi2_token:np.float64(0.0012098602019250393) - rollout_corr/chi2_seq:np.float64(0.841775347944349) - actor/pg_loss:np.float64(-8.723500650376081e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012781737518707814) - actor/ppo_kl:np.float64(8.237922273224285e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.19401156529784203) - perf/mfu/actor:np.float64(0.13594204140089314) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.93074417114258) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:1 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006903177127242088 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006903177127242088 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:831.53515625 - response_length/max:1998.0 - response_length/min:192.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:831.53515625 - response_length_non_aborted/max:1998.0 - response_length_non_aborted/min:192.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.375 - prompt_length/max:836.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.847611904144287e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.493832629173994) - timing_s/agent_loop/generate_sequences/max:np.float64(17.390929045155644) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.606087130981905) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.390929045155644) - timing_s/agent_loop/slo
(TaskRunner pid=2224264) 
Training Progress:  99%|█████████▉| 99/100 [1:37:46<00:43, 43.41s/it]
(TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100}
(TaskRunner pid=2224264) validation generation end
(TaskRunner pid=2224264) [prompt] system
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2224264) <reasoning>
(TaskRunner pid=2224264) ...
(TaskRunner pid=2224264) </reasoning>
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) ...
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) A
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) user
(TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2224264) Please reason step by step.
(TaskRunner pid=2224264) assistant
(TaskRunner pid=2224264) <think>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) </think>
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) [response] <reasoning>
(TaskRunner pid=2224264) We are given a target product:  
(TaskRunner pid=2224264) **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"**  
(TaskRunner pid=2224264) This is a chemical formula representing a molecule. Let's break it down:
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) - The main structure is: **Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1**
(TaskRunner pid=2224264) - This can be interpreted as: a ring with a nitrogen (N) at position 1, and a substituent at position 1 (the "C..." part).
(TaskRunner pid=2224264) - The substituent is: **Cc2ccc(Cl)cc2C(F)(F)F**  
(TaskRunner pid=2224264)   This is a benzene ring (c2ccc...), with a chlorine (Cl) on one carbon, and a trifluoromethyl group (C(F)(F)F) on another carbon. The entire group is attached to a carbon (C) which is attached to the nitrogen (N) in the ring.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) Now, we are to identify which of the four options is the **only correct reactant** that would be used in the synthesis of this product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) Let’s analyze each option:
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) **Option A:**  
(TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) - This is a complex molecule.
(TaskRunner pid=2224264) - It contains a ring (C1) with two double bonds (c2ccccc2), and a carbonyl group (O=C).
(TaskRunner pid=2224264) - Then, **C(=O)N1** suggests a carbonyl group attached to a nitrogen (N1).
(TaskRunner pid=2224264) - Then, **c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** matches the target product's ring and substituent.
(TaskRunner pid=2224264) - So, this option contains the full structure of the target product, possibly as a derivative or a starting point for synthesis.  
(TaskRunner pid=2224264) ✅ **Possibly a correct reactant.**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) **Option B:**  
(TaskRunner pid=2224264) **O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) - Contains a ring with Cl attached.
(TaskRunner pid=2224264) - The rest of the structure does not match the substituent **Cc2ccc(Cl)cc2C(F)(F)F**.
(TaskRunner pid=2224264) - **Does not match the target structure.**
(TaskRunner pid=2224264) ❌ **Not correct.**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) **Option C:**  
(TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) - This has a ring with a substituent that includes a **C(F)(F)F** group.
(TaskRunner pid=2224264) - However, the **Oc2ccc(Cl)c...** is different from **Cc2ccc(Cl)cc2C(F)(F)F**.
(TaskRunner pid=2224264) - The structure is not matching the target.
(TaskRunner pid=2224264) ❌ **Not correct.**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) **Option D:**  
(TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) - Contains a **C(F)(F)F** group.
(TaskRunner pid=2224264) - However, the **CC=C(Cc1ccccc1)** is not matching the **Cc2ccc(Cl)cc2C(F)(F)F** part.
(TaskRunner pid=2224264) - No Cl is present in the substituent.
(TaskRunner pid=2224264) ❌ **Not correct.**
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ---
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) ### Final Evaluation:
(TaskRunner pid=2224264) Only **Option A** contains the exact substituent **Cc2ccc(Cl)cc2C(F)(F)F**, which is part of the target molecule, and the rest of the structure is consistent with the ring and functional group of the product.
(TaskRunner pid=2224264) 
(TaskRunner pid=2224264) So, **Option A** is the only correct reactant that could be used in the synthesis of the given product.
(TaskRunner pid=2224264) </reasoning>
(TaskRunner pid=2224264) <answer>
(TaskRunner pid=2224264) A
(TaskRunner pid=2224264) </answer>
(TaskRunner pid=2224264) [ground_truth] A
(TaskRunner pid=2224264) [score] 1.0
(TaskRunner pid=2224264) [acc] 1.0
(TaskRunner pid=2224264) [pred] A
(TaskRunner pid=2224264) [incorrect_format] 1
(TaskRunner pid=2224264) [feedback] 
(TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_100
(TaskRunner pid=2224264) step:100 - global_seqlen/min:32181 - global_seqlen/max:40888 - global_seqlen/minmax_diff:8707 - global_seqlen/balanced_min:36112 - global_seqlen/balanced_max:36131 - global_seqlen/mean:36125.5 - actor/entropy:0.7533097267150879 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2046983242034912 - training/rollout_probs_diff_mean:0.0041552940383553505 - training/rollout_probs_diff_std:0.00816031452268362 - training/rollout_actor_probs_pearson_corr:0.9996163845062256 - rollout_corr/rollout_is_mean:0.999994695186615 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.690695732278982e-06 - rollout_corr/rollout_is_max:1.4722437858581543 - rollout_corr/rollout_is_min:0.08076818287372589 - rollout_corr/rollout_is_std:0.03385905548930168 - rollout_corr/rollout_is_eff_sample_size:0.9988548770722385 - rollout_corr/rollout_is_seq_mean:0.9999972581863403 - rollout_corr/rollout_is_seq_std:0.0013772835955023766 - rollout_corr/rollout_is_seq_max:1.0046179294586182 - rollout_corr/rollout_is_seq_min:0.9932000041007996 - rollout_corr/rollout_is_seq_max_deviation:0.0067999958992004395 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1957929017953575) - rollout_corr/training_log_ppl:np.float64(0.7637257019523531) - rollout_corr/kl:np.float64(0.0006403183531915602) - rollout_corr/k3_kl:np.float64(0.0006129170444637566) - rollout_corr/rollout_ppl:np.float64(2.194364957511425) - rollout_corr/rollout_log_ppl:np.float64(0.7630853808368556) - rollout_corr/log_ppl_diff:np.float64(0.0006403211154974997) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010970584698952734) - rollout_corr/log_ppl_diff_max:np.float64(0.007486581802368164) - rollout_corr/log_ppl_diff_min:np.float64(-0.0030992627143859863) - rollout_corr/ppl_ratio:np.float64(1.000641372287646) - rollout_corr/chi2_token:np.float64(0.0011783752124756575) - rollout_corr/chi2_seq:np.float64(1.0062491311691701) - actor/pg_loss:np.float64(1.9038328900933266e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016279506189675885) - actor/ppo_kl:np.float64(4.644049759150448e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20593710616230965) - perf/mfu/actor:np.float64(0.13613751711674105) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.93215942382812) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6901785714285714) - val-aux/sciknoweval/reward/std@16:np.float64(0.21497342883717385) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7771380952380953) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.17605374457438094) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6025857142857143) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.18719434569533389) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6898333333333334) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.21523251642703473) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8483380952380953) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.128790863116223) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5304333333333333) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.14423738321883778) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7146857142857143) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.17575431812173817) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8984857142857143) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08310585797508012) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.47553809523809526) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10546043434417916) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7290142857142858) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13381993498072356) - val-aux/sciknoweval/reward/best@16/mean:np.floa
(TaskRunner pid=2224264) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': "
(TaskRunner pid=2224264)  "np.float64(0.6901785714285714), 'val-aux/sciknoweval/reward/std@16': "
(TaskRunner pid=2224264)  "np.float64(0.21497342883717385), 'val-aux/sciknoweval/reward/best@2/mean': "
(TaskRunner pid=2224264)  "np.float64(0.7771380952380953), 'val-aux/sciknoweval/reward/best@2/std': "
(TaskRunner pid=2224264)  "np.float64(0.17605374457438094), 'val-aux/sciknoweval/reward/worst@2/mean': "
(TaskRunner pid=2224264)  "np.float64(0.6025857142857143), 'val-aux/sciknoweval/reward/worst@2/std': "
(TaskRunner pid=2224264)  "np.float64(0.18719434569533389), 'val-aux/sciknoweval/reward/maj@2/mean': "
(TaskRunner pid=2224264)  "np.float64(0.6898333333333334), 'val-aux/sciknoweval/reward/maj@2/std': "
(TaskRunner pid=2224264)  "np.float64(0.21523251642703473), 'val-aux/sciknoweval/reward/best@4/mean': "
(TaskRunner pid=2224264)  "np.float64(0.8483380952380953), 'val-aux/sciknoweval/reward/best@4/std': "
(TaskRunner pid=2224264)  "np.float64(0.128790863116223), 'val-aux/sciknoweval/reward/worst@4/mean': "
(TaskRunner pid=2224264)  "np.float64(0.5304333333333333), 'val-aux/sciknoweval/reward/worst@4/std': "
(TaskRunner pid=2224264)  "np.float64(0.14423738321883778), 'val-aux/sciknoweval/reward/maj@4/mean': "
(TaskRunner pid=2224264)  "np.float64(0.7146857142857143), 'val-aux/sciknoweval/reward/maj@4/std': "
(TaskRunner pid=2224264)  "np.float64(0.17575431812173817), 'val-aux/sciknoweval/reward/best@8/mean': "
(TaskRunner pid=2224264)  "np.float64(0.8984857142857143), 'val-aux/sciknoweval/reward/best@8/std': "
(TaskRunner pid=2224264)  "np.float64(0.08310585797508012), 'val-aux/sciknoweval/reward/worst@8/mean': "
(TaskRunner pid=2224264)  "np.float64(0.47553809523809526), 'val-aux/sciknoweval/reward/worst@8/std': "
(TaskRunner pid=2224264)  "np.float64(0.10546043434417916), 'val-aux/sciknoweval/reward/maj@8/mean': "
(TaskRunner pid=2224264)  "np.float64(0.7290142857142858), 'val-aux/sciknoweval/reward/maj@8/std': "
(TaskRunner pid=2224264)  "np.float64(0.13381993498072356), 'val-aux/sciknoweval/reward/best@16/mean': "
(TaskRunner pid=2224264)  "np.float64(0.9295428571428572), 'val-aux/sciknoweval/reward/best@16/std': "
(TaskRunner pid=2224264)  'np.float64(0.046578487700934236), '
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/reward/worst@16/mean': np.float64(0.4326952380952381), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/reward/worst@16/std': np.float64(0.07212858795272402), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/reward/maj@16/mean': np.float64(0.7341666666666667), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/reward/maj@16/std': np.float64(0.10114866635456188), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/mean@16': np.float64(0.6901785714285714), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/std@16': np.float64(0.21497342883717385), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/best@2/mean': np.float64(0.7771380952380953), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/best@2/std': np.float64(0.17605374457438094), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/worst@2/mean': np.float64(0.6025857142857143), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/worst@2/std': np.float64(0.18719434569533389), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/maj@2/mean': np.float64(0.6898333333333334), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/maj@2/std': np.float64(0.21523251642703473), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/best@4/mean': np.float64(0.8483380952380953), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/best@4/std': np.float64(0.128790863116223), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/worst@4/mean': np.float64(0.5304333333333333), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/worst@4/std': np.float64(0.14423738321883778), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/maj@4/mean': np.float64(0.7146857142857143), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/maj@4/std': np.float64(0.17575431812173817), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/best@8/mean': np.float64(0.8984857142857143), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/best@8/std': np.float64(0.08310585797508012), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/worst@8/mean': np.float64(0.47553809523809526), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/worst@8/std': np.float64(0.10546043434417916), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/maj@8/mean': np.float64(0.7290142857142858), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/maj@8/std': np.float64(0.13381993498072356), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/best@16/mean': np.float64(0.9295428571428572), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/best@16/std': np.float64(0.046578487700934236), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/worst@16/mean': np.float64(0.4326952380952381), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/worst@16/std': np.float64(0.07212858795272402), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/maj@16/mean': np.float64(0.7341666666666667), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/score/maj@16/std': np.float64(0.10114866635456188), "
(TaskRunner pid=2224264)  "'val-core/sciknoweval/acc/mean@16': np.float64(0.6901785714285714), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/acc/std@16': np.float64(0.21497342883717385), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/acc/best@2/mean': np.float64(0.7771380952380953), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/acc/best@2/std': np.float64(0.17605374457438094), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/acc/worst@2/mean': np.float64(0.6025857142857143), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/acc/worst@2/std': np.float64(0.18719434569533389), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/acc/maj@2/mean': np.float64(0.6898333333333334), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/acc/maj@2/std': np.float64(0.21523251642703473), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/acc/best@4/mean': np.float64(0.8483380952380953), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/acc/best@4/std': np.float64(0.128790863116223), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/acc/worst@4/mean': np.float64(0.5304333333333333), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/acc/worst@4/std': np.float64(0.14423738321883778), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/acc/maj@4/mean': np.float64(0.7146857142857143), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/acc/maj@4/std': np.float64(0.17575431812173817), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/acc/best@8/mean': np.float64(0.8984857142857143), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/acc/best@8/std': np.float64(0.08310585797508012), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/acc/worst@8/mean': np.float64(0.47553809523809526), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/acc/worst@8/std': np.float64(0.10546043434417916), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/acc/maj@8/mean': np.float64(0.7290142857142858), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/acc/maj@8/std': np.float64(0.13381993498072356), "
(TaskRunner pid=2224264)  "'val-core/sciknoweval/acc/best@16/mean': np.float64(0.9295428571428572), "
(TaskRunner pid=2224264)  "'val-core/sciknoweval/acc/best@16/std': np.float64(0.046578487700934236), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/acc/worst@16/mean': np.float64(0.4326952380952381), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/acc/worst@16/std': np.float64(0.07212858795272402), "
(TaskRunner pid=2224264)  "'val-core/sciknoweval/acc/maj@16/mean': np.float64(0.7341666666666667), "
(TaskRunner pid=2224264)  "'val-core/sciknoweval/acc/maj@16/std': np.float64(0.10114866635456188), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/mean@16': "
(TaskRunner pid=2224264)  'np.float64(0.9645833333333333), '
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/std@16': "
(TaskRunner pid=2224264)  'np.float64(0.08812713073175192), '
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/best@2/mean': "
(TaskRunner pid=2224264)  'np.float64(0.9939428571428572), '
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/best@2/std': "
(TaskRunner pid=2224264)  'np.float64(0.03432708478331659), '
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/worst@2/mean': "
(TaskRunner pid=2224264)  'np.float64(0.9356666666666668), '
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/worst@2/std': "
(TaskRunner pid=2224264)  'np.float64(0.11103403939640785), '
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/maj@2/mean': "
(TaskRunner pid=2224264)  'np.float64(0.9648952380952381), '
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/maj@2/std': "
(TaskRunner pid=2224264)  'np.float64(0.08777577522757704), '
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/best@4/mean': "
(TaskRunner pid=2224264)  'np.float64(0.9996333333333334), '
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/best@4/std': "
(TaskRunner pid=2224264)  'np.float64(0.004692755954297982), '
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/worst@4/mean': "
(TaskRunner pid=2224264)  'np.float64(0.890790476190476), '
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/worst@4/std': "
(TaskRunner pid=2224264)  'np.float64(0.12713323596686327), '
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/maj@4/mean': "
(TaskRunner pid=2224264)  'np.float64(0.9825095238095238), '
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/maj@4/std': "
(TaskRunner pid=2224264)  'np.float64(0.06141628384478388), '
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/worst@8/mean': "
(TaskRunner pid=2224264)  'np.float64(0.8332238095238094), '
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/worst@8/std': "
(TaskRunner pid=2224264)  'np.float64(0.12414222258147589), '
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/maj@8/mean': "
(TaskRunner pid=2224264)  'np.float64(0.9934761904761904), '
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/maj@8/std': "
(TaskRunner pid=2224264)  'np.float64(0.03401010083743055), '
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), "
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/worst@16/mean': "
(TaskRunner pid=2224264)  'np.float64(0.7765571428571428), '
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/worst@16/std': "
(TaskRunner pid=2224264)  'np.float64(0.0965096914106855), '
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/maj@16/mean': "
(TaskRunner pid=2224264)  'np.float64(0.9985285714285715), '
(TaskRunner pid=2224264)  "'val-aux/sciknoweval/incorrect_format/maj@16/std': "
(TaskRunner pid=2224264)  "np.float64(0.011901839678247213), 'val-aux/num_turns/min': np.int32(2), "
(TaskRunner pid=2224264)  "'val-aux/num_turns/max': np.int32(2), 'val-aux/num_turns/mean': "
(TaskRunner pid=2224264)  'np.float64(2.0)}')
(TaskRunner pid=2224264) 
Training Progress: 100%|██████████| 100/100 [1:42:46<00:00, 120.29s/it]
Training Progress: 100%|██████████| 100/100 [1:42:46<00:00, 61.66s/it] 
(TaskRunner pid=2224264) wandb: uploading output.log; uploading wandb-summary.json
(TaskRunner pid=2224264) wandb: uploading output.log
(TaskRunner pid=2224264) wandb: uploading history steps 98-99, summary, console lines 1082-1211
(TaskRunner pid=2224264) wandb: 
(TaskRunner pid=2224264) wandb: Run history:
(TaskRunner pid=2224264) wandb:           actor/entropy ▁▁▂▃▄▄▅▅▅▅▅▅▅▆▆▆▆▆▆▄▆▅▇▇▆▆█▆▆▆▆▆▆▇▆▅▅▆▅▆
(TaskRunner pid=2224264) wandb:         actor/grad_norm █▇▇█▅▅▆▇▅▆▅▄▅▆▄▄▅▄▃▄▃▄▃▂▃▅▄▃▂▂▂▃▁▃▁▂▁▃▂▂
(TaskRunner pid=2224264) wandb:           actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
(TaskRunner pid=2224264) wandb:                actor/lr ▁▃▆█████████████████████████████████████
(TaskRunner pid=2224264) wandb:       actor/pg_clipfrac ▇▃██▆▆▆▄▃▄▃▅▅▄▅▆▆▄▅▄▃▅▄▇▃▇▆▃▃▁▄▃▄▆▃▆▄▃▂▃
(TaskRunner pid=2224264) wandb: actor/pg_clipfrac_lower ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█▁▁▁▁▅▁▁▁▁▁▁▁▁▁▅▁▁▁▁█▁▁▁▁
(TaskRunner pid=2224264) wandb:           actor/pg_loss ▂▂▅▆▂▃▃█▂▄▃▁▄▆▅▄▇▆▄▅▃▆▄▃▄▂▅▄▄▄▅▅▅▆▃▄▅▄▄▄
(TaskRunner pid=2224264) wandb:            actor/ppo_kl ▆▆▁▄▄▄▆▇▅▄▇▅▃▆▇▅▄▄█▆▃▃▄▅▅▄▇▅▁▄▅▆▇▄▄▃▂▇▄▅
(TaskRunner pid=2224264) wandb:   critic/advantages/max ███████████████████████████████▁████████
(TaskRunner pid=2224264) wandb:  critic/advantages/mean ▃▄▄▆▄▇▆▄▆▄▃▂▁▃▄▁▄▄▅▅▄▂▂▅▄▃▄█▆▇▇▆▇▃▅▄▃▁▇▆
(TaskRunner pid=2224264) wandb:                    +204 ...
(TaskRunner pid=2224264) wandb: 
(TaskRunner pid=2224264) wandb: Run summary:
(TaskRunner pid=2224264) wandb:           actor/entropy 0.75331
(TaskRunner pid=2224264) wandb:         actor/grad_norm 0.20594
(TaskRunner pid=2224264) wandb:           actor/kl_loss 0
(TaskRunner pid=2224264) wandb:                actor/lr 0.0
(TaskRunner pid=2224264) wandb:       actor/pg_clipfrac 0.00016
(TaskRunner pid=2224264) wandb: actor/pg_clipfrac_lower 0
(TaskRunner pid=2224264) wandb:           actor/pg_loss 2e-05
(TaskRunner pid=2224264) wandb:            actor/ppo_kl 5e-05
(TaskRunner pid=2224264) wandb:   critic/advantages/max 0.875
(TaskRunner pid=2224264) wandb:  critic/advantages/mean 0.00256
(TaskRunner pid=2224264) wandb:                    +204 ...
(TaskRunner pid=2224264) wandb: 
(TaskRunner pid=2224264) wandb: 🚀 View run qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/ffovp19e
(TaskRunner pid=2224264) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root
(TaskRunner pid=2224264) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)
(TaskRunner pid=2224264) wandb: Find logs at: ./wandb/run-20260702_235814-ffovp19e/logs
(TaskRunner pid=2224264) Exception ignored in atexit callback: <function _start_and_connect_service.<locals>.teardown_atexit at 0x7797755e3880>
(TaskRunner pid=2224264) Traceback (most recent call last):
(TaskRunner pid=2224264)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 65, in teardown_atexit
(TaskRunner pid=2224264)     conn.teardown(hooks.exit_code)
(TaskRunner pid=2224264)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 299, in teardown
(TaskRunner pid=2224264)     self._asyncer.run(publish_teardown_and_close)
(TaskRunner pid=2224264)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run
(TaskRunner pid=2224264)     return future.result()
(TaskRunner pid=2224264)            ^^^^^^^^^^^^^^^
(TaskRunner pid=2224264)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result
(TaskRunner pid=2224264)     return self.__get_result()
(TaskRunner pid=2224264)            ^^^^^^^^^^^^^^^^^^^
(TaskRunner pid=2224264)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
(TaskRunner pid=2224264)     raise self._exception
(TaskRunner pid=2224264)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap
(TaskRunner pid=2224264)     return await fn()
(TaskRunner pid=2224264)            ^^^^^^^^^^
(TaskRunner pid=2224264)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 297, in publish_teardown_and_close
(TaskRunner pid=2224264)     await self._client.close()
(TaskRunner pid=2224264)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 69, in close
(TaskRunner pid=2224264)     await self._writer.wait_closed()
(TaskRunner pid=2224264)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 364, in wait_closed
(TaskRunner pid=2224264)     await self._protocol._get_close_waiter(self)
(TaskRunner pid=2224264) BrokenPipeError: [Errno 32] Broken pipe
main_ppo exit code: 0
[2026-07-03 01:41:07] Finished Qwen/Qwen3-8B chemistry grpo.
[2026-07-03 01:41:07] Starting Qwen/Qwen3-8B chemistry rlsd_tr.
Experiment: qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8
Dataset:    chemistry
Method:     rlsd_tr
Config:     rlsd
Model:      Qwen/Qwen3-8B
Train:      /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet
Val:        /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet
Ray tmp:    /tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_8B
2026-07-03 01:41:09,351	INFO scripts.py:1364 -- Did not find any active Ray processes.
Experiment: qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8
Config: rlsd
Task: datasets/sciknoweval/chemistry
Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-RLSD-TR-GRPO-matched-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_8B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/chemistry +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-8B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 trainer.max_actor_ckpt_to_keep=100 actor_rollout_ref.model.use_fused_kernels=False actor_rollout_ref.actor.policy_loss.loss_mode=rlsd actor_rollout_ref.actor.self_distillation.token_reweight_lambda=0.5 actor_rollout_ref.actor.self_distillation.token_reweight_eps_w=0.2 actor_rollout_ref.actor.self_distillation.token_reweight_decay_steps=0 actor_rollout_ref.actor.self_distillation.teacher_regularization=trust-region actor_rollout_ref.actor.self_distillation.teacher_update_rate=0.1 actor_rollout_ref.actor.self_distillation.max_reprompt_len=10240
ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_8B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/chemistry', 'EXPERIMENT': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}}
2026-07-03 01:41:19,207	INFO worker.py:2007 -- Started a local Ray instance.
/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
  warnings.warn(
(TaskRunner pid=2255397) TaskRunner hostname: mole-workspace-rw, PID: 2255397
(TaskRunner pid=2255397) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig',
(TaskRunner pid=2255397)                                  'calculate_entropy': False,
(TaskRunner pid=2255397)                                  'calculate_sum_pi_squared': False,
(TaskRunner pid=2255397)                                  'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig',
(TaskRunner pid=2255397)                                                 'async_save': False,
(TaskRunner pid=2255397)                                                 'load_contents': ['model',
(TaskRunner pid=2255397)                                                                   'optimizer',
(TaskRunner pid=2255397)                                                                   'extra'],
(TaskRunner pid=2255397)                                                 'save_contents': ['model',
(TaskRunner pid=2255397)                                                                   'optimizer',
(TaskRunner pid=2255397)                                                                   'extra']},
(TaskRunner pid=2255397)                                  'clip_ratio': 0.2,
(TaskRunner pid=2255397)                                  'clip_ratio_c': 3.0,
(TaskRunner pid=2255397)                                  'clip_ratio_high': 0.28,
(TaskRunner pid=2255397)                                  'clip_ratio_low': 0.2,
(TaskRunner pid=2255397)                                  'data_loader_seed': 42,
(TaskRunner pid=2255397)                                  'entropy_checkpointing': False,
(TaskRunner pid=2255397)                                  'entropy_coeff': 0,
(TaskRunner pid=2255397)                                  'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2255397)                                  'freeze_vision_tower': False,
(TaskRunner pid=2255397)                                  'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2255397)                                                  'dtype': 'bfloat16',
(TaskRunner pid=2255397)                                                  'entropy_checkpointing': False,
(TaskRunner pid=2255397)                                                  'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2255397)                                                  'forward_only': False,
(TaskRunner pid=2255397)                                                  'forward_prefetch': False,
(TaskRunner pid=2255397)                                                  'fsdp_size': -1,
(TaskRunner pid=2255397)                                                  'full_determinism': False,
(TaskRunner pid=2255397)                                                  'model_dtype': 'fp32',
(TaskRunner pid=2255397)                                                  'offload_policy': False,
(TaskRunner pid=2255397)                                                  'optimizer_offload': False,
(TaskRunner pid=2255397)                                                  'param_offload': False,
(TaskRunner pid=2255397)                                                  'reshard_after_forward': True,
(TaskRunner pid=2255397)                                                  'seed': 42,
(TaskRunner pid=2255397)                                                  'strategy': 'fsdp',
(TaskRunner pid=2255397)                                                  'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2255397)                                                  'use_orig_params': False,
(TaskRunner pid=2255397)                                                  'use_torch_compile': True,
(TaskRunner pid=2255397)                                                  'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2255397)                                  'grad_clip': 1.0,
(TaskRunner pid=2255397)                                  'kl_loss_coef': 0.001,
(TaskRunner pid=2255397)                                  'kl_loss_type': 'low_var_kl',
(TaskRunner pid=2255397)                                  'loss_agg_mode': 'token-mean',
(TaskRunner pid=2255397)                                  'loss_scale_factor': None,
(TaskRunner pid=2255397)                                  'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig',
(TaskRunner pid=2255397)                                            'betas': [0.9, 0.999],
(TaskRunner pid=2255397)                                            'clip_grad': 1.0,
(TaskRunner pid=2255397)                                            'lr': 1e-06,
(TaskRunner pid=2255397)                                            'lr_scheduler_type': 'constant',
(TaskRunner pid=2255397)                                            'lr_warmup_steps': 10,
(TaskRunner pid=2255397)                                            'lr_warmup_steps_ratio': 0.0,
(TaskRunner pid=2255397)                                            'min_lr_ratio': 0.0,
(TaskRunner pid=2255397)                                            'num_cycles': 0.5,
(TaskRunner pid=2255397)                                            'optimizer': 'AdamW',
(TaskRunner pid=2255397)                                            'optimizer_impl': 'torch.optim',
(TaskRunner pid=2255397)                                            'override_optimizer_config': None,
(TaskRunner pid=2255397)                                            'total_training_steps': -1,
(TaskRunner pid=2255397)                                            'warmup_style': None,
(TaskRunner pid=2255397)                                            'weight_decay': 0.01},
(TaskRunner pid=2255397)                                  'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig',
(TaskRunner pid=2255397)                                                  'clip_cov_lb': 1.0,
(TaskRunner pid=2255397)                                                  'clip_cov_ratio': 0.0002,
(TaskRunner pid=2255397)                                                  'clip_cov_ub': 5.0,
(TaskRunner pid=2255397)                                                  'kl_cov_ratio': 0.0002,
(TaskRunner pid=2255397)                                                  'loss_mode': 'rlsd',
(TaskRunner pid=2255397)                                                  'ppo_kl_coef': 0.1},
(TaskRunner pid=2255397)                                  'ppo_epochs': 1,
(TaskRunner pid=2255397)                                  'ppo_max_token_len_per_gpu': 10240,
(TaskRunner pid=2255397)                                  'ppo_micro_batch_size': None,
(TaskRunner pid=2255397)                                  'ppo_micro_batch_size_per_gpu': 1,
(TaskRunner pid=2255397)                                  'ppo_mini_batch_size': 8,
(TaskRunner pid=2255397)                                  'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2255397)                                               'all_ranks': False,
(TaskRunner pid=2255397)                                               'enable': False,
(TaskRunner pid=2255397)                                               'ranks': [],
(TaskRunner pid=2255397)                                               'save_path': 'outputs/profile',
(TaskRunner pid=2255397)                                               'tool': None,
(TaskRunner pid=2255397)                                               'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2255397)                                                                       'analysis': True,
(TaskRunner pid=2255397)                                                                       'contents': [],
(TaskRunner pid=2255397)                                                                       'discrete': False,
(TaskRunner pid=2255397)                                                                       'level': 'level0'},
(TaskRunner pid=2255397)                                                               'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2255397)                                                                        'discrete': False},
(TaskRunner pid=2255397)                                                               'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2255397)                                                                         'step_end': None,
(TaskRunner pid=2255397)                                                                         'step_start': 0},
(TaskRunner pid=2255397)                                                               'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2255397)                                                                                'stack_depth': 32,
(TaskRunner pid=2255397)                                                                                'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2255397)                                  'rollout_n': 8,
(TaskRunner pid=2255397)                                  'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig',
(TaskRunner pid=2255397)                                                    'mode': 'disabled',
(TaskRunner pid=2255397)                                                    'record_file': None,
(TaskRunner pid=2255397)                                                    'replay_file': None},
(TaskRunner pid=2255397)                                  'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig',
(TaskRunner pid=2255397)                                                        'alpha': 0.5,
(TaskRunner pid=2255397)                                                        'distillation_add_tail': True,
(TaskRunner pid=2255397)                                                        'distillation_topk': 100,
(TaskRunner pid=2255397)                                                        'dont_reprompt_on_self_success': True,
(TaskRunner pid=2255397)                                                        'environment_feedback_only_without_solution': True,
(TaskRunner pid=2255397)                                                        'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig',
(TaskRunner pid=2255397)                                                                             'enable': False,
(TaskRunner pid=2255397)                                                                             'loss_weight': 0.0,
(TaskRunner pid=2255397)                                                                             'mask': 'all'},
(TaskRunner pid=2255397)                                                        'feedback_template': '\n'
(TaskRunner pid=2255397)                                                                             'The '
(TaskRunner pid=2255397)                                                                             'following '
(TaskRunner pid=2255397)                                                                             'is '
(TaskRunner pid=2255397)                                                                             'feedback '
(TaskRunner pid=2255397)                                                                             'from '
(TaskRunner pid=2255397)                                                                             'your '
(TaskRunner pid=2255397)                                                                             'unsuccessful '
(TaskRunner pid=2255397)                                                                             'earlier '
(TaskRunner pid=2255397)                                                                             'attempt:\n'
(TaskRunner pid=2255397)                                                                             '\n'
(TaskRunner pid=2255397)                                                                             '{feedback_raw}',
(TaskRunner pid=2255397)                                                        'full_logit_distillation': True,
(TaskRunner pid=2255397)                                                        'include_environment_feedback': True,
(TaskRunner pid=2255397)                                                        'is_clip': 2,
(TaskRunner pid=2255397)                                                        'max_reprompt_len': 10240,
(TaskRunner pid=2255397)                                                        'remove_thinking_from_demonstration': False,
(TaskRunner pid=2255397)                                                        'reprompt_template': '{prompt}{solution}{feedback}\n'
(TaskRunner pid=2255397)                                                                             '\n'
(TaskRunner pid=2255397)                                                                             'Correctly '
(TaskRunner pid=2255397)                                                                             'solve '
(TaskRunner pid=2255397)                                                                             'the '
(TaskRunner pid=2255397)                                                                             'original '
(TaskRunner pid=2255397)                                                                             'question.',
(TaskRunner pid=2255397)                                                        'reprompt_truncation': 'right',
(TaskRunner pid=2255397)                                                        'solution_template': '\n'
(TaskRunner pid=2255397)                                                                             'Correct '
(TaskRunner pid=2255397)                                                                             'solution:\n'
(TaskRunner pid=2255397)                                                                             '\n'
(TaskRunner pid=2255397)                                                                             '{successful_previous_attempt}',
(TaskRunner pid=2255397)                                                        'srpo_dynamic_weighting': False,
(TaskRunner pid=2255397)                                                        'srpo_dynamic_weighting_temperature': 1.0,
(TaskRunner pid=2255397)                                                        'success_reward_threshold': 0.5,
(TaskRunner pid=2255397)                                                        'teacher_regularization': 'trust-region',
(TaskRunner pid=2255397)                                                        'teacher_update_rate': 0.1,
(TaskRunner pid=2255397)                                                        'token_reweight_decay_steps': 0,
(TaskRunner pid=2255397)                                                        'token_reweight_eps_w': 0.2,
(TaskRunner pid=2255397)                                                        'token_reweight_lambda': 0.5},
(TaskRunner pid=2255397)                                  'shuffle': False,
(TaskRunner pid=2255397)                                  'strategy': 'fsdp',
(TaskRunner pid=2255397)                                  'sum_pi_squared_checkpointing': False,
(TaskRunner pid=2255397)                                  'tau_neg': 1.05,
(TaskRunner pid=2255397)                                  'tau_pos': 1.0,
(TaskRunner pid=2255397)                                  'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2255397)                                  'use_dynamic_bsz': False,
(TaskRunner pid=2255397)                                  'use_fused_kernels': False,
(TaskRunner pid=2255397)                                  'use_kl_loss': False,
(TaskRunner pid=2255397)                                  'use_prefix_grouper': False,
(TaskRunner pid=2255397)                                  'use_remove_padding': True,
(TaskRunner pid=2255397)                                  'use_torch_compile': True},
(TaskRunner pid=2255397)                        'hybrid_engine': True,
(TaskRunner pid=2255397)                        'model': {'_target_': 'verl.workers.config.HFModelConfig',
(TaskRunner pid=2255397)                                  'custom_chat_template': None,
(TaskRunner pid=2255397)                                  'enable_activation_offload': False,
(TaskRunner pid=2255397)                                  'enable_gradient_checkpointing': True,
(TaskRunner pid=2255397)                                  'exclude_modules': None,
(TaskRunner pid=2255397)                                  'external_lib': None,
(TaskRunner pid=2255397)                                  'fused_kernel_options': {'impl_backend': 'torch'},
(TaskRunner pid=2255397)                                  'hf_config_path': None,
(TaskRunner pid=2255397)                                  'lora_adapter_path': None,
(TaskRunner pid=2255397)                                  'lora_alpha': 16,
(TaskRunner pid=2255397)                                  'lora_rank': 0,
(TaskRunner pid=2255397)                                  'override_config': {},
(TaskRunner pid=2255397)                                  'path': 'Qwen/Qwen3-8B',
(TaskRunner pid=2255397)                                  'target_modules': 'all-linear',
(TaskRunner pid=2255397)                                  'tiled_mlp': {'enabled': False,
(TaskRunner pid=2255397)                                                'num_shards': 4},
(TaskRunner pid=2255397)                                  'tokenizer_path': None,
(TaskRunner pid=2255397)                                  'trust_remote_code': True,
(TaskRunner pid=2255397)                                  'use_fused_kernels': False,
(TaskRunner pid=2255397)                                  'use_liger': False,
(TaskRunner pid=2255397)                                  'use_remove_padding': True,
(TaskRunner pid=2255397)                                  'use_shm': False},
(TaskRunner pid=2255397)                        'nccl_timeout': 600,
(TaskRunner pid=2255397)                        'ref': {'_target_': 'verl.workers.config.FSDPActorConfig',
(TaskRunner pid=2255397)                                'entropy_checkpointing': False,
(TaskRunner pid=2255397)                                'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2255397)                                'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2255397)                                                'dtype': 'bfloat16',
(TaskRunner pid=2255397)                                                'entropy_checkpointing': False,
(TaskRunner pid=2255397)                                                'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2255397)                                                'forward_only': True,
(TaskRunner pid=2255397)                                                'forward_prefetch': False,
(TaskRunner pid=2255397)                                                'fsdp_size': -1,
(TaskRunner pid=2255397)                                                'full_determinism': False,
(TaskRunner pid=2255397)                                                'model_dtype': 'fp32',
(TaskRunner pid=2255397)                                                'offload_policy': False,
(TaskRunner pid=2255397)                                                'optimizer_offload': False,
(TaskRunner pid=2255397)                                                'param_offload': False,
(TaskRunner pid=2255397)                                                'reshard_after_forward': True,
(TaskRunner pid=2255397)                                                'seed': 42,
(TaskRunner pid=2255397)                                                'strategy': 'fsdp',
(TaskRunner pid=2255397)                                                'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2255397)                                                'use_orig_params': False,
(TaskRunner pid=2255397)                                                'use_torch_compile': True,
(TaskRunner pid=2255397)                                                'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2255397)                                'log_prob_max_token_len_per_gpu': 10240,
(TaskRunner pid=2255397)                                'log_prob_micro_batch_size': None,
(TaskRunner pid=2255397)                                'log_prob_micro_batch_size_per_gpu': 1,
(TaskRunner pid=2255397)                                'log_prob_use_dynamic_bsz': False,
(TaskRunner pid=2255397)                                'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2255397)                                             'all_ranks': False,
(TaskRunner pid=2255397)                                             'enable': False,
(TaskRunner pid=2255397)                                             'ranks': [],
(TaskRunner pid=2255397)                                             'save_path': 'outputs/profile',
(TaskRunner pid=2255397)                                             'tool': None,
(TaskRunner pid=2255397)                                             'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2255397)                                                                     'analysis': True,
(TaskRunner pid=2255397)                                                                     'contents': [],
(TaskRunner pid=2255397)                                                                     'discrete': False,
(TaskRunner pid=2255397)                                                                     'level': 'level0'},
(TaskRunner pid=2255397)                                                             'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2255397)                                                                      'discrete': False},
(TaskRunner pid=2255397)                                                             'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2255397)                                                                       'step_end': None,
(TaskRunner pid=2255397)                                                                       'step_start': 0},
(TaskRunner pid=2255397)                                                             'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2255397)                                                                              'stack_depth': 32,
(TaskRunner pid=2255397)                                                                              'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2255397)                                'rollout_n': 8,
(TaskRunner pid=2255397)                                'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig',
(TaskRunner pid=2255397)                                                  'mode': 'disabled',
(TaskRunner pid=2255397)                                                  'record_file': None,
(TaskRunner pid=2255397)                                                  'replay_file': None},
(TaskRunner pid=2255397)                                'strategy': 'fsdp',
(TaskRunner pid=2255397)                                'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2255397)                                'use_torch_compile': True},
(TaskRunner pid=2255397)                        'rollout': {'_target_': 'verl.workers.config.RolloutConfig',
(TaskRunner pid=2255397)                                    'agent': {'_target_': 'verl.workers.config.AgentLoopConfig',
(TaskRunner pid=2255397)                                              'agent_loop_config_path': None,
(TaskRunner pid=2255397)                                              'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig',
(TaskRunner pid=2255397)                                                                      'name': None,
(TaskRunner pid=2255397)                                                                      'path': None},
(TaskRunner pid=2255397)                                              'default_agent_loop': 'single_turn_agent',
(TaskRunner pid=2255397)                                              'num_workers': 8},
(TaskRunner pid=2255397)                                    'calculate_log_probs': True,
(TaskRunner pid=2255397)                                    'cudagraph_capture_sizes': None,
(TaskRunner pid=2255397)                                    'data_parallel_size': 1,
(TaskRunner pid=2255397)                                    'disable_log_stats': True,
(TaskRunner pid=2255397)                                    'do_sample': True,
(TaskRunner pid=2255397)                                    'dtype': 'bfloat16',
(TaskRunner pid=2255397)                                    'enable_chunked_prefill': True,
(TaskRunner pid=2255397)                                    'enable_prefix_caching': True,
(TaskRunner pid=2255397)                                    'enable_rollout_routing_replay': False,
(TaskRunner pid=2255397)                                    'enforce_eager': False,
(TaskRunner pid=2255397)                                    'engine_kwargs': {'sglang': {}, 'vllm': {}},
(TaskRunner pid=2255397)                                    'expert_parallel_size': 1,
(TaskRunner pid=2255397)                                    'free_cache_engine': True,
(TaskRunner pid=2255397)                                    'gpu_memory_utilization': 0.8,
(TaskRunner pid=2255397)                                    'ignore_eos': False,
(TaskRunner pid=2255397)                                    'layered_summon': False,
(TaskRunner pid=2255397)                                    'load_format': 'dummy',
(TaskRunner pid=2255397)                                    'log_prob_max_token_len_per_gpu': 10240,
(TaskRunner pid=2255397)                                    'log_prob_micro_batch_size': None,
(TaskRunner pid=2255397)                                    'log_prob_micro_batch_size_per_gpu': 1,
(TaskRunner pid=2255397)                                    'log_prob_use_dynamic_bsz': False,
(TaskRunner pid=2255397)                                    'logprobs_mode': 'processed_logprobs',
(TaskRunner pid=2255397)                                    'max_model_len': 10240,
(TaskRunner pid=2255397)                                    'max_num_batched_tokens': 10240,
(TaskRunner pid=2255397)                                    'max_num_seqs': 1024,
(TaskRunner pid=2255397)                                    'mode': 'async',
(TaskRunner pid=2255397)                                    'multi_stage_wake_up': False,
(TaskRunner pid=2255397)                                    'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig',
(TaskRunner pid=2255397)                                                   'enable': False,
(TaskRunner pid=2255397)                                                   'format': 'hermes',
(TaskRunner pid=2255397)                                                   'interaction_config_path': None,
(TaskRunner pid=2255397)                                                   'max_assistant_turns': None,
(TaskRunner pid=2255397)                                                   'max_parallel_calls': 1,
(TaskRunner pid=2255397)                                                   'max_tool_response_length': 256,
(TaskRunner pid=2255397)                                                   'max_user_turns': None,
(TaskRunner pid=2255397)                                                   'num_repeat_rollouts': None,
(TaskRunner pid=2255397)                                                   'tokenization_sanity_check_mode': 'strict',
(TaskRunner pid=2255397)                                                   'tool_config_path': None,
(TaskRunner pid=2255397)                                                   'tool_response_truncate_side': 'middle',
(TaskRunner pid=2255397)                                                   'use_inference_chat_template': False},
(TaskRunner pid=2255397)                                    'n': 8,
(TaskRunner pid=2255397)                                    'name': 'vllm',
(TaskRunner pid=2255397)                                    'over_sample_rate': 0,
(TaskRunner pid=2255397)                                    'pipeline_model_parallel_size': 1,
(TaskRunner pid=2255397)                                    'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2255397)                                                 'all_ranks': False,
(TaskRunner pid=2255397)                                                 'enable': False,
(TaskRunner pid=2255397)                                                 'ranks': [],
(TaskRunner pid=2255397)                                                 'save_path': 'outputs/profile',
(TaskRunner pid=2255397)                                                 'tool': None,
(TaskRunner pid=2255397)                                                 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2255397)                                                                         'analysis': True,
(TaskRunner pid=2255397)                                                                         'contents': [],
(TaskRunner pid=2255397)                                                                         'discrete': False,
(TaskRunner pid=2255397)                                                                         'level': 'level0'},
(TaskRunner pid=2255397)                                                                 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2255397)                                                                          'discrete': False},
(TaskRunner pid=2255397)                                                                 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2255397)                                                                           'step_end': None,
(TaskRunner pid=2255397)                                                                           'step_start': 0},
(TaskRunner pid=2255397)                                                                 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2255397)                                                                                  'stack_depth': 32,
(TaskRunner pid=2255397)                                                                                  'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2255397)                                    'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig',
(TaskRunner pid=2255397)                                                   'enable': False,
(TaskRunner pid=2255397)                                                   'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml',
(TaskRunner pid=2255397)                                                   'port': 9090,
(TaskRunner pid=2255397)                                                   'served_model_name': 'Qwen/Qwen3-8B'},
(TaskRunner pid=2255397)                                    'prompt_length': 2048,
(TaskRunner pid=2255397)                                    'quantization': None,
(TaskRunner pid=2255397)                                    'quantization_config_file': None,
(TaskRunner pid=2255397)                                    'response_length': 8192,
(TaskRunner pid=2255397)                                    'scheduling_policy': 'fcfs',
(TaskRunner pid=2255397)                                    'skip_dump_dir': '/tmp/rollout_dump',
(TaskRunner pid=2255397)                                    'skip_rollout': False,
(TaskRunner pid=2255397)                                    'skip_tokenizer_init': True,
(TaskRunner pid=2255397)                                    'temperature': 1.0,
(TaskRunner pid=2255397)                                    'tensor_model_parallel_size': 2,
(TaskRunner pid=2255397)                                    'top_k': -1,
(TaskRunner pid=2255397)                                    'top_p': 1.0,
(TaskRunner pid=2255397)                                    'trace': {'_target_': 'verl.workers.config.TraceConfig',
(TaskRunner pid=2255397)                                              'backend': None,
(TaskRunner pid=2255397)                                              'max_samples_per_step_per_worker': None,
(TaskRunner pid=2255397)                                              'token2text': False},
(TaskRunner pid=2255397)                                    'update_weights_bucket_megabytes': 512,
(TaskRunner pid=2255397)                                    'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig',
(TaskRunner pid=2255397)                                                   'do_sample': True,
(TaskRunner pid=2255397)                                                   'n': 16,
(TaskRunner pid=2255397)                                                   'temperature': 0.6,
(TaskRunner pid=2255397)                                                   'top_k': -1,
(TaskRunner pid=2255397)                                                   'top_p': 0.95}}},
(TaskRunner pid=2255397)  'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig',
(TaskRunner pid=2255397)                'adv_estimator': 'grpo',
(TaskRunner pid=2255397)                'gamma': 1.0,
(TaskRunner pid=2255397)                'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig',
(TaskRunner pid=2255397)                            'horizon': 10000,
(TaskRunner pid=2255397)                            'kl_coef': 0.001,
(TaskRunner pid=2255397)                            'target_kl': 0.1,
(TaskRunner pid=2255397)                            'type': 'fixed'},
(TaskRunner pid=2255397)                'kl_penalty': 'kl',
(TaskRunner pid=2255397)                'lam': 1.0,
(TaskRunner pid=2255397)                'norm_adv_by_std_in_grpo': False,
(TaskRunner pid=2255397)                'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0},
(TaskRunner pid=2255397)                'rollout_correction': {'bypass_mode': False,
(TaskRunner pid=2255397)                                       'loss_type': 'ppo_clip',
(TaskRunner pid=2255397)                                       'rollout_is': 'token',
(TaskRunner pid=2255397)                                       'rollout_is_batch_normalize': False,
(TaskRunner pid=2255397)                                       'rollout_is_threshold': 2.0,
(TaskRunner pid=2255397)                                       'rollout_rs': None,
(TaskRunner pid=2255397)                                       'rollout_rs_threshold': None},
(TaskRunner pid=2255397)                'use_kl_in_reward': False,
(TaskRunner pid=2255397)                'use_pf_ppo': False},
(TaskRunner pid=2255397)  'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig',
(TaskRunner pid=2255397)             'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig',
(TaskRunner pid=2255397)                            'async_save': False,
(TaskRunner pid=2255397)                            'load_contents': ['model', 'optimizer', 'extra'],
(TaskRunner pid=2255397)                            'save_contents': ['model', 'optimizer', 'extra']},
(TaskRunner pid=2255397)             'cliprange_value': 0.5,
(TaskRunner pid=2255397)             'data_loader_seed': 42,
(TaskRunner pid=2255397)             'enable': None,
(TaskRunner pid=2255397)             'forward_max_token_len_per_gpu': 32768,
(TaskRunner pid=2255397)             'forward_micro_batch_size': None,
(TaskRunner pid=2255397)             'forward_micro_batch_size_per_gpu': None,
(TaskRunner pid=2255397)             'grad_clip': 1.0,
(TaskRunner pid=2255397)             'loss_agg_mode': 'token-mean',
(TaskRunner pid=2255397)             'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg',
(TaskRunner pid=2255397)                       'enable_activation_offload': False,
(TaskRunner pid=2255397)                       'enable_gradient_checkpointing': True,
(TaskRunner pid=2255397)                       'external_lib': None,
(TaskRunner pid=2255397)                       'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2255397)                                       'dtype': 'bfloat16',
(TaskRunner pid=2255397)                                       'entropy_checkpointing': False,
(TaskRunner pid=2255397)                                       'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2255397)                                       'forward_only': False,
(TaskRunner pid=2255397)                                       'forward_prefetch': False,
(TaskRunner pid=2255397)                                       'fsdp_size': -1,
(TaskRunner pid=2255397)                                       'full_determinism': False,
(TaskRunner pid=2255397)                                       'model_dtype': 'fp32',
(TaskRunner pid=2255397)                                       'offload_policy': False,
(TaskRunner pid=2255397)                                       'optimizer_offload': False,
(TaskRunner pid=2255397)                                       'param_offload': False,
(TaskRunner pid=2255397)                                       'reshard_after_forward': True,
(TaskRunner pid=2255397)                                       'seed': 42,
(TaskRunner pid=2255397)                                       'strategy': 'fsdp',
(TaskRunner pid=2255397)                                       'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2255397)                                       'use_orig_params': False,
(TaskRunner pid=2255397)                                       'use_torch_compile': True,
(TaskRunner pid=2255397)                                       'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2255397)                       'lora_alpha': 16,
(TaskRunner pid=2255397)                       'lora_rank': 0,
(TaskRunner pid=2255397)                       'override_config': {},
(TaskRunner pid=2255397)                       'path': 'Qwen/Qwen3-8B',
(TaskRunner pid=2255397)                       'target_modules': 'all-linear',
(TaskRunner pid=2255397)                       'tiled_mlp': {'enabled': False, 'num_shards': 4},
(TaskRunner pid=2255397)                       'tokenizer_path': 'Qwen/Qwen3-8B',
(TaskRunner pid=2255397)                       'trust_remote_code': True,
(TaskRunner pid=2255397)                       'use_remove_padding': False,
(TaskRunner pid=2255397)                       'use_shm': False},
(TaskRunner pid=2255397)             'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig',
(TaskRunner pid=2255397)                       'betas': [0.9, 0.999],
(TaskRunner pid=2255397)                       'clip_grad': 1.0,
(TaskRunner pid=2255397)                       'lr': 1e-05,
(TaskRunner pid=2255397)                       'lr_scheduler_type': 'constant',
(TaskRunner pid=2255397)                       'lr_warmup_steps': -1,
(TaskRunner pid=2255397)                       'lr_warmup_steps_ratio': 0.0,
(TaskRunner pid=2255397)                       'min_lr_ratio': 0.0,
(TaskRunner pid=2255397)                       'num_cycles': 0.5,
(TaskRunner pid=2255397)                       'optimizer': 'AdamW',
(TaskRunner pid=2255397)                       'optimizer_impl': 'torch.optim',
(TaskRunner pid=2255397)                       'override_optimizer_config': None,
(TaskRunner pid=2255397)                       'total_training_steps': -1,
(TaskRunner pid=2255397)                       'warmup_style': None,
(TaskRunner pid=2255397)                       'weight_decay': 0.01},
(TaskRunner pid=2255397)             'ppo_epochs': 1,
(TaskRunner pid=2255397)             'ppo_max_token_len_per_gpu': 32768,
(TaskRunner pid=2255397)             'ppo_micro_batch_size': None,
(TaskRunner pid=2255397)             'ppo_micro_batch_size_per_gpu': None,
(TaskRunner pid=2255397)             'ppo_mini_batch_size': 8,
(TaskRunner pid=2255397)             'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2255397)                          'all_ranks': False,
(TaskRunner pid=2255397)                          'enable': False,
(TaskRunner pid=2255397)                          'ranks': [],
(TaskRunner pid=2255397)                          'save_path': 'outputs/profile',
(TaskRunner pid=2255397)                          'tool': None,
(TaskRunner pid=2255397)                          'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2255397)                                                  'analysis': True,
(TaskRunner pid=2255397)                                                  'contents': [],
(TaskRunner pid=2255397)                                                  'discrete': False,
(TaskRunner pid=2255397)                                                  'level': 'level0'},
(TaskRunner pid=2255397)                                          'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2255397)                                                   'discrete': False},
(TaskRunner pid=2255397)                                          'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2255397)                                                    'step_end': None,
(TaskRunner pid=2255397)                                                    'step_start': 0},
(TaskRunner pid=2255397)                                          'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2255397)                                                           'stack_depth': 32,
(TaskRunner pid=2255397)                                                           'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2255397)             'rollout_n': 8,
(TaskRunner pid=2255397)             'shuffle': False,
(TaskRunner pid=2255397)             'strategy': 'fsdp',
(TaskRunner pid=2255397)             'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2255397)             'use_dynamic_bsz': False},
(TaskRunner pid=2255397)  'custom_reward_function': {'name': 'compute_score',
(TaskRunner pid=2255397)                             'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'},
(TaskRunner pid=2255397)  'data': {'apply_chat_template_kwargs': {'enable_thinking': False},
(TaskRunner pid=2255397)           'custom_cls': {'name': None, 'path': None},
(TaskRunner pid=2255397)           'datagen': {'name': None, 'path': None},
(TaskRunner pid=2255397)           'dataloader_num_workers': 8,
(TaskRunner pid=2255397)           'filter_overlong_prompts': True,
(TaskRunner pid=2255397)           'filter_overlong_prompts_workers': 1,
(TaskRunner pid=2255397)           'image_key': 'images',
(TaskRunner pid=2255397)           'image_patch_size': 14,
(TaskRunner pid=2255397)           'max_prompt_length': 2048,
(TaskRunner pid=2255397)           'max_response_length': 8192,
(TaskRunner pid=2255397)           'prompt_key': 'prompt',
(TaskRunner pid=2255397)           'return_full_prompt': False,
(TaskRunner pid=2255397)           'return_multi_modal_inputs': True,
(TaskRunner pid=2255397)           'return_raw_chat': True,
(TaskRunner pid=2255397)           'return_raw_input_ids': False,
(TaskRunner pid=2255397)           'reward_fn_key': 'data_source',
(TaskRunner pid=2255397)           'sampler': {'class_name': None, 'class_path': None},
(TaskRunner pid=2255397)           'seed': None,
(TaskRunner pid=2255397)           'shuffle': True,
(TaskRunner pid=2255397)           'tokenizer': None,
(TaskRunner pid=2255397)           'tool_config_path': None,
(TaskRunner pid=2255397)           'train_batch_size': 32,
(TaskRunner pid=2255397)           'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet'],
(TaskRunner pid=2255397)           'train_max_samples': 3200,
(TaskRunner pid=2255397)           'truncation': 'error',
(TaskRunner pid=2255397)           'trust_remote_code': True,
(TaskRunner pid=2255397)           'use_shm': False,
(TaskRunner pid=2255397)           'val_batch_size': None,
(TaskRunner pid=2255397)           'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet'],
(TaskRunner pid=2255397)           'val_max_samples': -1,
(TaskRunner pid=2255397)           'validation_shuffle': False,
(TaskRunner pid=2255397)           'video_key': 'videos'},
(TaskRunner pid=2255397)  'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2255397)                      'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2255397)                                                      'controller_nsight_options': {'cuda-graph-trace': 'graph',
(TaskRunner pid=2255397)                                                                                    'cuda-memory-usage': 'true',
(TaskRunner pid=2255397)                                                                                    'trace': 'cuda,nvtx,cublas,ucx'},
(TaskRunner pid=2255397)                                                      'discrete': False,
(TaskRunner pid=2255397)                                                      'worker_nsight_options': {'capture-range': 'cudaProfilerApi',
(TaskRunner pid=2255397)                                                                                'capture-range-end': None,
(TaskRunner pid=2255397)                                                                                'cuda-graph-trace': 'graph',
(TaskRunner pid=2255397)                                                                                'cuda-memory-usage': 'true',
(TaskRunner pid=2255397)                                                                                'kill': 'none',
(TaskRunner pid=2255397)                                                                                'trace': 'cuda,nvtx,cublas,ucx'}},
(TaskRunner pid=2255397)                                             'torch_memory': {'context': 'all',
(TaskRunner pid=2255397)                                                              'kw_args': {},
(TaskRunner pid=2255397)                                                              'stack_depth': 32,
(TaskRunner pid=2255397)                                                              'stacks': 'all',
(TaskRunner pid=2255397)                                                              'trace_alloc_max_entries': 100000}},
(TaskRunner pid=2255397)                      'profile_continuous_steps': False,
(TaskRunner pid=2255397)                      'save_path': 'outputs/profile',
(TaskRunner pid=2255397)                      'steps': None,
(TaskRunner pid=2255397)                      'tool': None},
(TaskRunner pid=2255397)  'max_model_len': 10240,
(TaskRunner pid=2255397)  'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_8B',
(TaskRunner pid=2255397)                              'include_dashboard': False,
(TaskRunner pid=2255397)                              'num_cpus': None,
(TaskRunner pid=2255397)                              'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8',
(TaskRunner pid=2255397)                                                           'TASK': 'datasets/sciknoweval/chemistry',
(TaskRunner pid=2255397)                                                           'USER': 'root'}}},
(TaskRunner pid=2255397)                 'timeline_json_file': None},
(TaskRunner pid=2255397)  'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig',
(TaskRunner pid=2255397)                     'module': {'_target_': 'verl.trainer.config.config.ModuleConfig',
(TaskRunner pid=2255397)                                'name': 'custom_reward_manager',
(TaskRunner pid=2255397)                                'path': None},
(TaskRunner pid=2255397)                     'name': 'naive',
(TaskRunner pid=2255397)                     'source': 'register'},
(TaskRunner pid=2255397)  'reward_model': {'enable': False,
(TaskRunner pid=2255397)                   'enable_resource_pool': False,
(TaskRunner pid=2255397)                   'forward_max_token_len_per_gpu': 32768,
(TaskRunner pid=2255397)                   'launch_reward_fn_async': False,
(TaskRunner pid=2255397)                   'max_length': None,
(TaskRunner pid=2255397)                   'micro_batch_size': None,
(TaskRunner pid=2255397)                   'micro_batch_size_per_gpu': None,
(TaskRunner pid=2255397)                   'model': {'external_lib': None,
(TaskRunner pid=2255397)                             'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2255397)                                             'forward_prefetch': False,
(TaskRunner pid=2255397)                                             'fsdp_size': -1,
(TaskRunner pid=2255397)                                             'param_offload': False,
(TaskRunner pid=2255397)                                             'reshard_after_forward': True,
(TaskRunner pid=2255397)                                             'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2255397)                             'input_tokenizer': 'Qwen/Qwen3-8B',
(TaskRunner pid=2255397)                             'override_config': {},
(TaskRunner pid=2255397)                             'path': '~/models/FsfairX-LLaMA3-RM-v0.1',
(TaskRunner pid=2255397)                             'trust_remote_code': False,
(TaskRunner pid=2255397)                             'use_fused_kernels': False,
(TaskRunner pid=2255397)                             'use_remove_padding': False,
(TaskRunner pid=2255397)                             'use_shm': False},
(TaskRunner pid=2255397)                   'n_gpus_per_node': 8,
(TaskRunner pid=2255397)                   'nnodes': 0,
(TaskRunner pid=2255397)                   'num_workers': 1,
(TaskRunner pid=2255397)                   'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2255397)                                'all_ranks': False,
(TaskRunner pid=2255397)                                'enable': False,
(TaskRunner pid=2255397)                                'ranks': [],
(TaskRunner pid=2255397)                                'save_path': 'outputs/profile',
(TaskRunner pid=2255397)                                'tool': None,
(TaskRunner pid=2255397)                                'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2255397)                                                        'analysis': True,
(TaskRunner pid=2255397)                                                        'contents': [],
(TaskRunner pid=2255397)                                                        'discrete': False,
(TaskRunner pid=2255397)                                                        'level': 'level0'},
(TaskRunner pid=2255397)                                                'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2255397)                                                         'discrete': False},
(TaskRunner pid=2255397)                                                'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2255397)                                                          'step_end': None,
(TaskRunner pid=2255397)                                                          'step_start': 0},
(TaskRunner pid=2255397)                                                'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2255397)                                                                 'stack_depth': 32,
(TaskRunner pid=2255397)                                                                 'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2255397)                   'reward_loop_class_name': None,
(TaskRunner pid=2255397)                   'reward_loop_module_path': None,
(TaskRunner pid=2255397)                   'reward_loop_source': 'register',
(TaskRunner pid=2255397)                   'reward_manager': 'naive',
(TaskRunner pid=2255397)                   'rollout': {'_target_': 'verl.workers.config.RolloutConfig',
(TaskRunner pid=2255397)                               'cudagraph_capture_sizes': None,
(TaskRunner pid=2255397)                               'data_parallel_size': 1,
(TaskRunner pid=2255397)                               'disable_log_stats': True,
(TaskRunner pid=2255397)                               'dtype': 'bfloat16',
(TaskRunner pid=2255397)                               'enable_chunked_prefill': True,
(TaskRunner pid=2255397)                               'enable_prefix_caching': True,
(TaskRunner pid=2255397)                               'enforce_eager': True,
(TaskRunner pid=2255397)                               'engine_kwargs': {},
(TaskRunner pid=2255397)                               'expert_parallel_size': 1,
(TaskRunner pid=2255397)                               'free_cache_engine': True,
(TaskRunner pid=2255397)                               'gpu_memory_utilization': 0.5,
(TaskRunner pid=2255397)                               'limit_images': None,
(TaskRunner pid=2255397)                               'load_format': 'auto',
(TaskRunner pid=2255397)                               'max_model_len': None,
(TaskRunner pid=2255397)                               'max_num_batched_tokens': 8192,
(TaskRunner pid=2255397)                               'max_num_seqs': 1024,
(TaskRunner pid=2255397)                               'name': '???',
(TaskRunner pid=2255397)                               'prompt_length': 2048,
(TaskRunner pid=2255397)                               'response_length': 2048,
(TaskRunner pid=2255397)                               'skip_tokenizer_init': False,
(TaskRunner pid=2255397)                               'tensor_model_parallel_size': 2},
(TaskRunner pid=2255397)                   'sandbox_fusion': {'max_concurrent': 64,
(TaskRunner pid=2255397)                                      'memory_limit_mb': 1024,
(TaskRunner pid=2255397)                                      'url': None},
(TaskRunner pid=2255397)                   'strategy': 'fsdp',
(TaskRunner pid=2255397)                   'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2255397)                   'use_dynamic_bsz': False,
(TaskRunner pid=2255397)                   'use_reward_loop': False},
(TaskRunner pid=2255397)  'trainer': {'balance_batch': True,
(TaskRunner pid=2255397)              'critic_warmup': 0,
(TaskRunner pid=2255397)              'default_hdfs_dir': None,
(TaskRunner pid=2255397)              'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8',
(TaskRunner pid=2255397)              'del_local_ckpt_after_load': False,
(TaskRunner pid=2255397)              'device': 'cuda',
(TaskRunner pid=2255397)              'esi_redundant_time': 0,
(TaskRunner pid=2255397)              'experiment_name': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8',
(TaskRunner pid=2255397)              'group_name': 'QWEN3-RLSD-TR-GRPO-matched-generalization',
(TaskRunner pid=2255397)              'log_val_generations': 0,
(TaskRunner pid=2255397)              'logger': ['console', 'wandb'],
(TaskRunner pid=2255397)              'max_actor_ckpt_to_keep': 100,
(TaskRunner pid=2255397)              'max_critic_ckpt_to_keep': None,
(TaskRunner pid=2255397)              'n_gpus_per_node': 8,
(TaskRunner pid=2255397)              'nnodes': 1,
(TaskRunner pid=2255397)              'project_name': 'SDPO-root',
(TaskRunner pid=2255397)              'ray_wait_register_center_timeout': 300,
(TaskRunner pid=2255397)              'resume_from_path': None,
(TaskRunner pid=2255397)              'resume_mode': 'auto',
(TaskRunner pid=2255397)              'rollout_data_dir': None,
(TaskRunner pid=2255397)              'save_freq': 10,
(TaskRunner pid=2255397)              'test_freq': 10,
(TaskRunner pid=2255397)              'total_epochs': 30,
(TaskRunner pid=2255397)              'total_training_steps': 100,
(TaskRunner pid=2255397)              'use_legacy_worker_impl': 'auto',
(TaskRunner pid=2255397)              'val_before_train': False,
(TaskRunner pid=2255397)              'val_only': False,
(TaskRunner pid=2255397)              'validation_data_dir': None},
(TaskRunner pid=2255397)  'transfer_queue': {'enable': False},
(TaskRunner pid=2255397)  'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/chemistry',
(TaskRunner pid=2255397)           'dir': '/users/root/SDPO',
(TaskRunner pid=2255397)           'log_dir': '/users/root/output',
(TaskRunner pid=2255397)           'task': 'datasets/sciknoweval/chemistry'}}
(TaskRunner pid=2255397) [validate_config] All configuration checks passed successfully!
(TaskRunner pid=2255397) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True
(TaskRunner pid=2255397)   use_critic=need_critic(config),
(TaskRunner pid=2255397) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now.
(TaskRunner pid=2255397)   warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1)
(TaskRunner pid=2255397) Using dataset class: RLHFDataset
(TaskRunner pid=2255397) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(TaskRunner pid=2255397)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(TaskRunner pid=2255397) dataset len: 1890
(TaskRunner pid=2255397) Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2255397) WARNING:2026-07-03 01:41:28,094:Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2255397) 
Filtering prompts longer than 2048 tokens (num_proc=1):   0%|          | 0/1890 [00:00<?, ? examples/s]
(TaskRunner pid=2255397) 
Filtering prompts longer than 2048 tokens (num_proc=1):  53%|█████▎    | 1000/1890 [00:00<00:00, 1094.76 examples/s]
(TaskRunner pid=2255397) 
Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 1890/1890 [00:01<00:00, 1587.58 examples/s]
(TaskRunner pid=2255397) filter dataset len: 1890
(TaskRunner pid=2255397) Using dataset class: RLHFDataset
(TaskRunner pid=2255397) 
Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 1890/1890 [00:01<00:00, 1380.34 examples/s]
(TaskRunner pid=2255397) dataset len: 210
(TaskRunner pid=2255397) Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2255397) WARNING:2026-07-03 01:41:29,940:Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2255397) 
Filtering prompts longer than 2048 tokens (num_proc=1):   0%|          | 0/210 [00:00<?, ? examples/s]
(TaskRunner pid=2255397) 
Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 210/210 [00:00<00:00, 316.19 examples/s]
(TaskRunner pid=2255397) filter dataset len: 210
(TaskRunner pid=2255397) Size of train dataloader: 59, Size of val dataloader: 1
(TaskRunner pid=2255397) Total training steps: 100
(TaskRunner pid=2255397) colocated worker base class <class 'verl.single_controller.base.worker.Worker'>
(TaskRunner pid=2255397) bind role actor_rollout_ref method chat_completion to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2255397) bind role actor_rollout_ref method generate to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2255397) bind role actor_rollout_ref method get_zeromq_address to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2255397) bind role actor_rollout_ref method sleep to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2255397) bind role actor_rollout_ref method wake_up to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2255397) 
Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 210/210 [00:00<00:00, 276.01 examples/s]
(TaskRunner pid=2255397) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True
(TaskRunner pid=2255397)   self.use_critic = need_critic(self.config)
(WorkerDict pid=2255793) [W703 01:41:37.908542315 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:41933 (errno: 97 - Address family not supported by protocol).
(WorkerDict pid=2255788) [Gloo] Rank 1 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7
(WorkerDict pid=2255789) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(WorkerDict pid=2255789)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(WorkerDict pid=2255792) [W703 01:41:38.533011673 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:41933 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.)
(WorkerDict pid=2255787) Model config after override: Qwen3Config {
(WorkerDict pid=2255787)   "architectures": [
(WorkerDict pid=2255787)     "Qwen3ForCausalLM"
(WorkerDict pid=2255787)   ],
(WorkerDict pid=2255787)   "attention_bias": false,
(WorkerDict pid=2255787)   "attention_dropout": 0.0,
(WorkerDict pid=2255787)   "dtype": "bfloat16",
(WorkerDict pid=2255787)   "eos_token_id": 151645,
(WorkerDict pid=2255787)   "head_dim": 128,
(WorkerDict pid=2255787)   "hidden_act": "silu",
(WorkerDict pid=2255787)   "hidden_size": 4096,
(WorkerDict pid=2255787)   "initializer_range": 0.02,
(WorkerDict pid=2255787)   "intermediate_size": 12288,
(WorkerDict pid=2255787)   "layer_types": [
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention"
(WorkerDict pid=2255787)   ],
(WorkerDict pid=2255787)   "max_position_embeddings": 40960,
(WorkerDict pid=2255787)   "max_window_layers": 36,
(WorkerDict pid=2255787)   "model_type": "qwen3",
(WorkerDict pid=2255787)   "num_attention_heads": 32,
(WorkerDict pid=2255787)   "num_hidden_layers": 36,
(WorkerDict pid=2255787)   "num_key_value_heads": 8,
(WorkerDict pid=2255787)   "pad_token_id": 151643,
(WorkerDict pid=2255787)   "rms_norm_eps": 1e-06,
(WorkerDict pid=2255787)   "rope_scaling": null,
(WorkerDict pid=2255787)   "rope_theta": 1000000,
(WorkerDict pid=2255787)   "sliding_window": null,
(WorkerDict pid=2255787)   "tie_word_embeddings": false,
(WorkerDict pid=2255787)   "transformers_version": "4.57.1",
(WorkerDict pid=2255787)   "use_cache": true,
(WorkerDict pid=2255787)   "use_sliding_window": false,
(WorkerDict pid=2255787)   "vocab_size": 151936
(WorkerDict pid=2255787) }
(WorkerDict pid=2255787) 
(WorkerDict pid=2255794) `torch_dtype` is deprecated! Use `dtype` instead!
(WorkerDict pid=2255794) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)`
(WorkerDict pid=2255794) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)`
(WorkerDict pid=2255789) 
Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s]
(WorkerDict pid=2255787) 
Loading checkpoint shards:  20%|██        | 1/5 [00:02<00:11,  2.99s/it]
(WorkerDict pid=2255788) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster]
(WorkerDict pid=2255788)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster]
(WorkerDict pid=2255788) `torch_dtype` is deprecated! Use `dtype` instead! [repeated 7x across cluster]
(WorkerDict pid=2255788) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` [repeated 14x across cluster]
(WorkerDict pid=2255791) 
Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s] [repeated 7x across cluster]
(WorkerDict pid=2255787) 
Loading checkpoint shards:  60%|██████    | 3/5 [00:08<00:05,  2.97s/it] [repeated 16x across cluster]
(WorkerDict pid=2255787) 
Loading checkpoint shards: 100%|██████████| 5/5 [00:12<00:00,  2.08s/it]
Loading checkpoint shards: 100%|██████████| 5/5 [00:12<00:00,  2.44s/it]
(WorkerDict pid=2255787) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention
(WorkerDict pid=2255787) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch
(WorkerDict pid=2255792) [Gloo] Rank 5 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 [repeated 7x across cluster]
(WorkerDict pid=2255787) Qwen3ForCausalLM contains 8.19B parameters
(WorkerDict pid=2255787) wrap_policy: functools.partial(<function _or_policy at 0x76064de9c540>, policies=[functools.partial(<function transformer_auto_wrap_policy at 0x76064de9c400>, transformer_layer_cls={<class 'transformers.models.qwen3.modeling_qwen3.Qwen3DecoderLayer'>})])
(WorkerDict pid=2255787) NCCL version 2.27.5+cuda12.9
(WorkerDict pid=2255788) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now.
(WorkerDict pid=2255788)   warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1)
(WorkerDict pid=2255790) 
Loading checkpoint shards:  80%|████████  | 4/5 [00:12<00:02,  2.90s/it] [repeated 15x across cluster]
(WorkerDict pid=2255790) 
Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00,  2.20s/it]
Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00,  2.60s/it] [repeated 7x across cluster]
(WorkerDict pid=2255787) Total steps: 100, num_warmup_steps: 10
(WorkerDict pid=2255790) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster]
(WorkerDict pid=2255790) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster]
(WorkerDict pid=2255787) Actor use_remove_padding=True
(WorkerDict pid=2255787) Actor use_fused_kernels=False
(WorkerDict pid=2255787) Actor use_prefix_grouper=False
(WorkerDict pid=2255794) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(WorkerDict pid=2255794)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(WorkerDict pid=2255788) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3
(WorkerDict pid=2255787) reference model: Qwen/Qwen3-8B
(WorkerDict pid=2255787) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html .
(WorkerDict pid=2255787)   warnings.warn(
(WorkerDict pid=2255787) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster]
(WorkerDict pid=2255787)   warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster]
(WorkerDict pid=2255787) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 8x across cluster]
(WorkerDict pid=2255787)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 8x across cluster]
(WorkerDict pid=2255787) Model config after override: Qwen3Config {
(WorkerDict pid=2255787)   "architectures": [
(WorkerDict pid=2255787)     "Qwen3ForCausalLM"
(WorkerDict pid=2255787)   ],
(WorkerDict pid=2255787)   "attention_bias": false,
(WorkerDict pid=2255787)   "attention_dropout": 0.0,
(WorkerDict pid=2255787)   "dtype": "bfloat16",
(WorkerDict pid=2255787)   "eos_token_id": 151645,
(WorkerDict pid=2255787)   "head_dim": 128,
(WorkerDict pid=2255787)   "hidden_act": "silu",
(WorkerDict pid=2255787)   "hidden_size": 4096,
(WorkerDict pid=2255787)   "initializer_range": 0.02,
(WorkerDict pid=2255787)   "intermediate_size": 12288,
(WorkerDict pid=2255787)   "layer_types": [
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention",
(WorkerDict pid=2255787)     "full_attention"
(WorkerDict pid=2255787)   ],
(WorkerDict pid=2255787)   "max_position_embeddings": 40960,
(WorkerDict pid=2255787)   "max_window_layers": 36,
(WorkerDict pid=2255787)   "model_type": "qwen3",
(WorkerDict pid=2255787)   "num_attention_heads": 32,
(WorkerDict pid=2255787)   "num_hidden_layers": 36,
(WorkerDict pid=2255787)   "num_key_value_heads": 8,
(WorkerDict pid=2255787)   "pad_token_id": 151643,
(WorkerDict pid=2255787)   "rms_norm_eps": 1e-06,
(WorkerDict pid=2255787)   "rope_scaling": null,
(WorkerDict pid=2255787)   "rope_theta": 1000000,
(WorkerDict pid=2255787)   "sliding_window": null,
(WorkerDict pid=2255787)   "tie_word_embeddings": false,
(WorkerDict pid=2255787)   "transformers_version": "4.57.1",
(WorkerDict pid=2255787)   "use_cache": true,
(WorkerDict pid=2255787)   "use_sliding_window": false,
(WorkerDict pid=2255787)   "vocab_size": 151936
(WorkerDict pid=2255787) }
(WorkerDict pid=2255787) 
(WorkerDict pid=2255792) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster]
(WorkerDict pid=2255787) 
Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s]
(WorkerDict pid=2255787) 
Loading checkpoint shards:  20%|██        | 1/5 [00:02<00:11,  2.96s/it]
(WorkerDict pid=2255791) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster]
(WorkerDict pid=2255791)   warnings.warn( [repeated 7x across cluster]
(WorkerDict pid=2255791) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster]
(WorkerDict pid=2255791)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster]
(WorkerDict pid=2255793) 
Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s] [repeated 7x across cluster]
(WorkerDict pid=2255787) 
Loading checkpoint shards:  60%|██████    | 3/5 [00:08<00:05,  2.92s/it] [repeated 16x across cluster]
(WorkerDict pid=2255787) 
Loading checkpoint shards: 100%|██████████| 5/5 [00:12<00:00,  2.05s/it]
Loading checkpoint shards: 100%|██████████| 5/5 [00:12<00:00,  2.41s/it]
(WorkerDict pid=2255787) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention
(WorkerDict pid=2255787) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch
(WorkerDict pid=2255787) Qwen3ForCausalLM contains 8.19B parameters
(WorkerDict pid=2255787) wrap_policy: functools.partial(<function _or_policy at 0x76064de9c540>, policies=[functools.partial(<function transformer_auto_wrap_policy at 0x76064de9c400>, transformer_layer_cls={<class 'transformers.models.qwen3.modeling_qwen3.Qwen3DecoderLayer'>})])
(WorkerDict pid=2255787) Ref use_remove_padding=True
(WorkerDict pid=2255787) Ref use_fused_kernels=False
(WorkerDict pid=2255787) Ref use_prefix_grouper=False
(WorkerDict pid=2255792) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster]
(WorkerDict pid=2255792) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster]
(TaskRunner pid=2255397) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now.
(TaskRunner pid=2255397)   warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1)
(WorkerDict pid=2255792) 
Loading checkpoint shards:  80%|████████  | 4/5 [00:12<00:02,  2.93s/it] [repeated 15x across cluster]
(WorkerDict pid=2255792) 
Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00,  2.22s/it]
Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00,  2.62s/it] [repeated 7x across cluster]
(vLLMHttpServer pid=2256706) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(vLLMHttpServer pid=2256706)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(vLLMHttpServer pid=2256707) ['serve',
(vLLMHttpServer pid=2256707)  'Qwen/Qwen3-8B',
(vLLMHttpServer pid=2256707)  '--dtype',
(vLLMHttpServer pid=2256707)  'bfloat16',
(vLLMHttpServer pid=2256707)  '--load_format',
(vLLMHttpServer pid=2256707)  'dummy',
(vLLMHttpServer pid=2256707)  '--trust_remote_code',
(vLLMHttpServer pid=2256707)  '--max_model_len',
(vLLMHttpServer pid=2256707)  '40960',
(vLLMHttpServer pid=2256707)  '--max_num_seqs',
(vLLMHttpServer pid=2256707)  '1024',
(vLLMHttpServer pid=2256707)  '--enable_chunked_prefill',
(vLLMHttpServer pid=2256707)  '--max_num_batched_tokens',
(vLLMHttpServer pid=2256707)  '10240',
(vLLMHttpServer pid=2256707)  '--enable_prefix_caching',
(vLLMHttpServer pid=2256707)  '--enable_sleep_mode',
(vLLMHttpServer pid=2256707)  '--logprobs_mode',
(vLLMHttpServer pid=2256707)  'processed_logprobs',
(vLLMHttpServer pid=2256707)  '--disable_custom_all_reduce',
(vLLMHttpServer pid=2256707)  '--gpu_memory_utilization',
(vLLMHttpServer pid=2256707)  '0.8',
(vLLMHttpServer pid=2256707)  '--disable_log_stats',
(vLLMHttpServer pid=2256707)  '--tensor_parallel_size',
(vLLMHttpServer pid=2256707)  '2',
(vLLMHttpServer pid=2256707)  '--seed',
(vLLMHttpServer pid=2256707)  '0',
(vLLMHttpServer pid=2256707)  '--override_generation_config',
(vLLMHttpServer pid=2256707)  '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, '
(vLLMHttpServer pid=2256707)  '"max_new_tokens": 8192}',
(vLLMHttpServer pid=2256707)  '--hf_overrides',
(vLLMHttpServer pid=2256707)  '{}',
(vLLMHttpServer pid=2256707)  '--scheduling_policy',
(vLLMHttpServer pid=2256707)  'fcfs']
(vLLMHttpServer pid=2256707) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead.
(vLLMHttpServer pid=2256706) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(vLLMHttpServer pid=2256706) WARNING 07-03 01:42:55 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned
(WorkerDict pid=2255788) WARNING 07-03 01:43:03 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'.
(vLLMHttpServer pid=2256709) WARNING 07-03 01:42:55 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster]
(WorkerDict pid=2255789) NCCL version 2.27.5+cuda12.9
(WorkerDict pid=2255793) 2026-07-03 01:43:17,091 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ...
(vLLMHttpServer pid=2256709) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 3x across cluster]
(vLLMHttpServer pid=2256709)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 3x across cluster]
(vLLMHttpServer pid=2256709) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster]
(vLLMHttpServer pid=2256709) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster]
(WorkerDict pid=2255793) 2026-07-03 01:43:17,106 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   0%|          | 0/51 [00:00<?, ?it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   4%|▍         | 2/51 [00:00<00:03, 13.96it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   8%|▊         | 4/51 [00:00<00:02, 16.30it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  12%|█▏        | 6/51 [00:00<00:02, 16.43it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  16%|█▌        | 8/51 [00:00<00:02, 16.93it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  20%|█▉        | 10/51 [00:00<00:03, 13.32it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  24%|██▎       | 12/51 [00:00<00:03, 12.54it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  27%|██▋       | 14/51 [00:00<00:02, 13.59it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  31%|███▏      | 16/51 [00:01<00:02, 14.21it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  35%|███▌      | 18/51 [00:01<00:02, 14.62it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  39%|███▉      | 20/51 [00:01<00:02, 15.08it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  43%|████▎     | 22/51 [00:01<00:01, 15.37it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  47%|████▋     | 24/51 [00:01<00:01, 15.62it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  51%|█████     | 26/51 [00:01<00:01, 15.41it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  55%|█████▍    | 28/51 [00:01<00:01, 15.52it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  59%|█████▉    | 30/51 [00:02<00:01, 15.18it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  63%|██████▎   | 32/51 [00:02<00:01, 15.39it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  67%|██████▋   | 34/51 [00:02<00:01, 14.83it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  71%|███████   | 36/51 [00:02<00:01, 14.44it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  75%|███████▍  | 38/51 [00:02<00:00, 14.61it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  78%|███████▊  | 40/51 [00:02<00:00, 14.18it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  82%|████████▏ | 42/51 [00:02<00:00, 14.27it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  86%|████████▋ | 44/51 [00:02<00:00, 14.40it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  90%|█████████ | 46/51 [00:03<00:00, 14.34it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  94%|█████████▍| 48/51 [00:03<00:00, 14.27it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  98%|█████████▊| 50/51 [00:03<00:00, 14.69it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 51/51 [00:03<00:00, 14.68it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (decode, FULL):   0%|          | 0/51 [00:00<?, ?it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (decode, FULL):   6%|▌         | 3/51 [00:00<00:02, 21.25it/s]
(WorkerDict pid=2255787) 2026-07-03 01:43:18,119 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... [repeated 7x across cluster]
(WorkerDict pid=2255787) 2026-07-03 01:43:18,149 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends [repeated 7x across cluster]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (decode, FULL):  12%|█▏        | 6/51 [00:00<00:02, 22.37it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (decode, FULL):  18%|█▊        | 9/51 [00:00<00:01, 22.78it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (decode, FULL):  24%|██▎       | 12/51 [00:00<00:01, 22.91it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (decode, FULL):  29%|██▉       | 15/51 [00:00<00:01, 22.90it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (decode, FULL):  35%|███▌      | 18/51 [00:00<00:01, 22.79it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (decode, FULL):  41%|████      | 21/51 [00:00<00:01, 22.34it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (decode, FULL):  47%|████▋     | 24/51 [00:01<00:01, 22.57it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (decode, FULL):  53%|█████▎    | 27/51 [00:01<00:01, 22.16it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (decode, FULL):  59%|█████▉    | 30/51 [00:01<00:00, 22.46it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (decode, FULL):  65%|██████▍   | 33/51 [00:01<00:00, 22.50it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (decode, FULL):  71%|███████   | 36/51 [00:01<00:00, 22.74it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (decode, FULL):  76%|███████▋  | 39/51 [00:01<00:00, 22.32it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (decode, FULL):  82%|████████▏ | 42/51 [00:01<00:00, 22.25it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (decode, FULL):  88%|████████▊ | 45/51 [00:02<00:00, 22.45it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (decode, FULL):  94%|█████████▍| 48/51 [00:02<00:00, 22.95it/s]
(WorkerDict pid=2255787) 
Capturing CUDA graphs (decode, FULL): 100%|██████████| 51/51 [00:02<00:00, 23.43it/s]
Capturing CUDA graphs (decode, FULL): 100%|██████████| 51/51 [00:02<00:00, 22.69it/s]
(vLLMHttpServer pid=2256708) WARNING 07-03 01:43:25 [api_server.py:1358] LoRA dynamic loading & unloading is enabled in the API server. This should ONLY be used for local development!
(WorkerDict pid=2255792) WARNING 07-03 01:43:04 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. [repeated 7x across cluster]
(WorkerDict pid=2255793) NCCL version 2.27.5+cuda12.9 [repeated 2x across cluster]
(vLLMHttpServer pid=2256708) WARNING 07-03 01:43:25 [model.py:1576] Default sampling parameters have been overridden by the model's Hugging Face generation config recommended from the model creator. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`.
(TaskRunner pid=2255397) AgentLoopManager: ['198.19.44.212:37481', '198.19.44.212:36385', '198.19.44.212:42611', '198.19.44.212:38461']
(TaskRunner pid=2255397) wandb: Currently logged in as: seongryongjung (seongryongjung-chung-ang-university) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
(TaskRunner pid=2255397) wandb: setting up run zs3h1j56
(TaskRunner pid=2255397) wandb: Tracking run with wandb version 0.23.1
(TaskRunner pid=2255397) wandb: Run data is saved locally in /mnt/mole/SDPO/L2T/wandb/run-20260703_014331-zs3h1j56
(TaskRunner pid=2255397) wandb: Run `wandb offline` to turn off syncing.
(TaskRunner pid=2255397) wandb: Syncing run qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8
(TaskRunner pid=2255397) wandb: ⭐️ View project at https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root
(TaskRunner pid=2255397) wandb: 🚀 View run at https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/zs3h1j56
(TaskRunner pid=2255397) Checkpoint tracker file does not exist: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/latest_checkpointed_iteration.txt
(TaskRunner pid=2255397) Training from scratch
(vLLMHttpServer pid=2256707) WARNING 07-03 01:43:26 [api_server.py:1358] LoRA dynamic loading & unloading is enabled in the API server. This should ONLY be used for local development! [repeated 3x across cluster]
(vLLMHttpServer pid=2256707) WARNING 07-03 01:43:26 [model.py:1576] Default sampling parameters have been overridden by the model's Hugging Face generation config recommended from the model creator. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`. [repeated 3x across cluster]
(TaskRunner pid=2255397) wandb: Detected [openai] in use.
(TaskRunner pid=2255397) wandb: Use W&B Weave for improved LLM call tracing. Install Weave with `pip install weave` then add `import weave` to the top of your script.
(TaskRunner pid=2255397) wandb: For more information, check out the docs at: https://weave-docs.wandb.ai/
(TaskRunner pid=2255397) 
Training Progress:   0%|          | 0/100 [00:00<?, ?it/s]
(AgentLoopWorker pid=2257667) Using dataset class: RLHFDataset
(AgentLoopWorker pid=2257667) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(AgentLoopWorker pid=2257667)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(AgentLoopWorker pid=2257667) You're using a Qwen2TokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.
(AgentLoopWorker pid=2257675) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster]
(AgentLoopWorker pid=2257675)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster]
(TaskRunner pid=2255397) step:1 - global_seqlen/min:18655 - global_seqlen/max:26853 - global_seqlen/minmax_diff:8198 - global_seqlen/balanced_min:23692 - global_seqlen/balanced_max:23701 - global_seqlen/mean:23696.5 - actor/entropy:0.5224554538726807 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.14097940921783447 - training/rollout_probs_diff_mean:0.0045183561742305756 - training/rollout_probs_diff_std:0.009310152381658554 - training/rollout_actor_probs_pearson_corr:0.9994592070579529 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.46484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:0.9999948143959045 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.428677908028476e-05 - rollout_corr/rollout_is_max:1.4933885335922241 - rollout_corr/rollout_is_min:0.4801149070262909 - rollout_corr/rollout_is_std:0.03428589925169945 - rollout_corr/rollout_is_eff_sample_size:0.9988258574271096 - rollout_corr/rollout_is_seq_mean:1.0000158548355103 - rollout_corr/rollout_is_seq_std:0.0017025339184328914 - rollout_corr/rollout_is_seq_max:1.0078747272491455 - rollout_corr/rollout_is_seq_min:0.9956079721450806 - rollout_corr/rollout_is_seq_max_deviation:0.007874727249145508 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7518747616559267) - rollout_corr/training_log_ppl:np.float64(0.5485204759170301) - rollout_corr/kl:np.float64(0.0006133353596755597) - rollout_corr/k3_kl:np.float64(0.0006291264688229603) - rollout_corr/rollout_ppl:np.float64(1.750815692357719) - rollout_corr/rollout_log_ppl:np.float64(0.5479071354493499) - rollout_corr/log_ppl_diff:np.float64(0.0006133404676802456) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013540681102313101) - rollout_corr/log_ppl_diff_max:np.float64(0.005063414573669434) - rollout_corr/log_ppl_diff_min:np.float64(-0.006822109222412109) - rollout_corr/ppl_ratio:np.float64(1.0006149462424219) - rollout_corr/chi2_token:np.float64(0.0012849036138504744) - rollout_corr/chi2_seq:np.float64(0.5767202903516591) - actor/pg_loss:np.float64(0.0015394691145047545) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(0.0) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.46484375) - self_distillation/token_reweight_w_mean:np.float64(29057.488782484084) - self_distillation/token_reweight_w_std:np.float64(551978.9261493584) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0031458756420761) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14994207414565608) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.34795472025871277) - perf/mfu/actor:np.float64(0.039808731599246736) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(208.84796524047852) - actor/lr:np.float64(0.0) - training/global_step:1 - training/epoch:0 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004086250439286232 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004086250439286232 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:482.515625 - response_length/max:1194.0 
(AgentLoopWorker pid=2257672) Using dataset class: RLHFDataset [repeated 7x across cluster]
(TaskRunner pid=2255397) 
Training Progress:   1%|          | 1/100 [01:11<1:58:40, 71.93s/it]
(AgentLoopWorker pid=2257675) You're using a Qwen2TokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding. [repeated 7x across cluster]
(TaskRunner pid=2255397) step:2 - global_seqlen/min:18554 - global_seqlen/max:28889 - global_seqlen/minmax_diff:10335 - global_seqlen/balanced_min:23930 - global_seqlen/balanced_max:23950 - global_seqlen/mean:23945.25 - actor/entropy:0.5649150013923645 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21355730295181274 - training/rollout_probs_diff_mean:0.004759751260280609 - training/rollout_probs_diff_std:0.009735357947647572 - training/rollout_actor_probs_pearson_corr:0.9994669556617737 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.609375 - self_distillation/correct_sample_fraction:0.3671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.609375 - rollout_corr/rollout_is_mean:0.999841034412384 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.8218586774310097e-05 - rollout_corr/rollout_is_max:1.8565502166748047 - rollout_corr/rollout_is_min:0.4281544089317322 - rollout_corr/rollout_is_std:0.03569299355149269 - rollout_corr/rollout_is_eff_sample_size:0.9987271556828731 - rollout_corr/rollout_is_seq_mean:0.9997751712799072 - rollout_corr/rollout_is_seq_std:0.002073427429422736 - rollout_corr/rollout_is_seq_max:1.0066989660263062 - rollout_corr/rollout_is_seq_min:0.993266224861145 - rollout_corr/rollout_is_seq_max_deviation:0.0067337751388549805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8422705363482237) - rollout_corr/training_log_ppl:np.float64(0.5950889247469604) - rollout_corr/kl:np.float64(0.000921569124747279) - rollout_corr/k3_kl:np.float64(0.0006883058893549787) - rollout_corr/rollout_ppl:np.float64(1.8405294218100607) - rollout_corr/rollout_log_ppl:np.float64(0.5941673531197011) - rollout_corr/log_ppl_diff:np.float64(0.0009215716272592545) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017208073986694217) - rollout_corr/log_ppl_diff_max:np.float64(0.007348239421844482) - rollout_corr/log_ppl_diff_min:np.float64(-0.005036115646362305) - rollout_corr/ppl_ratio:np.float64(1.0009240037761629) - rollout_corr/chi2_token:np.float64(0.0009099591989070177) - rollout_corr/chi2_seq:np.float64(0.3998524572234601) - actor/pg_loss:np.float64(0.005818751524202526) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002621370244924037) - actor/ppo_kl:np.float64(9.864444990625998e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3671875) - self_distillation/token_reweight_w_mean:np.float64(25264.271957140416) - self_distillation/token_reweight_w_std:np.float64(438644.3446098004) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0013297640252858) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13903450994985178) - self_distillation/empty_target_batch:np.float64(0.390625) - actor/grad_norm:np.float64(0.3431096374988556) - perf/mfu/actor:np.float64(0.0564479877562713) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(204.3056116104126) - actor/lr:np.float64(1e-07) - training/global_step:2 - training/epoch:0 - critic/score/mean:0.3671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009724170668050647 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009724170668050647 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:428.8203125
(TaskRunner pid=2255397) 
Training Progress:   2%|▏         | 2/100 [01:58<1:32:44, 56.78s/it]
(TaskRunner pid=2255397) step:3 - global_seqlen/min:19604 - global_seqlen/max:28466 - global_seqlen/minmax_diff:8862 - global_seqlen/balanced_min:24149 - global_seqlen/balanced_max:24177 - global_seqlen/mean:24164.75 - actor/entropy:0.5242184400558472 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1616789698600769 - training/rollout_probs_diff_mean:0.004656430333852768 - training/rollout_probs_diff_std:0.009660309180617332 - training/rollout_actor_probs_pearson_corr:0.9994632601737976 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.765625 - self_distillation/correct_sample_fraction:0.4140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.765625 - rollout_corr/rollout_is_mean:1.0001205205917358 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.629170224594418e-06 - rollout_corr/rollout_is_max:1.6356302499771118 - rollout_corr/rollout_is_min:0.49262791872024536 - rollout_corr/rollout_is_std:0.0352848656475544 - rollout_corr/rollout_is_eff_sample_size:0.9987568830709286 - rollout_corr/rollout_is_seq_mean:1.000150442123413 - rollout_corr/rollout_is_seq_std:0.0021288390271365643 - rollout_corr/rollout_is_seq_max:1.007064700126648 - rollout_corr/rollout_is_seq_min:0.9945412278175354 - rollout_corr/rollout_is_seq_max_deviation:0.007064700126647949 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7977697649039328) - rollout_corr/training_log_ppl:np.float64(0.5697851715376601) - rollout_corr/kl:np.float64(0.0006146689036183872) - rollout_corr/k3_kl:np.float64(0.0006752699637218029) - rollout_corr/rollout_ppl:np.float64(1.796580535825342) - rollout_corr/rollout_log_ppl:np.float64(0.5691705014905892) - rollout_corr/log_ppl_diff:np.float64(0.0006146700470708311) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015643324586562812) - rollout_corr/log_ppl_diff_max:np.float64(0.006385087966918945) - rollout_corr/log_ppl_diff_min:np.float64(-0.006580471992492676) - rollout_corr/ppl_ratio:np.float64(1.0006169071421027) - rollout_corr/chi2_token:np.float64(0.0014803188387304544) - rollout_corr/chi2_seq:np.float64(0.8378349014092237) - actor/pg_loss:np.float64(0.008528164820745587) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00031198100373330817) - actor/ppo_kl:np.float64(8.786334534249818e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.765625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4140625) - self_distillation/token_reweight_w_mean:np.float64(19923.08156236657) - self_distillation/token_reweight_w_std:np.float64(356201.15468769113) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001924671465531) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1814161114161834) - self_distillation/empty_target_batch:np.float64(0.234375) - actor/grad_norm:np.float64(0.4067366272211075) - perf/mfu/actor:np.float64(0.05678632217924573) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(202.01263046264648) - actor/lr:np.float64(2e-07) - training/global_step:3 - training/epoch:0 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002736525610089302 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002736525610089302 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:452.6796875 - r
(TaskRunner pid=2255397) 
Training Progress:   3%|▎         | 3/100 [02:43<1:23:24, 51.60s/it]
(TaskRunner pid=2255397) step:4 - global_seqlen/min:23313 - global_seqlen/max:27346 - global_seqlen/minmax_diff:4033 - global_seqlen/balanced_min:24709 - global_seqlen/balanced_max:24717 - global_seqlen/mean:24714.5 - actor/entropy:0.54982590675354 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7950142621994019 - training/rollout_probs_diff_mean:0.004631490912288427 - training/rollout_probs_diff_std:0.009951530024409294 - training/rollout_actor_probs_pearson_corr:0.9994604587554932 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.79296875 - self_distillation/correct_sample_fraction:0.4453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.79296875 - rollout_corr/rollout_is_mean:1.0000749826431274 - rollout_corr/rollout_is_ratio_fraction_high:8.3895429270342e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.67790858540684e-05 - rollout_corr/rollout_is_max:4.939357280731201 - rollout_corr/rollout_is_min:0.19150345027446747 - rollout_corr/rollout_is_std:0.0351443812251091 - rollout_corr/rollout_is_eff_sample_size:0.9987665151242391 - rollout_corr/rollout_is_seq_mean:1.0000817775726318 - rollout_corr/rollout_is_seq_std:0.0017939795507118106 - rollout_corr/rollout_is_seq_max:1.0054137706756592 - rollout_corr/rollout_is_seq_min:0.9938213229179382 - rollout_corr/rollout_is_seq_max_deviation:0.006178677082061768 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8050872194580734) - rollout_corr/training_log_ppl:np.float64(0.5758670450886711) - rollout_corr/kl:np.float64(0.0006997874668996928) - rollout_corr/k3_kl:np.float64(0.0006716491621432397) - rollout_corr/rollout_ppl:np.float64(1.803772633895278) - rollout_corr/rollout_log_ppl:np.float64(0.5751672576880082) - rollout_corr/log_ppl_diff:np.float64(0.000699787400662899) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015515540726482868) - rollout_corr/log_ppl_diff_max:np.float64(0.00667417049407959) - rollout_corr/log_ppl_diff_min:np.float64(-0.004105627536773682) - rollout_corr/ppl_ratio:np.float64(1.0007018391042948) - rollout_corr/chi2_token:np.float64(0.0013059712946414948) - rollout_corr/chi2_seq:np.float64(0.8464284441433847) - actor/pg_loss:np.float64(0.0077759657287970185) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014584792415917036) - actor/ppo_kl:np.float64(0.00010806482939074158) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.79296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.79296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4453125) - self_distillation/token_reweight_w_mean:np.float64(18390.18527350994) - self_distillation/token_reweight_w_std:np.float64(278661.7640647436) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0027553322724998) - self_distillation/token_reweight_w_clip_frac:np.float64(0.18228401592932642) - self_distillation/empty_target_batch:np.float64(0.20703125) - actor/grad_norm:np.float64(0.41205956041812897) - perf/mfu/actor:np.float64(0.05901765112715433) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(199.91095733642578) - actor/lr:np.float64(3e-07) - training/global_step:4 - training/epoch:0 - critic/score/mean:0.4453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0022473488934338093 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0022473488934338093 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_l
(TaskRunner pid=2255397) 
Training Progress:   4%|▍         | 4/100 [03:25<1:16:32, 47.83s/it]
(TaskRunner pid=2255397) step:5 - global_seqlen/min:18324 - global_seqlen/max:26762 - global_seqlen/minmax_diff:8438 - global_seqlen/balanced_min:22550 - global_seqlen/balanced_max:22563 - global_seqlen/mean:22556.25 - actor/entropy:0.5743926167488098 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18463686108589172 - training/rollout_probs_diff_mean:0.004951741546392441 - training/rollout_probs_diff_std:0.009703434072434902 - training/rollout_actor_probs_pearson_corr:0.9994615912437439 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.42578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.000128984451294 - rollout_corr/rollout_is_ratio_fraction_high:9.058463547262363e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.8116927094524726e-05 - rollout_corr/rollout_is_max:2.5132858753204346 - rollout_corr/rollout_is_min:0.4844912588596344 - rollout_corr/rollout_is_std:0.03672673553228378 - rollout_corr/rollout_is_eff_sample_size:0.9986534393962608 - rollout_corr/rollout_is_seq_mean:1.0001883506774902 - rollout_corr/rollout_is_seq_std:0.0020744451321661472 - rollout_corr/rollout_is_seq_max:1.0107048749923706 - rollout_corr/rollout_is_seq_min:0.9943471550941467 - rollout_corr/rollout_is_seq_max_deviation:0.010704874992370605 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8240970252081752) - rollout_corr/training_log_ppl:np.float64(0.5867100825998932) - rollout_corr/kl:np.float64(0.000413151877571849) - rollout_corr/k3_kl:np.float64(0.0007104324042757071) - rollout_corr/rollout_ppl:np.float64(1.8233414590358734) - rollout_corr/rollout_log_ppl:np.float64(0.5862969291047193) - rollout_corr/log_ppl_diff:np.float64(0.0004131534951739013) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015994933783076704) - rollout_corr/log_ppl_diff_max:np.float64(0.005790770053863525) - rollout_corr/log_ppl_diff_min:np.float64(-0.006320059299468994) - rollout_corr/ppl_ratio:np.float64(1.0004152327310294) - rollout_corr/chi2_token:np.float64(0.0020708865486085415) - rollout_corr/chi2_seq:np.float64(1.291173740522936) - actor/pg_loss:np.float64(0.00783018337097019) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00029261777604006056) - actor/ppo_kl:np.float64(-0.00011415763041355831) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.42578125) - self_distillation/token_reweight_w_mean:np.float64(7564.456899565645) - self_distillation/token_reweight_w_std:np.float64(147554.27462118154) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0038912589661777) - self_distillation/token_reweight_w_clip_frac:np.float64(0.19292002776637673) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.37297531962394714) - perf/mfu/actor:np.float64(0.05395284544548605) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(198.01452255249023) - actor/lr:np.float64(4e-07) - training/global_step:5 - training/epoch:0 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0022725421003997326 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0022725421003997326 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_lengt
(TaskRunner pid=2255397) 
Training Progress:   5%|▌         | 5/100 [04:08<1:13:10, 46.22s/it]
(TaskRunner pid=2255397) step:6 - global_seqlen/min:19330 - global_seqlen/max:29022 - global_seqlen/minmax_diff:9692 - global_seqlen/balanced_min:23612 - global_seqlen/balanced_max:23621 - global_seqlen/mean:23617.625 - actor/entropy:0.5028008818626404 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5328631401062012 - training/rollout_probs_diff_mean:0.004628530703485012 - training/rollout_probs_diff_std:0.009826785884797573 - training/rollout_actor_probs_pearson_corr:0.9994211196899414 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8515625 - self_distillation/correct_sample_fraction:0.421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8515625 - rollout_corr/rollout_is_mean:0.9999843239784241 - rollout_corr/rollout_is_ratio_fraction_high:8.433054063061718e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.433054063061718e-06 - rollout_corr/rollout_is_max:2.1156668663024902 - rollout_corr/rollout_is_min:0.4510452151298523 - rollout_corr/rollout_is_std:0.03478982672095299 - rollout_corr/rollout_is_eff_sample_size:0.998791012294431 - rollout_corr/rollout_is_seq_mean:0.9999687671661377 - rollout_corr/rollout_is_seq_std:0.0017612202791497111 - rollout_corr/rollout_is_seq_max:1.0042917728424072 - rollout_corr/rollout_is_seq_min:0.994244396686554 - rollout_corr/rollout_is_seq_max_deviation:0.005755603313446045 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.732960639987141) - rollout_corr/training_log_ppl:np.float64(0.5360075897770002) - rollout_corr/kl:np.float64(0.000639045071211708) - rollout_corr/k3_kl:np.float64(0.0006780325196018566) - rollout_corr/rollout_ppl:np.float64(1.7318218029104173) - rollout_corr/rollout_log_ppl:np.float64(0.5353685396839865) - rollout_corr/log_ppl_diff:np.float64(0.0006390500930137932) - rollout_corr/log_ppl_abs_diff:np.float64(0.001630233891773969) - rollout_corr/log_ppl_diff_max:np.float64(0.007409811019897461) - rollout_corr/log_ppl_diff_min:np.float64(-0.0069411396980285645) - rollout_corr/ppl_ratio:np.float64(1.0006413247901946) - rollout_corr/chi2_token:np.float64(0.0014372044242918491) - rollout_corr/chi2_seq:np.float64(1.4436022932641208) - actor/pg_loss:np.float64(0.00812386441975832) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004630846383406606) - actor/ppo_kl:np.float64(-4.196475500339858e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8515625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8515625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.421875) - self_distillation/token_reweight_w_mean:np.float64(19923.261127433972) - self_distillation/token_reweight_w_std:np.float64(319160.37308162905) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0048198685981333) - self_distillation/token_reweight_w_clip_frac:np.float64(0.18613416605512612) - self_distillation/empty_target_batch:np.float64(0.1484375) - actor/grad_norm:np.float64(0.3917096257209778) - perf/mfu/actor:np.float64(0.05549581290987645) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.47495651245117) - actor/lr:np.float64(5e-07) - training/global_step:6 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008403538726270199 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008403538726270199 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/m
(TaskRunner pid=2255397) 
Training Progress:   6%|▌         | 6/100 [04:52<1:10:57, 45.29s/it]
(TaskRunner pid=2255397) step:7 - global_seqlen/min:18757 - global_seqlen/max:24212 - global_seqlen/minmax_diff:5455 - global_seqlen/balanced_min:22489 - global_seqlen/balanced_max:22504 - global_seqlen/mean:22499.75 - actor/entropy:0.5316406488418579 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8049387335777283 - training/rollout_probs_diff_mean:0.004808515310287476 - training/rollout_probs_diff_std:0.0102939298376441 - training/rollout_actor_probs_pearson_corr:0.9993799924850464 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.69921875 - self_distillation/correct_sample_fraction:0.33203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.69921875 - rollout_corr/rollout_is_mean:1.000180721282959 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.7163583076326177e-05 - rollout_corr/rollout_is_max:1.9837769269943237 - rollout_corr/rollout_is_min:0.12898790836334229 - rollout_corr/rollout_is_std:0.0356261320412159 - rollout_corr/rollout_is_eff_sample_size:0.9987328632041119 - rollout_corr/rollout_is_seq_mean:1.0002025365829468 - rollout_corr/rollout_is_seq_std:0.0019561529625207186 - rollout_corr/rollout_is_seq_max:1.0071183443069458 - rollout_corr/rollout_is_seq_min:0.9942708015441895 - rollout_corr/rollout_is_seq_max_deviation:0.007118344306945801 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7598580992780626) - rollout_corr/training_log_ppl:np.float64(0.5522571774781682) - rollout_corr/kl:np.float64(0.0006528583797624776) - rollout_corr/k3_kl:np.float64(0.0007534040726682179) - rollout_corr/rollout_ppl:np.float64(1.7586921667680144) - rollout_corr/rollout_log_ppl:np.float64(0.5516043193638325) - rollout_corr/log_ppl_diff:np.float64(0.000652858114335686) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017435748013667762) - rollout_corr/log_ppl_diff_max:np.float64(0.010204493999481201) - rollout_corr/log_ppl_diff_min:np.float64(-0.006472587585449219) - rollout_corr/ppl_ratio:np.float64(1.000655469717458) - rollout_corr/chi2_token:np.float64(0.001773336436599493) - rollout_corr/chi2_seq:np.float64(1.4482529927045107) - actor/pg_loss:np.float64(0.007577315904200077) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005699304592781118) - actor/ppo_kl:np.float64(0.00017036601963926756) - actor/pg_clipfrac_lower:np.float64(1.5439723938470706e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.69921875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.33203125) - self_distillation/token_reweight_w_mean:np.float64(40804.10983084305) - self_distillation/token_reweight_w_std:np.float64(550034.2616531486) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.002954869531095) - self_distillation/token_reweight_w_clip_frac:np.float64(0.17178265267284587) - self_distillation/empty_target_batch:np.float64(0.30078125) - actor/grad_norm:np.float64(0.42640380561351776) - perf/mfu/actor:np.float64(0.053623795084905154) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.01875686645508) - actor/lr:np.float64(6e-07) - training/global_step:7 - training/epoch:0 - critic/score/mean:0.33203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.33203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00599684938788414 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.00599684938788414 - critic/returns/max:0.875 - critic/returns/min:-0.75 - respons
(TaskRunner pid=2255397) 
Training Progress:   7%|▋         | 7/100 [05:35<1:08:52, 44.44s/it]
(TaskRunner pid=2255397) step:8 - global_seqlen/min:19293 - global_seqlen/max:28026 - global_seqlen/minmax_diff:8733 - global_seqlen/balanced_min:24653 - global_seqlen/balanced_max:24794 - global_seqlen/mean:24675.25 - actor/entropy:0.5080747604370117 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.408370703458786 - training/rollout_probs_diff_mean:0.0044975439086556435 - training/rollout_probs_diff_std:0.009530728682875633 - training/rollout_actor_probs_pearson_corr:0.9994667768478394 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8828125 - self_distillation/correct_sample_fraction:0.46875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8828125 - rollout_corr/rollout_is_mean:1.0000174045562744 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.162332051142585e-06 - rollout_corr/rollout_is_max:1.8054882287979126 - rollout_corr/rollout_is_min:0.462515652179718 - rollout_corr/rollout_is_std:0.0343310683965683 - rollout_corr/rollout_is_eff_sample_size:0.9988226463404877 - rollout_corr/rollout_is_seq_mean:1.000030279159546 - rollout_corr/rollout_is_seq_std:0.0018812705529853702 - rollout_corr/rollout_is_seq_max:1.0089062452316284 - rollout_corr/rollout_is_seq_min:0.9933127760887146 - rollout_corr/rollout_is_seq_max_deviation:0.008906245231628418 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.747896637301892) - rollout_corr/training_log_ppl:np.float64(0.541930019389838) - rollout_corr/kl:np.float64(0.0006643355780084903) - rollout_corr/k3_kl:np.float64(0.0006548753977995148) - rollout_corr/rollout_ppl:np.float64(1.7467446983791888) - rollout_corr/rollout_log_ppl:np.float64(0.5412656838307157) - rollout_corr/log_ppl_diff:np.float64(0.0006643355591222644) - rollout_corr/log_ppl_abs_diff:np.float64(0.00149785284884274) - rollout_corr/log_ppl_diff_max:np.float64(0.007577002048492432) - rollout_corr/log_ppl_diff_min:np.float64(-0.0072296857833862305) - rollout_corr/ppl_ratio:np.float64(1.000666505889967) - rollout_corr/chi2_token:np.float64(0.0012856200337409973) - rollout_corr/chi2_seq:np.float64(0.7123665248509496) - actor/pg_loss:np.float64(0.007868309505283833) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004503007885432453) - actor/ppo_kl:np.float64(5.489206566622329e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.46875) - self_distillation/token_reweight_w_mean:np.float64(9891.629087405046) - self_distillation/token_reweight_w_std:np.float64(229237.51225678227) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0036344188265502) - self_distillation/token_reweight_w_clip_frac:np.float64(0.18639719724887982) - self_distillation/empty_target_batch:np.float64(0.1171875) - actor/grad_norm:np.float64(0.3860222175717354) - perf/mfu/actor:np.float64(0.05765871612685469) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.10034942626953) - actor/lr:np.float64(7e-07) - training/global_step:8 - training/epoch:0 - critic/score/mean:0.46875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008611260913312435 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008611260913312435 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:478.5703125 - respons
(TaskRunner pid=2255397) 
Training Progress:   8%|▊         | 8/100 [06:20<1:08:25, 44.62s/it]
(TaskRunner pid=2255397) step:9 - global_seqlen/min:19805 - global_seqlen/max:26584 - global_seqlen/minmax_diff:6779 - global_seqlen/balanced_min:23155 - global_seqlen/balanced_max:23161 - global_seqlen/mean:23157.5 - actor/entropy:0.5331449508666992 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5838292837142944 - training/rollout_probs_diff_mean:0.004796977620571852 - training/rollout_probs_diff_std:0.010206756182014942 - training/rollout_actor_probs_pearson_corr:0.9993701577186584 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.66796875 - self_distillation/correct_sample_fraction:0.359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.66796875 - rollout_corr/rollout_is_mean:0.9999645352363586 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.796686956367921e-05 - rollout_corr/rollout_is_max:1.6195783615112305 - rollout_corr/rollout_is_min:0.3546355068683624 - rollout_corr/rollout_is_std:0.03580470383167267 - rollout_corr/rollout_is_eff_sample_size:0.9987194268518426 - rollout_corr/rollout_is_seq_mean:0.9999064207077026 - rollout_corr/rollout_is_seq_std:0.001988921547308564 - rollout_corr/rollout_is_seq_max:1.004913330078125 - rollout_corr/rollout_is_seq_min:0.9934803247451782 - rollout_corr/rollout_is_seq_max_deviation:0.006519675254821777 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7555229673162103) - rollout_corr/training_log_ppl:np.float64(0.5492386808618903) - rollout_corr/kl:np.float64(0.000775432549749766) - rollout_corr/k3_kl:np.float64(0.0008021395320838565) - rollout_corr/rollout_ppl:np.float64(1.7541374694555998) - rollout_corr/rollout_log_ppl:np.float64(0.5484632462612353) - rollout_corr/log_ppl_diff:np.float64(0.0007754346006549895) - rollout_corr/log_ppl_abs_diff:np.float64(0.001795541902538389) - rollout_corr/log_ppl_diff_max:np.float64(0.010334789752960205) - rollout_corr/log_ppl_diff_min:np.float64(-0.0038338303565979004) - rollout_corr/ppl_ratio:np.float64(1.000778182875365) - rollout_corr/chi2_token:np.float64(0.0016523946542292833) - rollout_corr/chi2_seq:np.float64(0.9519539077300578) - actor/pg_loss:np.float64(0.007556321215815842) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006206532375472307) - actor/ppo_kl:np.float64(-5.760855917635865e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.66796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.359375) - self_distillation/token_reweight_w_mean:np.float64(22852.849850326777) - self_distillation/token_reweight_w_std:np.float64(403812.0357487643) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0031312247738242) - self_distillation/token_reweight_w_clip_frac:np.float64(0.15560277574695647) - self_distillation/empty_target_batch:np.float64(0.33203125) - actor/grad_norm:np.float64(0.34761351346969604) - perf/mfu/actor:np.float64(0.05432608742997578) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.07957077026367) - actor/lr:np.float64(8e-07) - training/global_step:9 - training/epoch:0 - critic/score/mean:0.359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008080599829554558 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008080599829554558 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:434.82812
(TaskRunner pid=2255397) 
Training Progress:   9%|▉         | 9/100 [07:02<1:06:50, 44.07s/it]
(TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 10}
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) validation generation end
(TaskRunner pid=2255397) [prompt] system
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2255397) <reasoning>
(TaskRunner pid=2255397) ...
(TaskRunner pid=2255397) </reasoning>
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) ...
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) A
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) user
(TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2255397) Please reason step by step.
(TaskRunner pid=2255397) assistant
(TaskRunner pid=2255397) <think>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) </think>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) [response] <reasoning>
(TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to examine the structure of the product and match it with the structure of the reactant provided in the options.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) The product has the following key features:
(TaskRunner pid=2255397) - A nitrogen-containing ring (likely a pyrrole or imidazole-like structure).
(TaskRunner pid=2255397) - A substituent "Cc2ccc(Cl)cc2C(F)(F)F", which is a benzene ring substituted with a chlorine atom and a trifluoromethyl group.
(TaskRunner pid=2255397) - The overall structure suggests that the reactant must have a similar core structure, with the necessary functional groups for the formation of the final product.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Let's analyze the options:
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option A**:  
(TaskRunner pid=2255397) "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"  
(TaskRunner pid=2255397) This structure contains a nitrogen-containing ring with the correct substituent "Cc2ccc(Cl)cc2C(F)(F)F". It also has a carbonyl group and a nitrogen atom, which could be part of a condensation or coupling reaction. This matches the structure of the product closely.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option B**:  
(TaskRunner pid=2255397) "O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1"  
(TaskRunner pid=2255397) This option introduces a chlorine atom directly on the nitrogen, which is not present in the product. Also, the structure seems to have a different arrangement of substituents, making it unlikely to be the correct reactant.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option C**:  
(TaskRunner pid=2255397) "O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1"  
(TaskRunner pid=2255397) This structure has a hydroxyl group attached to the benzene ring, which is not present in the product. The presence of this hydroxyl group suggests a different synthetic pathway and thus is not the correct reactant.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option D**:  
(TaskRunner pid=2255397) "O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F"  
(TaskRunner pid=2255397) This option has a different arrangement of the substituents and introduces a double bond between the carbon atoms, which is not part of the product structure. It also lacks the necessary nitrogen-containing ring.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Conclusion**:  
(TaskRunner pid=2255397) Only **Option A** contains the correct substituent "Cc2ccc(Cl)cc2C(F)(F)F" and the necessary nitrogen-containing ring structure. It is the only option that matches the structure of the product directly.
(TaskRunner pid=2255397) </reasoning>
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) A
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) [ground_truth] A
(TaskRunner pid=2255397) [score] 1.0
(TaskRunner pid=2255397) [acc] 1.0
(TaskRunner pid=2255397) [pred] A
(TaskRunner pid=2255397) [incorrect_format] 1
(TaskRunner pid=2255397) [feedback] 
(TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10
(TaskRunner pid=2255397) step:10 - global_seqlen/min:18995 - global_seqlen/max:25222 - global_seqlen/minmax_diff:6227 - global_seqlen/balanced_min:22810 - global_seqlen/balanced_max:22819 - global_seqlen/mean:22815.375 - actor/entropy:0.5086405277252197 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48631101846694946 - training/rollout_probs_diff_mean:0.004814056679606438 - training/rollout_probs_diff_std:0.010112768970429897 - training/rollout_actor_probs_pearson_corr:0.9993922114372253 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6796875 - self_distillation/correct_sample_fraction:0.37109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6796875 - rollout_corr/rollout_is_mean:1.0000672340393066 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.7611369912629016e-05 - rollout_corr/rollout_is_max:1.758644938468933 - rollout_corr/rollout_is_min:0.47177526354789734 - rollout_corr/rollout_is_std:0.0358346551656723 - rollout_corr/rollout_is_eff_sample_size:0.9987177621962007 - rollout_corr/rollout_is_seq_mean:1.0001494884490967 - rollout_corr/rollout_is_seq_std:0.001915071508847177 - rollout_corr/rollout_is_seq_max:1.0051805973052979 - rollout_corr/rollout_is_seq_min:0.9944745898246765 - rollout_corr/rollout_is_seq_max_deviation:0.005525410175323486 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7283513699658215) - rollout_corr/training_log_ppl:np.float64(0.5342370558064431) - rollout_corr/kl:np.float64(0.00048592705383754264) - rollout_corr/k3_kl:np.float64(0.0007897751748942028) - rollout_corr/rollout_ppl:np.float64(1.7275344631634653) - rollout_corr/rollout_log_ppl:np.float64(0.5337511280667968) - rollout_corr/log_ppl_diff:np.float64(0.00048592773964628577) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017402138910256326) - rollout_corr/log_ppl_diff_max:np.float64(0.008036613464355469) - rollout_corr/log_ppl_diff_min:np.float64(-0.007182866334915161) - rollout_corr/ppl_ratio:np.float64(1.0004885760135949) - rollout_corr/chi2_token:np.float64(0.002185102319344878) - rollout_corr/chi2_seq:np.float64(1.4425214361399412) - actor/pg_loss:np.float64(0.008299664361402392) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006863867054107686) - actor/ppo_kl:np.float64(-5.119195938618759e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.37109375) - self_distillation/token_reweight_w_mean:np.float64(42394.31746227876) - self_distillation/token_reweight_w_std:np.float64(702394.3469068523) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012782558333129) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1625702439341694) - self_distillation/empty_target_batch:np.float64(0.3203125) - actor/grad_norm:np.float64(0.3786138668656349) - perf/mfu/actor:np.float64(0.054599702355521036) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.04931259155273) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.4217261904761905) - val-aux/sciknoweval/reward/std@16:np.float64(0.28826355015906874) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5430761904761905) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.26128001800402223) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.3013761904761905) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2277461781455412) - val-a
(TaskRunner pid=2255397) 
Training Progress:  10%|█         | 10/100 [10:23<2:18:30, 92.33s/it]
(TaskRunner pid=2255397) step:11 - global_seqlen/min:18238 - global_seqlen/max:25942 - global_seqlen/minmax_diff:7704 - global_seqlen/balanced_min:22325 - global_seqlen/balanced_max:22730 - global_seqlen/mean:22380.25 - actor/entropy:0.5434318780899048 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.38079291582107544 - training/rollout_probs_diff_mean:0.005151212681084871 - training/rollout_probs_diff_std:0.010387294925749302 - training/rollout_actor_probs_pearson_corr:0.9993837475776672 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.48046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.0001193284988403 - rollout_corr/rollout_is_ratio_fraction_high:9.758572559803724e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:4.194515228271484 - rollout_corr/rollout_is_min:0.5521866679191589 - rollout_corr/rollout_is_std:0.03714950755238533 - rollout_corr/rollout_is_eff_sample_size:0.9986220538290654 - rollout_corr/rollout_is_seq_mean:1.0001423358917236 - rollout_corr/rollout_is_seq_std:0.0020691656973212957 - rollout_corr/rollout_is_seq_max:1.0071853399276733 - rollout_corr/rollout_is_seq_min:0.993625283241272 - rollout_corr/rollout_is_seq_max_deviation:0.00718533992767334 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7692446084693074) - rollout_corr/training_log_ppl:np.float64(0.5590797059703618) - rollout_corr/kl:np.float64(0.0006710044406990079) - rollout_corr/k3_kl:np.float64(0.0007906234051233696) - rollout_corr/rollout_ppl:np.float64(1.7680208389647305) - rollout_corr/rollout_log_ppl:np.float64(0.5584087027818896) - rollout_corr/log_ppl_diff:np.float64(0.0006710031884722412) - rollout_corr/log_ppl_abs_diff:np.float64(0.001697185740340501) - rollout_corr/log_ppl_diff_max:np.float64(0.006523817777633667) - rollout_corr/log_ppl_diff_min:np.float64(-0.005903065204620361) - rollout_corr/ppl_ratio:np.float64(1.0006734766066074) - rollout_corr/chi2_token:np.float64(0.0018631014972925186) - rollout_corr/chi2_seq:np.float64(1.0500562572851777) - actor/pg_loss:np.float64(0.010553930653259158) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004439412439296575) - actor/ppo_kl:np.float64(6.877072929745154e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48046875) - self_distillation/token_reweight_w_mean:np.float64(85283.25226895092) - self_distillation/token_reweight_w_std:np.float64(1114820.7389926258) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0022825964260846) - self_distillation/token_reweight_w_clip_frac:np.float64(0.21137651632307097) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.4761762246489525) - perf/mfu/actor:np.float64(0.05262875043012334) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.41241455078125) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.48046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003749731695279479 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003749731695279479 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:
(TaskRunner pid=2255397) 
Training Progress:  11%|█         | 11/100 [11:08<1:55:21, 77.77s/it]
(TaskRunner pid=2255397) step:12 - global_seqlen/min:19921 - global_seqlen/max:27357 - global_seqlen/minmax_diff:7436 - global_seqlen/balanced_min:23213 - global_seqlen/balanced_max:23297 - global_seqlen/mean:23233.875 - actor/entropy:0.5037309527397156 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.16815555095672607 - training/rollout_probs_diff_mean:0.004780740477144718 - training/rollout_probs_diff_std:0.009941385127604008 - training/rollout_actor_probs_pearson_corr:0.9994201064109802 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.85546875 - self_distillation/correct_sample_fraction:0.46484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.85546875 - rollout_corr/rollout_is_mean:0.999710738658905 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.6984993635560386e-05 - rollout_corr/rollout_is_max:1.6297073364257812 - rollout_corr/rollout_is_min:0.3698853552341461 - rollout_corr/rollout_is_std:0.035432368516922 - rollout_corr/rollout_is_eff_sample_size:0.9987455864550007 - rollout_corr/rollout_is_seq_mean:0.9997212886810303 - rollout_corr/rollout_is_seq_std:0.0021238892804831266 - rollout_corr/rollout_is_seq_max:1.0065120458602905 - rollout_corr/rollout_is_seq_min:0.9934496879577637 - rollout_corr/rollout_is_seq_max_deviation:0.006550312042236328 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7103436021134257) - rollout_corr/training_log_ppl:np.float64(0.5244237306469586) - rollout_corr/kl:np.float64(0.0010640324547948055) - rollout_corr/k3_kl:np.float64(0.0007240620033144296) - rollout_corr/rollout_ppl:np.float64(1.7084603938274086) - rollout_corr/rollout_log_ppl:np.float64(0.5233596969628707) - rollout_corr/log_ppl_diff:np.float64(0.0010640336840879172) - rollout_corr/log_ppl_abs_diff:np.float64(0.001916373468702659) - rollout_corr/log_ppl_diff_max:np.float64(0.007551968097686768) - rollout_corr/log_ppl_diff_min:np.float64(-0.006095826625823975) - rollout_corr/ppl_ratio:np.float64(1.001067083561793) - rollout_corr/chi2_token:np.float64(0.0007682361174374819) - rollout_corr/chi2_seq:np.float64(0.6293782293796539) - actor/pg_loss:np.float64(0.007974700187332928) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00048827441059984267) - actor/ppo_kl:np.float64(0.00010889819034787251) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.85546875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.85546875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.46484375) - self_distillation/token_reweight_w_mean:np.float64(5987.44374671299) - self_distillation/token_reweight_w_std:np.float64(110518.26587392716) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0042279572226107) - self_distillation/token_reweight_w_clip_frac:np.float64(0.17869807308306918) - self_distillation/empty_target_batch:np.float64(0.14453125) - actor/grad_norm:np.float64(0.39314158260822296) - perf/mfu/actor:np.float64(0.05490032892199064) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.42404556274414) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0070593878626823425 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0070593878626823425 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_lengt
(TaskRunner pid=2255397) 
Training Progress:  12%|█▏        | 12/100 [11:52<1:39:16, 67.68s/it]
(TaskRunner pid=2255397) step:13 - global_seqlen/min:18814 - global_seqlen/max:23875 - global_seqlen/minmax_diff:5061 - global_seqlen/balanced_min:21556 - global_seqlen/balanced_max:21562 - global_seqlen/mean:21560.5 - actor/entropy:0.540065348148346 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2304113507270813 - training/rollout_probs_diff_mean:0.005129465367645025 - training/rollout_probs_diff_std:0.010314878076314926 - training/rollout_actor_probs_pearson_corr:0.9993942975997925 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.796875 - self_distillation/correct_sample_fraction:0.375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.796875 - rollout_corr/rollout_is_mean:0.9999523758888245 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.84678445092868e-06 - rollout_corr/rollout_is_max:1.643043875694275 - rollout_corr/rollout_is_min:0.3293769061565399 - rollout_corr/rollout_is_std:0.03673160448670387 - rollout_corr/rollout_is_eff_sample_size:0.9986523694012616 - rollout_corr/rollout_is_seq_mean:0.9999930262565613 - rollout_corr/rollout_is_seq_std:0.0019651646725833416 - rollout_corr/rollout_is_seq_max:1.0069950819015503 - rollout_corr/rollout_is_seq_min:0.9950668215751648 - rollout_corr/rollout_is_seq_max_deviation:0.006995081901550293 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.75340425549075) - rollout_corr/training_log_ppl:np.float64(0.5529010602040216) - rollout_corr/kl:np.float64(0.0008567688251606143) - rollout_corr/k3_kl:np.float64(0.0008690143870353495) - rollout_corr/rollout_ppl:np.float64(1.7518776911310852) - rollout_corr/rollout_log_ppl:np.float64(0.5520442884298973) - rollout_corr/log_ppl_diff:np.float64(0.0008567717741243541) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018868366605602205) - rollout_corr/log_ppl_diff_max:np.float64(0.00990372896194458) - rollout_corr/log_ppl_diff_min:np.float64(-0.005962073802947998) - rollout_corr/ppl_ratio:np.float64(1.0008597834967077) - rollout_corr/chi2_token:np.float64(0.0017287565860897303) - rollout_corr/chi2_seq:np.float64(0.7434897748753428) - actor/pg_loss:np.float64(0.009388480219058692) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001301017089645029) - actor/ppo_kl:np.float64(0.0001421315267471357) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.375) - self_distillation/token_reweight_w_mean:np.float64(1917.8817092673853) - self_distillation/token_reweight_w_std:np.float64(43886.33348266815) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0036602555774152) - self_distillation/token_reweight_w_clip_frac:np.float64(0.2082537584064994) - self_distillation/empty_target_batch:np.float64(0.203125) - actor/grad_norm:np.float64(0.43746187537908554) - perf/mfu/actor:np.float64(0.051655857866616195) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.21465253829956) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003404525574296713 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003404525574296713 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:396.703125 - response_length/ma
(TaskRunner pid=2255397) 
Training Progress:  13%|█▎        | 13/100 [12:33<1:26:18, 59.52s/it]
(TaskRunner pid=2255397) step:14 - global_seqlen/min:17920 - global_seqlen/max:21796 - global_seqlen/minmax_diff:3876 - global_seqlen/balanced_min:20032 - global_seqlen/balanced_max:20152 - global_seqlen/mean:20054.25 - actor/entropy:0.5020557641983032 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18802747130393982 - training/rollout_probs_diff_mean:0.005197192542254925 - training/rollout_probs_diff_std:0.01069364883005619 - training/rollout_actor_probs_pearson_corr:0.9992953538894653 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.40625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:1.000118374824524 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6423780918121338 - rollout_corr/rollout_is_min:0.6023645997047424 - rollout_corr/rollout_is_std:0.03736548125743866 - rollout_corr/rollout_is_eff_sample_size:0.9986062429080049 - rollout_corr/rollout_is_seq_mean:1.0001424551010132 - rollout_corr/rollout_is_seq_std:0.002069722395390272 - rollout_corr/rollout_is_seq_max:1.0075104236602783 - rollout_corr/rollout_is_seq_min:0.9939577579498291 - rollout_corr/rollout_is_seq_max_deviation:0.00751042366027832 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7107822382822633) - rollout_corr/training_log_ppl:np.float64(0.5240909514250234) - rollout_corr/kl:np.float64(0.0007750868314850834) - rollout_corr/k3_kl:np.float64(0.0011061159723340097) - rollout_corr/rollout_ppl:np.float64(1.7093983073718846) - rollout_corr/rollout_log_ppl:np.float64(0.5233158656046726) - rollout_corr/log_ppl_diff:np.float64(0.000775085820350796) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019063884974457324) - rollout_corr/log_ppl_diff_max:np.float64(0.009017229080200195) - rollout_corr/log_ppl_diff_min:np.float64(-0.007934510707855225) - rollout_corr/ppl_ratio:np.float64(1.0007783074397594) - rollout_corr/chi2_token:np.float64(0.0027928389608860016) - rollout_corr/chi2_seq:np.float64(0.6327834515832365) - actor/pg_loss:np.float64(0.009141277987509966) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0023054060643517005) - actor/ppo_kl:np.float64(0.00017033189165172047) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.40625) - self_distillation/token_reweight_w_mean:np.float64(51790.3643477282) - self_distillation/token_reweight_w_std:np.float64(710581.8369147222) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0032089522574097) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1833768331562169) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.5500108152627945) - perf/mfu/actor:np.float64(0.04745480566246875) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.28086853027344) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.40625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:1.6309666534652933e-05 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:1.6309666534652933e-05 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:359.2578125 - response_leng
(TaskRunner pid=2255397) 
Training Progress:  14%|█▍        | 14/100 [13:15<1:17:50, 54.31s/it]
(TaskRunner pid=2255397) step:15 - global_seqlen/min:17292 - global_seqlen/max:23462 - global_seqlen/minmax_diff:6170 - global_seqlen/balanced_min:20153 - global_seqlen/balanced_max:20223 - global_seqlen/mean:20173.0 - actor/entropy:0.4989277422428131 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23447632789611816 - training/rollout_probs_diff_mean:0.005335042718797922 - training/rollout_probs_diff_std:0.011061806231737137 - training/rollout_actor_probs_pearson_corr:0.99927818775177 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.46875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9997453093528748 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5897420644760132 - rollout_corr/rollout_is_min:0.5000474452972412 - rollout_corr/rollout_is_std:0.03860742598772049 - rollout_corr/rollout_is_eff_sample_size:0.9985110313307147 - rollout_corr/rollout_is_seq_mean:0.9997358918190002 - rollout_corr/rollout_is_seq_std:0.0024659933988004923 - rollout_corr/rollout_is_seq_max:1.0084636211395264 - rollout_corr/rollout_is_seq_min:0.9930576682090759 - rollout_corr/rollout_is_seq_max_deviation:0.008463621139526367 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7213433790020645) - rollout_corr/training_log_ppl:np.float64(0.530678162467666) - rollout_corr/kl:np.float64(0.0010814671836847367) - rollout_corr/k3_kl:np.float64(0.000888961919315534) - rollout_corr/rollout_ppl:np.float64(1.7193831540644169) - rollout_corr/rollout_log_ppl:np.float64(0.5295966898556799) - rollout_corr/log_ppl_diff:np.float64(0.0010814726119861007) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019996468909084797) - rollout_corr/log_ppl_diff_max:np.float64(0.012681245803833008) - rollout_corr/log_ppl_diff_min:np.float64(-0.0078064799308776855) - rollout_corr/ppl_ratio:np.float64(1.0010852627456188) - rollout_corr/chi2_token:np.float64(0.0014137444086372852) - rollout_corr/chi2_seq:np.float64(0.5360429051797837) - actor/pg_loss:np.float64(0.007955617504194379) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006377134768627002) - actor/ppo_kl:np.float64(1.769121835337728e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.46875) - self_distillation/token_reweight_w_mean:np.float64(8754.255326425191) - self_distillation/token_reweight_w_std:np.float64(117017.50059508975) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001879719318822) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1550164521613624) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.4198375418782234) - perf/mfu/actor:np.float64(0.04879610399473919) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.12841796875) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.46875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00847491342574358 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00847491342574358 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:343.5 - response_length/max:1036.0 
(TaskRunner pid=2255397) 
Training Progress:  15%|█▌        | 15/100 [13:55<1:10:45, 49.94s/it]
(TaskRunner pid=2255397) step:16 - global_seqlen/min:17365 - global_seqlen/max:24430 - global_seqlen/minmax_diff:7065 - global_seqlen/balanced_min:20809 - global_seqlen/balanced_max:20815 - global_seqlen/mean:20814.0 - actor/entropy:0.4525703191757202 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44003361463546753 - training/rollout_probs_diff_mean:0.0051152328960597515 - training/rollout_probs_diff_std:0.011277093552052975 - training/rollout_actor_probs_pearson_corr:0.9991934299468994 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.82421875 - self_distillation/correct_sample_fraction:0.47265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.82421875 - rollout_corr/rollout_is_mean:1.000049352645874 - rollout_corr/rollout_is_ratio_fraction_high:1.0565463526290841e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.339278479572386e-05 - rollout_corr/rollout_is_max:2.1190927028656006 - rollout_corr/rollout_is_min:0.16761082410812378 - rollout_corr/rollout_is_std:0.03827483579516411 - rollout_corr/rollout_is_eff_sample_size:0.9985372988785958 - rollout_corr/rollout_is_seq_mean:1.0000265836715698 - rollout_corr/rollout_is_seq_std:0.0022894139401614666 - rollout_corr/rollout_is_seq_max:1.0063250064849854 - rollout_corr/rollout_is_seq_min:0.992612361907959 - rollout_corr/rollout_is_seq_max_deviation:0.007387638092041016 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6002246476709843) - rollout_corr/training_log_ppl:np.float64(0.46000494839972816) - rollout_corr/kl:np.float64(0.0006955834787873982) - rollout_corr/k3_kl:np.float64(0.0009704830327450509) - rollout_corr/rollout_ppl:np.float64(1.5991095416247845) - rollout_corr/rollout_log_ppl:np.float64(0.45930936612421647) - rollout_corr/log_ppl_diff:np.float64(0.0006955822755116969) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021134040143806487) - rollout_corr/log_ppl_diff_max:np.float64(0.007766246795654297) - rollout_corr/log_ppl_diff_min:np.float64(-0.005490928888320923) - rollout_corr/ppl_ratio:np.float64(1.0006991014815867) - rollout_corr/chi2_token:np.float64(0.0025813004467636347) - rollout_corr/chi2_seq:np.float64(3.1832106430083513) - actor/pg_loss:np.float64(0.009350149193778634) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011533425140441977) - actor/ppo_kl:np.float64(-8.012323441874969e-05) - actor/pg_clipfrac_lower:np.float64(1.187310044770129e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.82421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.82421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.47265625) - self_distillation/token_reweight_w_mean:np.float64(35795.58769591758) - self_distillation/token_reweight_w_std:np.float64(482687.72877264104) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0030966859776527) - self_distillation/token_reweight_w_clip_frac:np.float64(0.18662144461995922) - self_distillation/empty_target_batch:np.float64(0.17578125) - actor/grad_norm:np.float64(0.49581966549158096) - perf/mfu/actor:np.float64(0.05050208271241423) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.1108741760254) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.47265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.47265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01058791484683752 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01058791484683752 - critic/returns/max:0.875 - critic
(TaskRunner pid=2255397) 
Training Progress:  16%|█▌        | 16/100 [14:35<1:05:43, 46.94s/it]
(TaskRunner pid=2255397) step:17 - global_seqlen/min:22601 - global_seqlen/max:25800 - global_seqlen/minmax_diff:3199 - global_seqlen/balanced_min:23884 - global_seqlen/balanced_max:24138 - global_seqlen/mean:23919.75 - actor/entropy:0.5084906220436096 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23104846477508545 - training/rollout_probs_diff_mean:0.004854131024330854 - training/rollout_probs_diff_std:0.010198368690907955 - training/rollout_actor_probs_pearson_corr:0.9993425011634827 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6796875 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6796875 - rollout_corr/rollout_is_mean:1.0001150369644165 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.8287475288379937e-05 - rollout_corr/rollout_is_max:1.7099158763885498 - rollout_corr/rollout_is_min:0.4197293221950531 - rollout_corr/rollout_is_std:0.036216843873262405 - rollout_corr/rollout_is_eff_sample_size:0.9986904150765131 - rollout_corr/rollout_is_seq_mean:1.0001122951507568 - rollout_corr/rollout_is_seq_std:0.0019326574401929975 - rollout_corr/rollout_is_seq_max:1.0060887336730957 - rollout_corr/rollout_is_seq_min:0.9933558106422424 - rollout_corr/rollout_is_seq_max_deviation:0.006644189357757568 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7412785273045301) - rollout_corr/training_log_ppl:np.float64(0.5394562017754652) - rollout_corr/kl:np.float64(0.00046429189457519726) - rollout_corr/k3_kl:np.float64(0.0007774811874128318) - rollout_corr/rollout_ppl:np.float64(1.7404227014631033) - rollout_corr/rollout_log_ppl:np.float64(0.538991907320451) - rollout_corr/log_ppl_diff:np.float64(0.0004642944550141692) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017179182032123208) - rollout_corr/log_ppl_diff_max:np.float64(0.007203280925750732) - rollout_corr/log_ppl_diff_min:np.float64(-0.006280422210693359) - rollout_corr/ppl_ratio:np.float64(1.0004668079782277) - rollout_corr/chi2_token:np.float64(0.002199894981458783) - rollout_corr/chi2_seq:np.float64(2.1964850607328117) - actor/pg_loss:np.float64(0.006313061923719943) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00043986083028357825) - actor/ppo_kl:np.float64(-0.00013203885243839864) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(1803.160887064878) - self_distillation/token_reweight_w_std:np.float64(34584.773948686314) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014728792011738) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12900599383283406) - self_distillation/empty_target_batch:np.float64(0.3203125) - actor/grad_norm:np.float64(0.38481806963682175) - perf/mfu/actor:np.float64(0.05734825677685727) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.12204360961914) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005190751980990171 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005190751980990171 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_lengt
(TaskRunner pid=2255397) 
Training Progress:  17%|█▋        | 17/100 [15:18<1:03:22, 45.81s/it]
(TaskRunner pid=2255397) step:18 - global_seqlen/min:17607 - global_seqlen/max:24855 - global_seqlen/minmax_diff:7248 - global_seqlen/balanced_min:20730 - global_seqlen/balanced_max:20735 - global_seqlen/mean:20733.125 - actor/entropy:0.4945189952850342 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30840808153152466 - training/rollout_probs_diff_mean:0.005118089262396097 - training/rollout_probs_diff_std:0.010652771219611168 - training/rollout_actor_probs_pearson_corr:0.9993004202842712 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.49609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:0.9998429417610168 - rollout_corr/rollout_is_ratio_fraction_high:2.0853969544987194e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.032266139984131 - rollout_corr/rollout_is_min:0.5130106210708618 - rollout_corr/rollout_is_std:0.03713827580213547 - rollout_corr/rollout_is_eff_sample_size:0.9986221727100675 - rollout_corr/rollout_is_seq_mean:0.9997930526733398 - rollout_corr/rollout_is_seq_std:0.0022385723423212767 - rollout_corr/rollout_is_seq_max:1.007550835609436 - rollout_corr/rollout_is_seq_min:0.9926072359085083 - rollout_corr/rollout_is_seq_max_deviation:0.007550835609436035 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6632476961240172) - rollout_corr/training_log_ppl:np.float64(0.49761353904614225) - rollout_corr/kl:np.float64(0.0008254678116763614) - rollout_corr/k3_kl:np.float64(0.0008354976650934987) - rollout_corr/rollout_ppl:np.float64(1.6618767492473125) - rollout_corr/rollout_log_ppl:np.float64(0.49678807152668014) - rollout_corr/log_ppl_diff:np.float64(0.0008254675194621086) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019702696008607745) - rollout_corr/log_ppl_diff_max:np.float64(0.008582055568695068) - rollout_corr/log_ppl_diff_min:np.float64(-0.006475538015365601) - rollout_corr/ppl_ratio:np.float64(1.0008287394884974) - rollout_corr/chi2_token:np.float64(0.0016966825351119041) - rollout_corr/chi2_seq:np.float64(0.7993498002178967) - actor/pg_loss:np.float64(0.006876209634356201) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009630748777453846) - actor/ppo_kl:np.float64(-0.00010461421089758005) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.49609375) - self_distillation/token_reweight_w_mean:np.float64(6797.09068774269) - self_distillation/token_reweight_w_std:np.float64(117938.16381233273) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0013849884271622) - self_distillation/token_reweight_w_clip_frac:np.float64(0.17171310502453707) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.4298411011695862) - perf/mfu/actor:np.float64(0.05005069497056926) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.3214340209961) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005354256834834814 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005354256834834814 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_len
(TaskRunner pid=2255397) 
Training Progress:  18%|█▊        | 18/100 [15:58<1:00:16, 44.11s/it]
(TaskRunner pid=2255397) step:19 - global_seqlen/min:16539 - global_seqlen/max:26420 - global_seqlen/minmax_diff:9881 - global_seqlen/balanced_min:21542 - global_seqlen/balanced_max:21548 - global_seqlen/mean:21545.625 - actor/entropy:0.48952940106391907 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21504220366477966 - training/rollout_probs_diff_mean:0.0051950011402368546 - training/rollout_probs_diff_std:0.010906816460192204 - training/rollout_actor_probs_pearson_corr:0.9992679953575134 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.5 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:1.0002551078796387 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.993241310119629 - rollout_corr/rollout_is_min:0.508678138256073 - rollout_corr/rollout_is_std:0.03832463547587395 - rollout_corr/rollout_is_eff_sample_size:0.9985340896470543 - rollout_corr/rollout_is_seq_mean:1.000322699546814 - rollout_corr/rollout_is_seq_std:0.0022897543385624886 - rollout_corr/rollout_is_seq_max:1.007610559463501 - rollout_corr/rollout_is_seq_min:0.9937204718589783 - rollout_corr/rollout_is_seq_max_deviation:0.0076105594635009766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6905396389774978) - rollout_corr/training_log_ppl:np.float64(0.5122466792818159) - rollout_corr/kl:np.float64(0.0005300049446929078) - rollout_corr/k3_kl:np.float64(0.0008489201253496503) - rollout_corr/rollout_ppl:np.float64(1.6896098447032273) - rollout_corr/rollout_log_ppl:np.float64(0.511716675653588) - rollout_corr/log_ppl_diff:np.float64(0.0005300036282278597) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018085126648657024) - rollout_corr/log_ppl_diff_max:np.float64(0.011858522891998291) - rollout_corr/log_ppl_diff_min:np.float64(-0.007662832736968994) - rollout_corr/ppl_ratio:np.float64(1.0005329437553883) - rollout_corr/chi2_token:np.float64(0.00233305711299181) - rollout_corr/chi2_seq:np.float64(0.9715974144637585) - actor/pg_loss:np.float64(0.006131203263066709) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005582695239354507) - actor/ppo_kl:np.float64(7.291653415109067e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5) - self_distillation/token_reweight_w_mean:np.float64(19979.935181184206) - self_distillation/token_reweight_w_std:np.float64(310836.0626574742) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012645116075873) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1186036104336381) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.3470497876405716) - perf/mfu/actor:np.float64(0.051614476906136436) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.35358047485352) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004195125307887793 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004195125307887793 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:371.17578125 - response_length/max:894.0 - response
(TaskRunner pid=2255397) 
Training Progress:  19%|█▉        | 19/100 [16:38<57:52, 42.87s/it]  
(TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20}
(TaskRunner pid=2255397) validation generation end
(TaskRunner pid=2255397) [prompt] system
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2255397) <reasoning>
(TaskRunner pid=2255397) ...
(TaskRunner pid=2255397) </reasoning>
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) ...
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) A
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) user
(TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2255397) Please reason step by step.
(TaskRunner pid=2255397) assistant
(TaskRunner pid=2255397) <think>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) </think>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) [response] <reasoning>
(TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"**, we must identify the structure of the product and match it with the most appropriate reactant that would lead to its formation through a plausible synthetic pathway.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) The product has the following key structural features:
(TaskRunner pid=2255397) - A **nitrogen-containing ring** (likely a pyrrole or imidazole-like structure).
(TaskRunner pid=2255397) - A **chlorinated aromatic ring** (e.g., **Cl-phenyl**).
(TaskRunner pid=2255397) - A **trifluoromethyl group** (C(F)(F)F).
(TaskRunner pid=2255397) - A **nitrogen atom** bonded to a carbon chain that includes the **Cl-phenyl** and **trifluoromethyl** groups.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Now, let’s examine the options:
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option A**:  
(TaskRunner pid=2255397) `O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1`  
(TaskRunner pid=2255397) This structure contains a **nitrogen ring** with the same **Cl-phenyl** and **trifluoromethyl** substituents as the product. The **carbonyl groups** and **nitrogen ring** suggest a possible **amide** or **nitrile** structure that could be a **precursor** to the product. It is the **most structurally similar** to the product and would likely be the **correct reactant** in a synthesis involving ring formation or substitution.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option B**:  
(TaskRunner pid=2255397) `O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1`  
(TaskRunner pid=2255397) This structure contains **two chlorines** and a **nitrogen ring**, but it lacks the **trifluoromethyl group** and the **correct arrangement of substituents** as in the product. It is **not a good match**.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option C**:  
(TaskRunner pid=2255397) `O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1`  
(TaskRunner pid=2255397) This structure includes a **trifluoromethyl group** and **Cl-phenyl**, but the **nitrogen ring** is **not directly bonded to the Cl-phenyl** or **trifluoromethyl**, and the **structure is more complex**. It might be a **byproduct** or **intermediate**, but not the **direct reactant**.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option D**:  
(TaskRunner pid=2255397) `O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F`  
(TaskRunner pid=2255397) This structure contains a **trifluoromethyl group**, but it **lacks the nitrogen ring** and the **Cl-phenyl** substituents. It is **structurally different** from the product.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Conclusion**:  
(TaskRunner pid=2255397) Only **Option A** contains the **correct substituents** (Cl-phenyl, trifluoromethyl, and a nitrogen ring), and the **structure is most consistent** with the product. It is the **only plausible reactant** that would lead to the synthesis of the given product.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) </reasoning>
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) A
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) [ground_truth] A
(TaskRunner pid=2255397) [score] 1.0
(TaskRunner pid=2255397) [acc] 1.0
(TaskRunner pid=2255397) [pred] A
(TaskRunner pid=2255397) [incorrect_format] 1
(TaskRunner pid=2255397) [feedback] 
(TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20
(TaskRunner pid=2255397) step:20 - global_seqlen/min:18853 - global_seqlen/max:23232 - global_seqlen/minmax_diff:4379 - global_seqlen/balanced_min:20703 - global_seqlen/balanced_max:20714 - global_seqlen/mean:20709.5 - actor/entropy:0.4852030277252197 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3635062277317047 - training/rollout_probs_diff_mean:0.0052037229761481285 - training/rollout_probs_diff_std:0.010991442948579788 - training/rollout_actor_probs_pearson_corr:0.9992546439170837 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.0001426935195923 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.1487816411536187e-05 - rollout_corr/rollout_is_max:1.6168099641799927 - rollout_corr/rollout_is_min:0.4058627188205719 - rollout_corr/rollout_is_std:0.03768158331513405 - rollout_corr/rollout_is_eff_sample_size:0.9985827057725277 - rollout_corr/rollout_is_seq_mean:1.0002155303955078 - rollout_corr/rollout_is_seq_std:0.002177149523049593 - rollout_corr/rollout_is_seq_max:1.0087448358535767 - rollout_corr/rollout_is_seq_min:0.9949045181274414 - rollout_corr/rollout_is_seq_max_deviation:0.00874483585357666 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.65907796099782) - rollout_corr/training_log_ppl:np.float64(0.49622454700875096) - rollout_corr/kl:np.float64(0.0007081711968908166) - rollout_corr/k3_kl:np.float64(0.000836590931498904) - rollout_corr/rollout_ppl:np.float64(1.6578862462192774) - rollout_corr/rollout_log_ppl:np.float64(0.49551637563854456) - rollout_corr/log_ppl_diff:np.float64(0.0007081713702064008) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018356915388721973) - rollout_corr/log_ppl_diff_max:np.float64(0.007012069225311279) - rollout_corr/log_ppl_diff_min:np.float64(-0.006520330905914307) - rollout_corr/ppl_ratio:np.float64(1.0007108724676073) - rollout_corr/chi2_token:np.float64(0.001943168230354786) - rollout_corr/chi2_seq:np.float64(0.7969548560213298) - actor/pg_loss:np.float64(0.006117190117947757) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006000254938953731) - actor/ppo_kl:np.float64(0.0001683622528645401) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(23049.971880125348) - self_distillation/token_reweight_w_std:np.float64(317665.6890155331) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9982667530421168) - self_distillation/token_reweight_w_clip_frac:np.float64(0.15381890590651892) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.3626445010304451) - perf/mfu/actor:np.float64(0.05021794625693002) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.33641052246094) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.5196428571428572) - val-aux/sciknoweval/reward/std@16:np.float64(0.2570318204515784) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6278380952380953) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.21727863221866373) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.41209523809523807) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2203367172556475) - val-aux/
(TaskRunner pid=2255397) 
Training Progress:  20%|██        | 20/100 [19:44<1:54:20, 85.76s/it]
(TaskRunner pid=2255397) step:21 - global_seqlen/min:17376 - global_seqlen/max:26208 - global_seqlen/minmax_diff:8832 - global_seqlen/balanced_min:22075 - global_seqlen/balanced_max:22088 - global_seqlen/mean:22085.25 - actor/entropy:0.4656538665294647 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102373421192169 - training/rollout_probs_diff_mean:0.004885641857981682 - training/rollout_probs_diff_std:0.010877853259444237 - training/rollout_actor_probs_pearson_corr:0.9992596507072449 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.84765625 - self_distillation/correct_sample_fraction:0.43359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84765625 - rollout_corr/rollout_is_mean:0.9998842477798462 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.879465500882361e-05 - rollout_corr/rollout_is_max:1.9200012683868408 - rollout_corr/rollout_is_min:0.25445759296417236 - rollout_corr/rollout_is_std:0.03666176274418831 - rollout_corr/rollout_is_eff_sample_size:0.9986574816202841 - rollout_corr/rollout_is_seq_mean:0.9998617172241211 - rollout_corr/rollout_is_seq_std:0.0022321075666695833 - rollout_corr/rollout_is_seq_max:1.0074141025543213 - rollout_corr/rollout_is_seq_min:0.9920339584350586 - rollout_corr/rollout_is_seq_max_deviation:0.007966041564941406 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6367876464501023) - rollout_corr/training_log_ppl:np.float64(0.47835954203037545) - rollout_corr/kl:np.float64(0.0009155909243130367) - rollout_corr/k3_kl:np.float64(0.0008391680780164279) - rollout_corr/rollout_ppl:np.float64(1.6352251395583153) - rollout_corr/rollout_log_ppl:np.float64(0.4774439476314001) - rollout_corr/log_ppl_diff:np.float64(0.0009155943989753723) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018019898561760783) - rollout_corr/log_ppl_diff_max:np.float64(0.008913397789001465) - rollout_corr/log_ppl_diff_min:np.float64(-0.006456807255744934) - rollout_corr/ppl_ratio:np.float64(1.0009184170048684) - rollout_corr/chi2_token:np.float64(0.001522636041045189) - rollout_corr/chi2_seq:np.float64(0.3302574837580323) - actor/pg_loss:np.float64(0.00977602880448103) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008731294096833153) - actor/ppo_kl:np.float64(2.3401729779659597e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84765625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84765625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.43359375) - self_distillation/token_reweight_w_mean:np.float64(19834.05608875933) - self_distillation/token_reweight_w_std:np.float64(292799.855937861) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0034907914232463) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1998413290712051) - self_distillation/empty_target_batch:np.float64(0.15234375) - actor/grad_norm:np.float64(0.506427951157093) - perf/mfu/actor:np.float64(0.05293222627939937) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.2050437927246) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.43359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.43359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00826286617666483 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00826286617666483 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:4
(TaskRunner pid=2255397) 
Training Progress:  21%|██        | 21/100 [20:25<1:35:16, 72.35s/it]
(TaskRunner pid=2255397) step:22 - global_seqlen/min:15780 - global_seqlen/max:20714 - global_seqlen/minmax_diff:4934 - global_seqlen/balanced_min:18793 - global_seqlen/balanced_max:18799 - global_seqlen/mean:18797.25 - actor/entropy:0.4407426416873932 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23966830968856812 - training/rollout_probs_diff_mean:0.0051012844778597355 - training/rollout_probs_diff_std:0.011158647015690804 - training/rollout_actor_probs_pearson_corr:0.9991984963417053 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6796875 - self_distillation/correct_sample_fraction:0.4375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6796875 - rollout_corr/rollout_is_mean:0.9999550580978394 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.586863398551941 - rollout_corr/rollout_is_min:0.6212320327758789 - rollout_corr/rollout_is_std:0.03741012141108513 - rollout_corr/rollout_is_eff_sample_size:0.9986023199750391 - rollout_corr/rollout_is_seq_mean:0.9999154806137085 - rollout_corr/rollout_is_seq_std:0.0023052047472447157 - rollout_corr/rollout_is_seq_max:1.0075262784957886 - rollout_corr/rollout_is_seq_min:0.9921414256095886 - rollout_corr/rollout_is_seq_max_deviation:0.007858574390411377 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6121524260379374) - rollout_corr/training_log_ppl:np.float64(0.46336599756614305) - rollout_corr/kl:np.float64(0.0012106716192334943) - rollout_corr/k3_kl:np.float64(0.001044533465915265) - rollout_corr/rollout_ppl:np.float64(1.6101201642304659) - rollout_corr/rollout_log_ppl:np.float64(0.4621553277829662) - rollout_corr/log_ppl_diff:np.float64(0.0012106697831768543) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022202231630217284) - rollout_corr/log_ppl_diff_max:np.float64(0.011315524578094482) - rollout_corr/log_ppl_diff_min:np.float64(-0.007631242275238037) - rollout_corr/ppl_ratio:np.float64(1.0012149058748037) - rollout_corr/chi2_token:np.float64(0.001704103546217084) - rollout_corr/chi2_seq:np.float64(0.4757801443338394) - actor/pg_loss:np.float64(0.006979008438065648) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011709280256582133) - actor/ppo_kl:np.float64(0.00036738665134805615) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4375) - self_distillation/token_reweight_w_mean:np.float64(53386.58389568445) - self_distillation/token_reweight_w_std:np.float64(762098.1771182105) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011509482283145) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1306488252885174) - self_distillation/empty_target_batch:np.float64(0.3203125) - actor/grad_norm:np.float64(0.4595351442694664) - perf/mfu/actor:np.float64(0.0457439323763552) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.23744583129883) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.4375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0024807543959468603 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0024807543959468603 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:331.8515625 - response_length/max
(TaskRunner pid=2255397) 
Training Progress:  22%|██▏       | 22/100 [21:05<1:21:20, 62.57s/it]
(TaskRunner pid=2255397) step:23 - global_seqlen/min:15655 - global_seqlen/max:22837 - global_seqlen/minmax_diff:7182 - global_seqlen/balanced_min:19794 - global_seqlen/balanced_max:19804 - global_seqlen/mean:19802.0 - actor/entropy:0.43892350792884827 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22363799810409546 - training/rollout_probs_diff_mean:0.005129741504788399 - training/rollout_probs_diff_std:0.011249431408941746 - training/rollout_actor_probs_pearson_corr:0.9991777539253235 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9296875 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9296875 - rollout_corr/rollout_is_mean:0.9999939203262329 - rollout_corr/rollout_is_ratio_fraction_high:1.1452129911049269e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.1670188903808594 - rollout_corr/rollout_is_min:0.5688050389289856 - rollout_corr/rollout_is_std:0.03843645006418228 - rollout_corr/rollout_is_eff_sample_size:0.9985248186495871 - rollout_corr/rollout_is_seq_mean:0.9999755024909973 - rollout_corr/rollout_is_seq_std:0.0022403132170438766 - rollout_corr/rollout_is_seq_max:1.008310079574585 - rollout_corr/rollout_is_seq_min:0.9935483336448669 - rollout_corr/rollout_is_seq_max_deviation:0.008310079574584961 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.585592514835298) - rollout_corr/training_log_ppl:np.float64(0.45206161157693714) - rollout_corr/kl:np.float64(0.0010271025659691624) - rollout_corr/k3_kl:np.float64(0.0009754648472721783) - rollout_corr/rollout_ppl:np.float64(1.583907154854387) - rollout_corr/rollout_log_ppl:np.float64(0.45103450742317364) - rollout_corr/log_ppl_diff:np.float64(0.0010271041537635028) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021158340969122946) - rollout_corr/log_ppl_diff_max:np.float64(0.008697479963302612) - rollout_corr/log_ppl_diff_min:np.float64(-0.007354050874710083) - rollout_corr/ppl_ratio:np.float64(1.0010308369528502) - rollout_corr/chi2_token:np.float64(0.0018195987213402987) - rollout_corr/chi2_seq:np.float64(1.1621388338971883) - actor/pg_loss:np.float64(0.007446382311172783) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000959160078309651) - actor/ppo_kl:np.float64(0.00021888582077522756) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(6609.447738413466) - self_distillation/token_reweight_w_std:np.float64(122546.46291852725) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9963779090903699) - self_distillation/token_reweight_w_clip_frac:np.float64(0.17003195008146577) - self_distillation/empty_target_batch:np.float64(0.0703125) - actor/grad_norm:np.float64(0.43443264812231064) - perf/mfu/actor:np.float64(0.04804480134893495) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.1064910888672) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:0 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0033225491642951965 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0033225491642951965 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/m
(TaskRunner pid=2255397) 
Training Progress:  23%|██▎       | 23/100 [21:44<1:11:10, 55.46s/it]
(TaskRunner pid=2255397) step:24 - global_seqlen/min:18708 - global_seqlen/max:22807 - global_seqlen/minmax_diff:4099 - global_seqlen/balanced_min:20352 - global_seqlen/balanced_max:20355 - global_seqlen/mean:20353.75 - actor/entropy:0.48541581630706787 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.16695591807365417 - training/rollout_probs_diff_mean:0.005245619919151068 - training/rollout_probs_diff_std:0.01105139683932066 - training/rollout_actor_probs_pearson_corr:0.9992536306381226 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.82421875 - self_distillation/correct_sample_fraction:0.58984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.82421875 - rollout_corr/rollout_is_mean:0.9997487664222717 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6379640102386475 - rollout_corr/rollout_is_min:0.55048668384552 - rollout_corr/rollout_is_std:0.038326967507600784 - rollout_corr/rollout_is_eff_sample_size:0.998532425609191 - rollout_corr/rollout_is_seq_mean:0.9996849894523621 - rollout_corr/rollout_is_seq_std:0.0021940250881016254 - rollout_corr/rollout_is_seq_max:1.0061367750167847 - rollout_corr/rollout_is_seq_min:0.9898834228515625 - rollout_corr/rollout_is_seq_max_deviation:0.0101165771484375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6475044381804764) - rollout_corr/training_log_ppl:np.float64(0.4905120711773634) - rollout_corr/kl:np.float64(0.0011423102059779922) - rollout_corr/k3_kl:np.float64(0.0009413222521743592) - rollout_corr/rollout_ppl:np.float64(1.6456106547266245) - rollout_corr/rollout_log_ppl:np.float64(0.48936975869582966) - rollout_corr/log_ppl_diff:np.float64(0.001142312481533736) - rollout_corr/log_ppl_abs_diff:np.float64(0.002205758646596223) - rollout_corr/log_ppl_diff_max:np.float64(0.011655747890472412) - rollout_corr/log_ppl_diff_min:np.float64(-0.006167978048324585) - rollout_corr/ppl_ratio:np.float64(1.0011465374846011) - rollout_corr/chi2_token:np.float64(0.001514944015070796) - rollout_corr/chi2_seq:np.float64(0.9692259337753057) - actor/pg_loss:np.float64(0.006828168756328523) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008421741476922762) - actor/ppo_kl:np.float64(5.722692924337025e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.82421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.82421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_w_mean:np.float64(16169.235121513018) - self_distillation/token_reweight_w_std:np.float64(204586.64157430374) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000086387153715) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14809982472797856) - self_distillation/empty_target_batch:np.float64(0.17578125) - actor/grad_norm:np.float64(0.38066764548420906) - perf/mfu/actor:np.float64(0.04935736118807353) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.85000133514404) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:0 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-8.236870780820027e-05 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-8.236870780820027e-05 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:355.6796875 -
(TaskRunner pid=2255397) 
Training Progress:  24%|██▍       | 24/100 [22:22<1:03:39, 50.25s/it]
(TaskRunner pid=2255397) step:25 - global_seqlen/min:16381 - global_seqlen/max:25188 - global_seqlen/minmax_diff:8807 - global_seqlen/balanced_min:21088 - global_seqlen/balanced_max:21097 - global_seqlen/mean:21093.125 - actor/entropy:0.4773712158203125 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26223766803741455 - training/rollout_probs_diff_mean:0.005165540147572756 - training/rollout_probs_diff_std:0.01117149367928505 - training/rollout_actor_probs_pearson_corr:0.9992371797561646 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.5546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:0.9999747276306152 - rollout_corr/rollout_is_ratio_fraction_high:1.0959985047520604e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.094183921813965 - rollout_corr/rollout_is_min:0.5062851309776306 - rollout_corr/rollout_is_std:0.038087502121925354 - rollout_corr/rollout_is_eff_sample_size:0.9985513246511216 - rollout_corr/rollout_is_seq_mean:1.0000536441802979 - rollout_corr/rollout_is_seq_std:0.0022864702623337507 - rollout_corr/rollout_is_seq_max:1.0067065954208374 - rollout_corr/rollout_is_seq_min:0.9937045574188232 - rollout_corr/rollout_is_seq_max_deviation:0.006706595420837402 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.669403042178601) - rollout_corr/training_log_ppl:np.float64(0.4993552331288811) - rollout_corr/kl:np.float64(0.0006479631488460758) - rollout_corr/k3_kl:np.float64(0.0008453664596572708) - rollout_corr/rollout_ppl:np.float64(1.668324178084731) - rollout_corr/rollout_log_ppl:np.float64(0.4987072692310903) - rollout_corr/log_ppl_diff:np.float64(0.0006479638977907598) - rollout_corr/log_ppl_abs_diff:np.float64(0.001830984663683921) - rollout_corr/log_ppl_diff_max:np.float64(0.008455544710159302) - rollout_corr/log_ppl_diff_min:np.float64(-0.007198035717010498) - rollout_corr/ppl_ratio:np.float64(1.0006508629303426) - rollout_corr/chi2_token:np.float64(0.002105830702930689) - rollout_corr/chi2_seq:np.float64(0.6466955123469234) - actor/pg_loss:np.float64(0.007964683580212295) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000689070585394802) - actor/ppo_kl:np.float64(-6.814886182837654e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5546875) - self_distillation/token_reweight_w_mean:np.float64(26990.30881792493) - self_distillation/token_reweight_w_std:np.float64(421488.5579388592) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.002008835086599) - self_distillation/token_reweight_w_clip_frac:np.float64(0.15447656423202716) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.4783755913376808) - perf/mfu/actor:np.float64(0.0494590320538676) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.90256881713867) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:0 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02120209112763405 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02120209112763405 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:356.41
(TaskRunner pid=2255397) 
Training Progress:  25%|██▌       | 25/100 [23:03<59:14, 47.40s/it]  
(TaskRunner pid=2255397) step:26 - global_seqlen/min:18171 - global_seqlen/max:23062 - global_seqlen/minmax_diff:4891 - global_seqlen/balanced_min:20385 - global_seqlen/balanced_max:20394 - global_seqlen/mean:20391.25 - actor/entropy:0.5068786144256592 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17874199151992798 - training/rollout_probs_diff_mean:0.005472649820148945 - training/rollout_probs_diff_std:0.011403916403651237 - training/rollout_actor_probs_pearson_corr:0.9991965889930725 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.6328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:1.0000735521316528 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.292263525305316e-05 - rollout_corr/rollout_is_max:1.8139365911483765 - rollout_corr/rollout_is_min:0.43659546971321106 - rollout_corr/rollout_is_std:0.039478376507759094 - rollout_corr/rollout_is_eff_sample_size:0.9984440018610561 - rollout_corr/rollout_is_seq_mean:1.0000500679016113 - rollout_corr/rollout_is_seq_std:0.0023560679983347654 - rollout_corr/rollout_is_seq_max:1.0076462030410767 - rollout_corr/rollout_is_seq_min:0.9908225536346436 - rollout_corr/rollout_is_seq_max_deviation:0.009177446365356445 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6881171250715852) - rollout_corr/training_log_ppl:np.float64(0.510973722091876) - rollout_corr/kl:np.float64(0.0007455622756609159) - rollout_corr/k3_kl:np.float64(0.0008864333188967066) - rollout_corr/rollout_ppl:np.float64(1.6867859680205584) - rollout_corr/rollout_log_ppl:np.float64(0.5102281617000699) - rollout_corr/log_ppl_diff:np.float64(0.000745560391806066) - rollout_corr/log_ppl_abs_diff:np.float64(0.002050930866971612) - rollout_corr/log_ppl_diff_max:np.float64(0.00910806655883789) - rollout_corr/log_ppl_diff_min:np.float64(-0.006127357482910156) - rollout_corr/ppl_ratio:np.float64(1.0007490692660213) - rollout_corr/chi2_token:np.float64(0.0020302317570894957) - rollout_corr/chi2_seq:np.float64(1.0315312510356307) - actor/pg_loss:np.float64(0.005001060431823134) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00048566531631877297) - actor/ppo_kl:np.float64(-2.1595971896104516e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6328125) - self_distillation/token_reweight_w_mean:np.float64(13624.06982974615) - self_distillation/token_reweight_w_std:np.float64(206433.80913565995) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9992130598984659) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09108548518270254) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.3169581815600395) - perf/mfu/actor:np.float64(0.04962560549786612) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.88824081420898) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:0 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0027206302620470524 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0027206302620470524 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/m
(TaskRunner pid=2255397) 
Training Progress:  26%|██▌       | 26/100 [23:40<54:55, 44.53s/it]
(TaskRunner pid=2255397) step:27 - global_seqlen/min:17480 - global_seqlen/max:23456 - global_seqlen/minmax_diff:5976 - global_seqlen/balanced_min:20067 - global_seqlen/balanced_max:20076 - global_seqlen/mean:20073.875 - actor/entropy:0.4675213694572449 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105892539024353 - training/rollout_probs_diff_mean:0.005088143516331911 - training/rollout_probs_diff_std:0.010994603857398033 - training/rollout_actor_probs_pearson_corr:0.9992254972457886 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.85546875 - self_distillation/correct_sample_fraction:0.51171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.85546875 - rollout_corr/rollout_is_mean:0.9997977018356323 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6167669296264648 - rollout_corr/rollout_is_min:0.5378821492195129 - rollout_corr/rollout_is_std:0.03790003061294556 - rollout_corr/rollout_is_eff_sample_size:0.9985651130827546 - rollout_corr/rollout_is_seq_mean:0.9997107982635498 - rollout_corr/rollout_is_seq_std:0.002149628708139062 - rollout_corr/rollout_is_seq_max:1.007951021194458 - rollout_corr/rollout_is_seq_min:0.9923518896102905 - rollout_corr/rollout_is_seq_max_deviation:0.007951021194458008 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6248419084586203) - rollout_corr/training_log_ppl:np.float64(0.47598731669131666) - rollout_corr/kl:np.float64(0.0010493483772009427) - rollout_corr/k3_kl:np.float64(0.0008248598063573809) - rollout_corr/rollout_ppl:np.float64(1.623152432963252) - rollout_corr/rollout_log_ppl:np.float64(0.47493796655908227) - rollout_corr/log_ppl_diff:np.float64(0.0010493501322343946) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018501168815419078) - rollout_corr/log_ppl_diff_max:np.float64(0.007712453603744507) - rollout_corr/log_ppl_diff_min:np.float64(-0.004121840000152588) - rollout_corr/ppl_ratio:np.float64(1.0010521123185754) - rollout_corr/chi2_token:np.float64(0.0011714110150933266) - rollout_corr/chi2_seq:np.float64(0.32961429841816425) - actor/pg_loss:np.float64(0.008322581415995955) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008014981349333539) - actor/ppo_kl:np.float64(4.5028834745330926e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.85546875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.85546875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.51171875) - self_distillation/token_reweight_w_mean:np.float64(30577.34729726822) - self_distillation/token_reweight_w_std:np.float64(521484.4462188269) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0020933467894793) - self_distillation/token_reweight_w_clip_frac:np.float64(0.16031731336261146) - self_distillation/empty_target_batch:np.float64(0.14453125) - actor/grad_norm:np.float64(0.42759493738412857) - perf/mfu/actor:np.float64(0.048367516590069666) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.80517578125) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:0 - critic/score/mean:0.51171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0014940870460122824 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0014940870460122824 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:369.62109375
(TaskRunner pid=2255397) 
Training Progress:  27%|██▋       | 27/100 [24:21<52:33, 43.20s/it]
(TaskRunner pid=2255397) step:28 - global_seqlen/min:18201 - global_seqlen/max:23560 - global_seqlen/minmax_diff:5359 - global_seqlen/balanced_min:20410 - global_seqlen/balanced_max:20497 - global_seqlen/mean:20425.125 - actor/entropy:0.4737909138202667 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2294958233833313 - training/rollout_probs_diff_mean:0.0052576144225895405 - training/rollout_probs_diff_std:0.011254720389842987 - training/rollout_actor_probs_pearson_corr:0.999194860458374 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0001050233840942 - rollout_corr/rollout_is_ratio_fraction_high:1.0676346391846891e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.8465442657470703 - rollout_corr/rollout_is_min:0.6126700639724731 - rollout_corr/rollout_is_std:0.03862826153635979 - rollout_corr/rollout_is_eff_sample_size:0.9985103182038877 - rollout_corr/rollout_is_seq_mean:1.0001041889190674 - rollout_corr/rollout_is_seq_std:0.002421688986942172 - rollout_corr/rollout_is_seq_max:1.00888991355896 - rollout_corr/rollout_is_seq_min:0.9913090467453003 - rollout_corr/rollout_is_seq_max_deviation:0.008889913558959961 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6370238508097827) - rollout_corr/training_log_ppl:np.float64(0.4820473712461535) - rollout_corr/kl:np.float64(0.0006596640439546775) - rollout_corr/k3_kl:np.float64(0.000909588395757055) - rollout_corr/rollout_ppl:np.float64(1.6358301592990756) - rollout_corr/rollout_log_ppl:np.float64(0.48138770982041024) - rollout_corr/log_ppl_diff:np.float64(0.000659661425743252) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018744217813946307) - rollout_corr/log_ppl_diff_max:np.float64(0.007977485656738281) - rollout_corr/log_ppl_diff_min:np.float64(-0.0074624717235565186) - rollout_corr/ppl_ratio:np.float64(1.0006625556852669) - rollout_corr/chi2_token:np.float64(0.0023598361294716597) - rollout_corr/chi2_seq:np.float64(0.9705478034447879) - actor/pg_loss:np.float64(0.006433643866330385) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006121892383816885) - actor/ppo_kl:np.float64(-2.0443672603676077e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(22090.04282637965) - self_distillation/token_reweight_w_std:np.float64(363460.48439942) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007038170006126) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12272839152137749) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.4047693833708763) - perf/mfu/actor:np.float64(0.0494901405269842) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.81372833251953) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:0 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004289222415536642 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004289222415536642 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/m
(TaskRunner pid=2255397) 
Training Progress:  28%|██▊       | 28/100 [25:01<50:46, 42.31s/it]
(TaskRunner pid=2255397) step:29 - global_seqlen/min:17728 - global_seqlen/max:24689 - global_seqlen/minmax_diff:6961 - global_seqlen/balanced_min:21875 - global_seqlen/balanced_max:21882 - global_seqlen/mean:21878.875 - actor/entropy:0.45051109790802 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25539880990982056 - training/rollout_probs_diff_mean:0.004729867912828922 - training/rollout_probs_diff_std:0.010498874820768833 - training/rollout_actor_probs_pearson_corr:0.9993234872817993 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.48046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:0.9999346733093262 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9196990251657553e-05 - rollout_corr/rollout_is_max:1.6618187427520752 - rollout_corr/rollout_is_min:0.48319488763809204 - rollout_corr/rollout_is_std:0.03575139120221138 - rollout_corr/rollout_is_eff_sample_size:0.998723350704955 - rollout_corr/rollout_is_seq_mean:0.9999228119850159 - rollout_corr/rollout_is_seq_std:0.00180347659625113 - rollout_corr/rollout_is_seq_max:1.0053354501724243 - rollout_corr/rollout_is_seq_min:0.9929618835449219 - rollout_corr/rollout_is_seq_max_deviation:0.007038116455078125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5977162146009505) - rollout_corr/training_log_ppl:np.float64(0.4593473710701801) - rollout_corr/kl:np.float64(0.0006978295002575408) - rollout_corr/k3_kl:np.float64(0.0007481996697720206) - rollout_corr/rollout_ppl:np.float64(1.5965832527726889) - rollout_corr/rollout_log_ppl:np.float64(0.45864954427815974) - rollout_corr/log_ppl_diff:np.float64(0.0006978267920203507) - rollout_corr/log_ppl_abs_diff:np.float64(0.001622283656615764) - rollout_corr/log_ppl_diff_max:np.float64(0.006626904010772705) - rollout_corr/log_ppl_diff_min:np.float64(-0.00486445426940918) - rollout_corr/ppl_ratio:np.float64(1.0007000237237662) - rollout_corr/chi2_token:np.float64(0.001576458802446723) - rollout_corr/chi2_seq:np.float64(0.6520844814367592) - actor/pg_loss:np.float64(0.0060147366020828485) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005360728382584057) - actor/ppo_kl:np.float64(-6.641209190716424e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48046875) - self_distillation/token_reweight_w_mean:np.float64(26070.862267469056) - self_distillation/token_reweight_w_std:np.float64(478879.7805042105) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0025979080237448) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1094071113911923) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.3129398711025715) - perf/mfu/actor:np.float64(0.052564897191348076) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.9274024963379) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:0 - critic/score/mean:0.48046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01068192534148693 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01068192534148693 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:406.96484375
(TaskRunner pid=2255397) 
Training Progress:  29%|██▉       | 29/100 [25:41<49:19, 41.68s/it]
(TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30}
(TaskRunner pid=2255397) validation generation end
(TaskRunner pid=2255397) [prompt] system
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2255397) <reasoning>
(TaskRunner pid=2255397) ...
(TaskRunner pid=2255397) </reasoning>
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) ...
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) A
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) user
(TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2255397) Please reason step by step.
(TaskRunner pid=2255397) assistant
(TaskRunner pid=2255397) <think>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) </think>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) [response] <reasoning>
(TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to analyze the structure of the product and identify the key functional groups and substituents that must be present in the reactant.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) The product has the following key features:
(TaskRunner pid=2255397) - A nitrogen-containing ring (likely a pyrrole or similar heterocycle).
(TaskRunner pid=2255397) - A benzene ring with a chlorine substituent.
(TaskRunner pid=2255397) - A trifluoromethyl group (C(F)(F)F) attached to the benzene ring.
(TaskRunner pid=2255397) - A substituent group (likely a nitrogen-containing chain) that connects to the ring.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Now, let's examine each option:
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option A:**  
(TaskRunner pid=2255397) O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1  
(TaskRunner pid=2255397) This option contains the trifluoromethyl group (C(F)(F)F) and the chlorinated benzene ring. It also contains a nitrogen-containing chain and a ring structure similar to the product. This option closely matches the structure of the product and is a strong candidate.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option B:**  
(TaskRunner pid=2255397) O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1  
(TaskRunner pid=2255397) This option has a chlorine substituent on the ring, but it lacks the trifluoromethyl group and the correct nitrogen-containing chain. It also has a different ring structure than the product. Therefore, it is not the correct reactant.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option C:**  
(TaskRunner pid=2255397) O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1  
(TaskRunner pid=2255397) This option contains the trifluoromethyl group and the chlorinated benzene ring. However, the structure is more complex and includes an oxygen-containing side chain that is not present in the product. It introduces an unnecessary oxygen group, making it incompatible with the product structure.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option D:**  
(TaskRunner pid=2255397) O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F  
(TaskRunner pid=2255397) This option contains the trifluoromethyl group and a benzene ring. However, it lacks the correct nitrogen-containing chain and the chlorinated benzene ring. The structure is not aligned with the product, so it is not the correct reactant.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Conclusion:**  
(TaskRunner pid=2255397) Only **Option A** contains all the necessary functional groups and substituents present in the product, making it the only correct reactant for the synthesis of the given product.
(TaskRunner pid=2255397) </reasoning>
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) A
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) [ground_truth] A
(TaskRunner pid=2255397) [score] 1.0
(TaskRunner pid=2255397) [acc] 1.0
(TaskRunner pid=2255397) [pred] A
(TaskRunner pid=2255397) [incorrect_format] 1
(TaskRunner pid=2255397) [feedback] 
(TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30
(TaskRunner pid=2255397) step:30 - global_seqlen/min:18031 - global_seqlen/max:24561 - global_seqlen/minmax_diff:6530 - global_seqlen/balanced_min:20880 - global_seqlen/balanced_max:20888 - global_seqlen/mean:20884.625 - actor/entropy:0.4658919870853424 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3171748220920563 - training/rollout_probs_diff_mean:0.005262547638267279 - training/rollout_probs_diff_std:0.01141402218490839 - training/rollout_actor_probs_pearson_corr:0.9991620779037476 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.66796875 - self_distillation/correct_sample_fraction:0.44921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.66796875 - rollout_corr/rollout_is_mean:0.9999656677246094 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.412619844311848e-05 - rollout_corr/rollout_is_max:1.9128998517990112 - rollout_corr/rollout_is_min:0.2724512219429016 - rollout_corr/rollout_is_std:0.03873920068144798 - rollout_corr/rollout_is_eff_sample_size:0.9985014042044956 - rollout_corr/rollout_is_seq_mean:1.0000505447387695 - rollout_corr/rollout_is_seq_std:0.002504864940419793 - rollout_corr/rollout_is_seq_max:1.0067648887634277 - rollout_corr/rollout_is_seq_min:0.9914875626564026 - rollout_corr/rollout_is_seq_max_deviation:0.008512437343597412 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6606428353115916) - rollout_corr/training_log_ppl:np.float64(0.49546970275696367) - rollout_corr/kl:np.float64(0.0007864780202773147) - rollout_corr/k3_kl:np.float64(0.000911510776063551) - rollout_corr/rollout_ppl:np.float64(1.659348614513874) - rollout_corr/rollout_log_ppl:np.float64(0.4946832267160062) - rollout_corr/log_ppl_diff:np.float64(0.0007864760409574956) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019896051089745015) - rollout_corr/log_ppl_diff_max:np.float64(0.009873658418655396) - rollout_corr/log_ppl_diff_min:np.float64(-0.007493495941162109) - rollout_corr/ppl_ratio:np.float64(1.0007900123018771) - rollout_corr/chi2_token:np.float64(0.002100849524140358) - rollout_corr/chi2_seq:np.float64(0.6980828407686204) - actor/pg_loss:np.float64(0.0044937689090147614) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004526598731899867) - actor/ppo_kl:np.float64(2.1002829232230624e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.66796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.44921875) - self_distillation/token_reweight_w_mean:np.float64(16658.25212058681) - self_distillation/token_reweight_w_std:np.float64(262596.2304097906) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.002752567641437) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09863085992401466) - self_distillation/empty_target_batch:np.float64(0.33203125) - actor/grad_norm:np.float64(0.3747030645608902) - perf/mfu/actor:np.float64(0.050935440136856504) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.77979278564453) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.606845238095238) - val-aux/sciknoweval/reward/std@16:np.float64(0.2138942013523591) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6934285714285715) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.16904326809342193) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.521195238095238) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1926339687939083) - val-aux
(TaskRunner pid=2255397) 
Training Progress:  30%|███       | 30/100 [27:53<1:20:12, 68.76s/it]
(TaskRunner pid=2255397) step:31 - global_seqlen/min:19249 - global_seqlen/max:25029 - global_seqlen/minmax_diff:5780 - global_seqlen/balanced_min:21607 - global_seqlen/balanced_max:21619 - global_seqlen/mean:21615.375 - actor/entropy:0.4569137990474701 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.303112655878067 - training/rollout_probs_diff_mean:0.004967319313436747 - training/rollout_probs_diff_std:0.010901066474616528 - training/rollout_actor_probs_pearson_corr:0.9992502331733704 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.6015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.000017762184143 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.816590428352356 - rollout_corr/rollout_is_min:0.5354147553443909 - rollout_corr/rollout_is_std:0.03712382912635803 - rollout_corr/rollout_is_eff_sample_size:0.9986238370470686 - rollout_corr/rollout_is_seq_mean:1.0000417232513428 - rollout_corr/rollout_is_seq_std:0.002106655156239867 - rollout_corr/rollout_is_seq_max:1.0081474781036377 - rollout_corr/rollout_is_seq_min:0.9948439002037048 - rollout_corr/rollout_is_seq_max_deviation:0.008147478103637695 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6235803924500942) - rollout_corr/training_log_ppl:np.float64(0.4749105938244611) - rollout_corr/kl:np.float64(0.0008353461136003659) - rollout_corr/k3_kl:np.float64(0.00080985213742224) - rollout_corr/rollout_ppl:np.float64(1.6222249972634017) - rollout_corr/rollout_log_ppl:np.float64(0.4740752446523402) - rollout_corr/log_ppl_diff:np.float64(0.0008353491721209139) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017292899719905108) - rollout_corr/log_ppl_diff_max:np.float64(0.006912916898727417) - rollout_corr/log_ppl_diff_min:np.float64(-0.0048788487911224365) - rollout_corr/ppl_ratio:np.float64(1.0008378000929952) - rollout_corr/chi2_token:np.float64(0.0016067922115325928) - rollout_corr/chi2_seq:np.float64(0.6984529327601194) - actor/pg_loss:np.float64(0.005961017799563706) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00028992991656195954) - actor/ppo_kl:np.float64(0.0001396356336407223) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6015625) - self_distillation/token_reweight_w_mean:np.float64(9105.852561026346) - self_distillation/token_reweight_w_std:np.float64(160757.80174179428) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988879463635385) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1225835490913596) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.33824220299720764) - perf/mfu/actor:np.float64(0.05203885094538403) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.26508331298828) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:0 - critic/score/mean:0.6015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001895678462460637 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001895678462460637 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:395.63671875 - respo
(TaskRunner pid=2255397) 
Training Progress:  31%|███       | 31/100 [28:34<1:09:34, 60.50s/it]
(TaskRunner pid=2255397) step:32 - global_seqlen/min:17172 - global_seqlen/max:25152 - global_seqlen/minmax_diff:7980 - global_seqlen/balanced_min:20878 - global_seqlen/balanced_max:20886 - global_seqlen/mean:20883.0 - actor/entropy:0.4830181300640106 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3013932406902313 - training/rollout_probs_diff_mean:0.00495893694460392 - training/rollout_probs_diff_std:0.010702594183385372 - training/rollout_actor_probs_pearson_corr:0.9992987513542175 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.55078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.0001959800720215 - rollout_corr/rollout_is_ratio_fraction_high:1.0022450624092016e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.0067350962781347e-05 - rollout_corr/rollout_is_max:2.244924545288086 - rollout_corr/rollout_is_min:0.39147886633872986 - rollout_corr/rollout_is_std:0.037390995770692825 - rollout_corr/rollout_is_eff_sample_size:0.998604459752836 - rollout_corr/rollout_is_seq_mean:1.0002524852752686 - rollout_corr/rollout_is_seq_std:0.0021919296123087406 - rollout_corr/rollout_is_seq_max:1.0080853700637817 - rollout_corr/rollout_is_seq_min:0.9943554401397705 - rollout_corr/rollout_is_seq_max_deviation:0.008085370063781738 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6529409568756819) - rollout_corr/training_log_ppl:np.float64(0.4933903770870529) - rollout_corr/kl:np.float64(0.0006465979992356097) - rollout_corr/k3_kl:np.float64(0.0008415120931317688) - rollout_corr/rollout_ppl:np.float64(1.651906920131296) - rollout_corr/rollout_log_ppl:np.float64(0.49274377603433095) - rollout_corr/log_ppl_diff:np.float64(0.0006466010527219623) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017735535220708698) - rollout_corr/log_ppl_diff_max:np.float64(0.007363229990005493) - rollout_corr/log_ppl_diff_min:np.float64(-0.010225236415863037) - rollout_corr/ppl_ratio:np.float64(1.000649468274787) - rollout_corr/chi2_token:np.float64(0.0021671864669770002) - rollout_corr/chi2_seq:np.float64(1.2125774417072535) - actor/pg_loss:np.float64(0.007539922837167978) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005248926790955011) - actor/ppo_kl:np.float64(0.00015378757747441796) - actor/pg_clipfrac_lower:np.float64(1.2849506674683653e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.55078125) - self_distillation/token_reweight_w_mean:np.float64(28962.190404822817) - self_distillation/token_reweight_w_std:np.float64(594865.4367179495) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994088341481984) - self_distillation/token_reweight_w_clip_frac:np.float64(0.15152001177193597) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.4039623737335205) - perf/mfu/actor:np.float64(0.050354520500870484) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.79799795150757) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:0 - critic/score/mean:0.55078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005901970434933901 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.005901970434933901 - critic/returns/max:0.75 - critic/returns/min:-0.875 - r
(TaskRunner pid=2255397) 
Training Progress:  32%|███▏      | 32/100 [29:15<1:01:50, 54.56s/it]
(TaskRunner pid=2255397) step:33 - global_seqlen/min:16273 - global_seqlen/max:24765 - global_seqlen/minmax_diff:8492 - global_seqlen/balanced_min:20358 - global_seqlen/balanced_max:20368 - global_seqlen/mean:20365.75 - actor/entropy:0.5308820605278015 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2607361674308777 - training/rollout_probs_diff_mean:0.005311037413775921 - training/rollout_probs_diff_std:0.010893904604017735 - training/rollout_actor_probs_pearson_corr:0.9993165731430054 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.60546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0000280141830444 - rollout_corr/rollout_is_ratio_fraction_high:2.2798258214606903e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1399129107303452e-05 - rollout_corr/rollout_is_max:2.4295754432678223 - rollout_corr/rollout_is_min:0.4068605899810791 - rollout_corr/rollout_is_std:0.038521647453308105 - rollout_corr/rollout_is_eff_sample_size:0.9985184003675729 - rollout_corr/rollout_is_seq_mean:1.0000193119049072 - rollout_corr/rollout_is_seq_std:0.0023478458169847727 - rollout_corr/rollout_is_seq_max:1.0072718858718872 - rollout_corr/rollout_is_seq_min:0.9926888346672058 - rollout_corr/rollout_is_seq_max_deviation:0.0073111653327941895 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7451868052594364) - rollout_corr/training_log_ppl:np.float64(0.5472010709927417) - rollout_corr/kl:np.float64(0.000750623744014689) - rollout_corr/k3_kl:np.float64(0.0009170059764755933) - rollout_corr/rollout_ppl:np.float64(1.7438560076989233) - rollout_corr/rollout_log_ppl:np.float64(0.5464504445553757) - rollout_corr/log_ppl_diff:np.float64(0.0007506264373660088) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020804237574338913) - rollout_corr/log_ppl_diff_max:np.float64(0.011185050010681152) - rollout_corr/log_ppl_diff_min:np.float64(-0.008495688438415527) - rollout_corr/ppl_ratio:np.float64(1.0007542204111814) - rollout_corr/chi2_token:np.float64(0.0021686023101210594) - rollout_corr/chi2_seq:np.float64(1.0599001871887594) - actor/pg_loss:np.float64(0.0046818595146760345) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000626342507530353) - actor/ppo_kl:np.float64(-7.711236621688045e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.60546875) - self_distillation/token_reweight_w_mean:np.float64(406.9383929811884) - self_distillation/token_reweight_w_std:np.float64(6888.8986393570085) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006252727471292) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10766920002060942) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.35059650987386703) - perf/mfu/actor:np.float64(0.04906215016751242) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.48283004760742) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:0 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0038529057055711746 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0038529057055711746 - critic/returns/max:0.875 - critic/returns/min
(TaskRunner pid=2255397) 
Training Progress:  33%|███▎      | 33/100 [29:54<55:40, 49.86s/it]  
(TaskRunner pid=2255397) step:34 - global_seqlen/min:19407 - global_seqlen/max:28954 - global_seqlen/minmax_diff:9547 - global_seqlen/balanced_min:20764 - global_seqlen/balanced_max:28104 - global_seqlen/mean:21689.625 - actor/entropy:0.4711659848690033 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26104798913002014 - training/rollout_probs_diff_mean:0.004729302600026131 - training/rollout_probs_diff_std:0.010431492701172829 - training/rollout_actor_probs_pearson_corr:0.9993188977241516 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.85546875 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.85546875 - rollout_corr/rollout_is_mean:0.9998348355293274 - rollout_corr/rollout_is_ratio_fraction_high:1.0172008842346258e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.372540235519409 - rollout_corr/rollout_is_min:0.5585512518882751 - rollout_corr/rollout_is_std:0.03565623611211777 - rollout_corr/rollout_is_eff_sample_size:0.9987297716220098 - rollout_corr/rollout_is_seq_mean:0.9998084902763367 - rollout_corr/rollout_is_seq_std:0.0021642311476171017 - rollout_corr/rollout_is_seq_max:1.0065914392471313 - rollout_corr/rollout_is_seq_min:0.9937243461608887 - rollout_corr/rollout_is_seq_max_deviation:0.006591439247131348 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.732406364288181) - rollout_corr/training_log_ppl:np.float64(0.53354305683024) - rollout_corr/kl:np.float64(0.0010742717632254706) - rollout_corr/k3_kl:np.float64(0.0007844316083662761) - rollout_corr/rollout_ppl:np.float64(1.7304212134331465) - rollout_corr/rollout_log_ppl:np.float64(0.5324687818174425) - rollout_corr/log_ppl_diff:np.float64(0.0010742750127974432) - rollout_corr/log_ppl_abs_diff:np.float64(0.002004527148528723) - rollout_corr/log_ppl_diff_max:np.float64(0.010212719440460205) - rollout_corr/log_ppl_diff_min:np.float64(-0.0042626261711120605) - rollout_corr/ppl_ratio:np.float64(1.0010776109993458) - rollout_corr/chi2_token:np.float64(0.000989705789834261) - rollout_corr/chi2_seq:np.float64(0.5540260763373226) - actor/pg_loss:np.float64(0.005981583381071687) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004929028409605962) - actor/ppo_kl:np.float64(0.00015171869483765477) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.85546875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.85546875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(11402.142899290891) - self_distillation/token_reweight_w_std:np.float64(223975.93910462921) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007735511753708) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13933006773004308) - self_distillation/empty_target_batch:np.float64(0.14453125) - actor/grad_norm:np.float64(0.3575190082192421) - perf/mfu/actor:np.float64(0.04880687442998337) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.40957641601562) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:0 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.05439735949039459 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.05439735949039459 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/me
(TaskRunner pid=2255397) 
Training Progress:  34%|███▍      | 34/100 [31:25<1:08:35, 62.35s/it]
(TaskRunner pid=2255397) step:35 - global_seqlen/min:19463 - global_seqlen/max:21625 - global_seqlen/minmax_diff:2162 - global_seqlen/balanced_min:20560 - global_seqlen/balanced_max:20574 - global_seqlen/mean:20569.125 - actor/entropy:0.4701376259326935 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27064740657806396 - training/rollout_probs_diff_mean:0.00488562835380435 - training/rollout_probs_diff_std:0.010647308081388474 - training/rollout_actor_probs_pearson_corr:0.999289870262146 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.5078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:0.9999282956123352 - rollout_corr/rollout_is_ratio_fraction_high:1.0713864867284428e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.1437816619873047 - rollout_corr/rollout_is_min:0.5562102794647217 - rollout_corr/rollout_is_std:0.036710504442453384 - rollout_corr/rollout_is_eff_sample_size:0.9986539149503298 - rollout_corr/rollout_is_seq_mean:0.9999589323997498 - rollout_corr/rollout_is_seq_std:0.0022408063523471355 - rollout_corr/rollout_is_seq_max:1.0093075037002563 - rollout_corr/rollout_is_seq_min:0.9939238429069519 - rollout_corr/rollout_is_seq_max_deviation:0.009307503700256348 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.643273327499628) - rollout_corr/training_log_ppl:np.float64(0.48507256753509864) - rollout_corr/kl:np.float64(0.0008028178892658389) - rollout_corr/k3_kl:np.float64(0.0007829796426221947) - rollout_corr/rollout_ppl:np.float64(1.641896395944059) - rollout_corr/rollout_log_ppl:np.float64(0.4842697436106391) - rollout_corr/log_ppl_diff:np.float64(0.000802823924459517) - rollout_corr/log_ppl_abs_diff:np.float64(0.002036678488366306) - rollout_corr/log_ppl_diff_max:np.float64(0.008617639541625977) - rollout_corr/log_ppl_diff_min:np.float64(-0.010091245174407959) - rollout_corr/ppl_ratio:np.float64(1.0008062841370702) - rollout_corr/chi2_token:np.float64(0.0015425898600369692) - rollout_corr/chi2_seq:np.float64(1.5650525961536914) - actor/pg_loss:np.float64(0.0024570953100919724) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004893736586382147) - actor/ppo_kl:np.float64(4.371891989851662e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5078125) - self_distillation/token_reweight_w_mean:np.float64(28164.23297266336) - self_distillation/token_reweight_w_std:np.float64(438345.5305433142) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0024336164351553) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09937885942053981) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.2858409211039543) - perf/mfu/actor:np.float64(0.04990645011261699) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.3169288635254) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:0 - critic/score/mean:0.5078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004351168405264616 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004351168405264616 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:364.59765625 - response_length/
(TaskRunner pid=2255397) 
Training Progress:  35%|███▌      | 35/100 [32:05<1:00:02, 55.42s/it]
(TaskRunner pid=2255397) step:36 - global_seqlen/min:15946 - global_seqlen/max:23368 - global_seqlen/minmax_diff:7422 - global_seqlen/balanced_min:21144 - global_seqlen/balanced_max:21148 - global_seqlen/mean:21146.25 - actor/entropy:0.4903087913990021 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18462920188903809 - training/rollout_probs_diff_mean:0.005082453601062298 - training/rollout_probs_diff_std:0.010714489035308361 - training/rollout_actor_probs_pearson_corr:0.9993000030517578 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:1.0000641345977783 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6886519193649292 - rollout_corr/rollout_is_min:0.6301243901252747 - rollout_corr/rollout_is_std:0.037293627858161926 - rollout_corr/rollout_is_eff_sample_size:0.9986113546547832 - rollout_corr/rollout_is_seq_mean:1.000097632408142 - rollout_corr/rollout_is_seq_std:0.002074406947940588 - rollout_corr/rollout_is_seq_max:1.0062254667282104 - rollout_corr/rollout_is_seq_min:0.9938981533050537 - rollout_corr/rollout_is_seq_max_deviation:0.006225466728210449 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6528221243061125) - rollout_corr/training_log_ppl:np.float64(0.4915998738142662) - rollout_corr/kl:np.float64(0.0008901074607035042) - rollout_corr/k3_kl:np.float64(0.000780510345350649) - rollout_corr/rollout_ppl:np.float64(1.6513647134415805) - rollout_corr/rollout_log_ppl:np.float64(0.49070976028451696) - rollout_corr/log_ppl_diff:np.float64(0.0008901135297492146) - rollout_corr/log_ppl_abs_diff:np.float64(0.001717470120638609) - rollout_corr/log_ppl_diff_max:np.float64(0.006238460540771484) - rollout_corr/log_ppl_diff_min:np.float64(-0.005200505256652832) - rollout_corr/ppl_ratio:np.float64(1.0008924398571253) - rollout_corr/chi2_token:np.float64(0.0013305631000548601) - rollout_corr/chi2_seq:np.float64(0.19096036697737873) - actor/pg_loss:np.float64(0.0061755290953442454) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007742183429400029) - actor/ppo_kl:np.float64(0.0002702638142206837) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(5297.740719528403) - self_distillation/token_reweight_w_std:np.float64(105322.54200576013) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011047541629523) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1423137153033167) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.3517342582345009) - perf/mfu/actor:np.float64(0.051460493101783154) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.31007385253906) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:0 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0027716271579265594 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0027716271579265594 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:382.8203125 - response_length/max:71
(TaskRunner pid=2255397) 
Training Progress:  36%|███▌      | 36/100 [32:44<54:07, 50.74s/it]  
(TaskRunner pid=2255397) step:37 - global_seqlen/min:18794 - global_seqlen/max:24933 - global_seqlen/minmax_diff:6139 - global_seqlen/balanced_min:22097 - global_seqlen/balanced_max:22101 - global_seqlen/mean:22100.0 - actor/entropy:0.4772471785545349 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8962392210960388 - training/rollout_probs_diff_mean:0.004988708533346653 - training/rollout_probs_diff_std:0.011156635358929634 - training/rollout_actor_probs_pearson_corr:0.9992755651473999 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.54296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:0.9999436140060425 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.1014806154416874e-05 - rollout_corr/rollout_is_max:1.5039196014404297 - rollout_corr/rollout_is_min:0.0692535936832428 - rollout_corr/rollout_is_std:0.03678025305271149 - rollout_corr/rollout_is_eff_sample_size:0.9986488027678234 - rollout_corr/rollout_is_seq_mean:0.9999807476997375 - rollout_corr/rollout_is_seq_std:0.002165940823033452 - rollout_corr/rollout_is_seq_max:1.0075201988220215 - rollout_corr/rollout_is_seq_min:0.9936956763267517 - rollout_corr/rollout_is_seq_max_deviation:0.007520198822021484 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6701486743986607) - rollout_corr/training_log_ppl:np.float64(0.5012833332875744) - rollout_corr/kl:np.float64(0.0008904281232275935) - rollout_corr/k3_kl:np.float64(0.0009389091193838794) - rollout_corr/rollout_ppl:np.float64(1.6685718595981598) - rollout_corr/rollout_log_ppl:np.float64(0.5003929063677788) - rollout_corr/log_ppl_diff:np.float64(0.0008904269197955728) - rollout_corr/log_ppl_abs_diff:np.float64(0.001979760592803359) - rollout_corr/log_ppl_diff_max:np.float64(0.02197730541229248) - rollout_corr/log_ppl_diff_min:np.float64(-0.00723642110824585) - rollout_corr/ppl_ratio:np.float64(1.0008947688620538) - rollout_corr/chi2_token:np.float64(0.0017904054839164019) - rollout_corr/chi2_seq:np.float64(0.7441312682349235) - actor/pg_loss:np.float64(0.0051974470261484385) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006028023444741848) - actor/ppo_kl:np.float64(0.00011284644474807148) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.54296875) - self_distillation/token_reweight_w_mean:np.float64(17486.880942083197) - self_distillation/token_reweight_w_std:np.float64(248698.93032729038) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008525024168193) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10978465108200908) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.33661027252674103) - perf/mfu/actor:np.float64(0.053537470413758295) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.20091247558594) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:0 - critic/score/mean:0.54296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.54296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00370756140910089 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00370756140910089 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:3
(TaskRunner pid=2255397) 
Training Progress:  37%|███▋      | 37/100 [33:24<49:52, 47.50s/it]
(TaskRunner pid=2255397) step:38 - global_seqlen/min:17401 - global_seqlen/max:23183 - global_seqlen/minmax_diff:5782 - global_seqlen/balanced_min:20614 - global_seqlen/balanced_max:20619 - global_seqlen/mean:20617.125 - actor/entropy:0.4591502249240875 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21379658579826355 - training/rollout_probs_diff_mean:0.005224332213401794 - training/rollout_probs_diff_std:0.011259016580879688 - training/rollout_actor_probs_pearson_corr:0.9992035627365112 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.6484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:1.0001171827316284 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.0498313460848294e-05 - rollout_corr/rollout_is_max:1.9823302030563354 - rollout_corr/rollout_is_min:0.3327290415763855 - rollout_corr/rollout_is_std:0.03824835270643234 - rollout_corr/rollout_is_eff_sample_size:0.9985396761004448 - rollout_corr/rollout_is_seq_mean:1.0001174211502075 - rollout_corr/rollout_is_seq_std:0.002267003059387207 - rollout_corr/rollout_is_seq_max:1.007875919342041 - rollout_corr/rollout_is_seq_min:0.9941402077674866 - rollout_corr/rollout_is_seq_max_deviation:0.007875919342041016 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.597476608119905) - rollout_corr/training_log_ppl:np.float64(0.4596842197352089) - rollout_corr/kl:np.float64(0.0007120158846873892) - rollout_corr/k3_kl:np.float64(0.0008242313452342387) - rollout_corr/rollout_ppl:np.float64(1.5963355526328087) - rollout_corr/rollout_log_ppl:np.float64(0.45897220366168767) - rollout_corr/log_ppl_diff:np.float64(0.0007120160735212266) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018952106474898756) - rollout_corr/log_ppl_diff_max:np.float64(0.006997823715209961) - rollout_corr/log_ppl_diff_min:np.float64(-0.006651520729064941) - rollout_corr/ppl_ratio:np.float64(1.0007150201126933) - rollout_corr/chi2_token:np.float64(0.001860827673226595) - rollout_corr/chi2_seq:np.float64(0.6607308345846832) - actor/pg_loss:np.float64(0.003047003992833197) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027509686378834886) - actor/ppo_kl:np.float64(6.150551095629453e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_w_mean:np.float64(24324.67800112092) - self_distillation/token_reweight_w_std:np.float64(319731.90878068085) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001283828169107) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07227780664106831) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.25013007409870625) - perf/mfu/actor:np.float64(0.05036454480977212) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.1811408996582) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:0 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0011094711953774095 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0011094711953774095 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:3
(TaskRunner pid=2255397) 
Training Progress:  38%|███▊      | 38/100 [34:04<46:30, 45.01s/it]
(TaskRunner pid=2255397) step:39 - global_seqlen/min:16098 - global_seqlen/max:24192 - global_seqlen/minmax_diff:8094 - global_seqlen/balanced_min:21469 - global_seqlen/balanced_max:21484 - global_seqlen/mean:21479.5 - actor/entropy:0.4425225853919983 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17422929406166077 - training/rollout_probs_diff_mean:0.004969022236764431 - training/rollout_probs_diff_std:0.010954358614981174 - training/rollout_actor_probs_pearson_corr:0.9992186427116394 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.6015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0003507137298584 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6815071105957031 - rollout_corr/rollout_is_min:0.5040706396102905 - rollout_corr/rollout_is_std:0.03746902570128441 - rollout_corr/rollout_is_eff_sample_size:0.9985991103255378 - rollout_corr/rollout_is_seq_mean:1.0004005432128906 - rollout_corr/rollout_is_seq_std:0.0023794230073690414 - rollout_corr/rollout_is_seq_max:1.012694001197815 - rollout_corr/rollout_is_seq_min:0.9931012392044067 - rollout_corr/rollout_is_seq_max_deviation:0.012694001197814941 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6068592290394008) - rollout_corr/training_log_ppl:np.float64(0.4637760063924361) - rollout_corr/kl:np.float64(0.0005125066249966892) - rollout_corr/k3_kl:np.float64(0.0008192024238837803) - rollout_corr/rollout_ppl:np.float64(1.6059974883683026) - rollout_corr/rollout_log_ppl:np.float64(0.46326349792070687) - rollout_corr/log_ppl_diff:np.float64(0.0005125084717292339) - rollout_corr/log_ppl_abs_diff:np.float64(0.001737583806971088) - rollout_corr/log_ppl_diff_max:np.float64(0.006271004676818848) - rollout_corr/log_ppl_diff_min:np.float64(-0.01115313172340393) - rollout_corr/ppl_ratio:np.float64(1.0005151890218258) - rollout_corr/chi2_token:np.float64(0.0022532844450324774) - rollout_corr/chi2_seq:np.float64(1.0010677098762244) - actor/pg_loss:np.float64(0.004201453877612948) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00037899658491369337) - actor/ppo_kl:np.float64(0.000158508868715046) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6015625) - self_distillation/token_reweight_w_mean:np.float64(21191.4891625538) - self_distillation/token_reweight_w_std:np.float64(305819.6095345903) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0013274641241878) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08719093064428307) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.2567609027028084) - perf/mfu/actor:np.float64(0.05189741811293595) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.25640106201172) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:0 - critic/score/mean:0.6015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00024856385425664485 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00024856385425664485 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:388.953125 - res
(TaskRunner pid=2255397) 
Training Progress:  39%|███▉      | 39/100 [34:45<44:34, 43.84s/it]
(TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40}
(TaskRunner pid=2255397) validation generation end
(TaskRunner pid=2255397) [prompt] system
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2255397) <reasoning>
(TaskRunner pid=2255397) ...
(TaskRunner pid=2255397) </reasoning>
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) ...
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) A
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) user
(TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2255397) Please reason step by step.
(TaskRunner pid=2255397) assistant
(TaskRunner pid=2255397) <think>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) </think>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) [response] <reasoning>
(TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we must examine the structure of the product and compare it to the structures of the given options.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) The product contains a nitrogen-containing ring with a substituent group that includes a chlorinated benzene ring and a trifluoromethyl group. Specifically, the product has a structure where a nitrogen atom is part of a ring that is connected to a carbon chain leading to a chlorinated benzene ring with a trifluoromethyl group.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Now, let's analyze the options:
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) - **Option A**: Contains a structure with a nitrogen ring and a chlorinated benzene ring, along with a trifluoromethyl group. It closely resembles the structure of the target product, especially in terms of the nitrogen-containing ring and the substituent groups.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) - **Option B**: Contains a chlorine atom directly attached to the nitrogen ring, which is not present in the target product. This makes it an incorrect match.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) - **Option C**: Features a hydroxyl group and a different substituent pattern, which does not match the target structure.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) - **Option D**: Includes a double bond and a fluorinated carbon chain, but not the specific chlorinated benzene ring and nitrogen ring arrangement of the target product.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Only **Option A** contains the correct structural elements that match the target product.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) </reasoning>
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) A
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) [ground_truth] A
(TaskRunner pid=2255397) [score] 1.0
(TaskRunner pid=2255397) [acc] 1.0
(TaskRunner pid=2255397) [pred] A
(TaskRunner pid=2255397) [incorrect_format] 1
(TaskRunner pid=2255397) [feedback] 
(TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40
(TaskRunner pid=2255397) step:40 - global_seqlen/min:18616 - global_seqlen/max:26413 - global_seqlen/minmax_diff:7797 - global_seqlen/balanced_min:22648 - global_seqlen/balanced_max:22652 - global_seqlen/mean:22651.25 - actor/entropy:0.46682530641555786 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102363586425781 - training/rollout_probs_diff_mean:0.005161741282790899 - training/rollout_probs_diff_std:0.011252960190176964 - training/rollout_actor_probs_pearson_corr:0.9991944432258606 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.6484375 - self_distillation/correct_sample_fraction:0.55859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6484375 - rollout_corr/rollout_is_mean:1.0001730918884277 - rollout_corr/rollout_is_ratio_fraction_high:9.88513511401834e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.977027022803668e-05 - rollout_corr/rollout_is_max:2.2787070274353027 - rollout_corr/rollout_is_min:0.2646176815032959 - rollout_corr/rollout_is_std:0.03861745819449425 - rollout_corr/rollout_is_eff_sample_size:0.9985113878945102 - rollout_corr/rollout_is_seq_mean:1.0001778602600098 - rollout_corr/rollout_is_seq_std:0.0021594883874058723 - rollout_corr/rollout_is_seq_max:1.0066732168197632 - rollout_corr/rollout_is_seq_min:0.9935801029205322 - rollout_corr/rollout_is_seq_max_deviation:0.006673216819763184 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6297845151275396) - rollout_corr/training_log_ppl:np.float64(0.4774156858911738) - rollout_corr/kl:np.float64(0.0007412694044468537) - rollout_corr/k3_kl:np.float64(0.0009004596647628205) - rollout_corr/rollout_ppl:np.float64(1.628512756433338) - rollout_corr/rollout_log_ppl:np.float64(0.47667441627709195) - rollout_corr/log_ppl_diff:np.float64(0.0007412696140818298) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018867639009840786) - rollout_corr/log_ppl_diff_max:np.float64(0.00960838794708252) - rollout_corr/log_ppl_diff_min:np.float64(-0.005440711975097656) - rollout_corr/ppl_ratio:np.float64(1.0007440785411745) - rollout_corr/chi2_token:np.float64(0.0020869714207947254) - rollout_corr/chi2_seq:np.float64(0.9351563951931894) - actor/pg_loss:np.float64(0.003354526706971228) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000359561878212844) - actor/ppo_kl:np.float64(0.00013469158743095733) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.55859375) - self_distillation/token_reweight_w_mean:np.float64(9349.600620643701) - self_distillation/token_reweight_w_std:np.float64(148328.15592557297) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9980465415865183) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08430611382937059) - self_distillation/empty_target_batch:np.float64(0.3515625) - actor/grad_norm:np.float64(0.28926024958491325) - perf/mfu/actor:np.float64(0.054041057160877155) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.16180038452148) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6455357142857143) - val-aux/sciknoweval/reward/std@16:np.float64(0.19709042049186706) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7234142857142857) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15652759126874422) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5656380952380953) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1768887
(TaskRunner pid=2255397) 
Training Progress:  40%|████      | 40/100 [37:57<1:28:24, 88.41s/it]
(TaskRunner pid=2255397) step:41 - global_seqlen/min:18283 - global_seqlen/max:24700 - global_seqlen/minmax_diff:6417 - global_seqlen/balanced_min:21101 - global_seqlen/balanced_max:21109 - global_seqlen/mean:21105.5 - actor/entropy:0.42351460456848145 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9826107025146484 - training/rollout_probs_diff_mean:0.005102348513901234 - training/rollout_probs_diff_std:0.01174638606607914 - training/rollout_actor_probs_pearson_corr:0.9991093873977661 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.999843180179596 - rollout_corr/rollout_is_ratio_fraction_high:9.873228009382728e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.873228009382728e-06 - rollout_corr/rollout_is_max:2.636786937713623 - rollout_corr/rollout_is_min:0.013885053806006908 - rollout_corr/rollout_is_std:0.03809376060962677 - rollout_corr/rollout_is_eff_sample_size:0.9985506114667393 - rollout_corr/rollout_is_seq_mean:0.9998304843902588 - rollout_corr/rollout_is_seq_std:0.001992590492591262 - rollout_corr/rollout_is_seq_max:1.0068778991699219 - rollout_corr/rollout_is_seq_min:0.9926929473876953 - rollout_corr/rollout_is_seq_max_deviation:0.0073070526123046875 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5631140493787825) - rollout_corr/training_log_ppl:np.float64(0.436920084990561) - rollout_corr/kl:np.float64(0.0011501897124975846) - rollout_corr/k3_kl:np.float64(0.0009579621572584074) - rollout_corr/rollout_ppl:np.float64(1.5612433003261685) - rollout_corr/rollout_log_ppl:np.float64(0.4357698948588222) - rollout_corr/log_ppl_diff:np.float64(0.001150190131738782) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019631539471447468) - rollout_corr/log_ppl_diff_max:np.float64(0.010622143745422363) - rollout_corr/log_ppl_diff_min:np.float64(-0.00589299201965332) - rollout_corr/ppl_ratio:np.float64(1.001153603894636) - rollout_corr/chi2_token:np.float64(0.0014891347382217646) - rollout_corr/chi2_seq:np.float64(0.3713759642560035) - actor/pg_loss:np.float64(0.007653480512090027) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008902258696252829) - actor/ppo_kl:np.float64(0.0001822680885723571) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(1181.0103185500484) - self_distillation/token_reweight_w_std:np.float64(19672.96590518701) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9963315855711699) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1375392735353671) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.34661057591438293) - perf/mfu/actor:np.float64(0.05125395817020568) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.22666931152344) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:0 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004591051023453474 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004591051023453474 - critic/returns/max:0.875 - critic/returns/min:-0.875 - respons
(TaskRunner pid=2255397) 
Training Progress:  41%|████      | 41/100 [38:37<1:12:34, 73.81s/it]
(TaskRunner pid=2255397) step:42 - global_seqlen/min:15988 - global_seqlen/max:22821 - global_seqlen/minmax_diff:6833 - global_seqlen/balanced_min:19797 - global_seqlen/balanced_max:19801 - global_seqlen/mean:19798.875 - actor/entropy:0.39622214436531067 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6605160236358643 - training/rollout_probs_diff_mean:0.005419079214334488 - training/rollout_probs_diff_std:0.012383614666759968 - training/rollout_actor_probs_pearson_corr:0.998907744884491 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.9609375 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9609375 - rollout_corr/rollout_is_mean:0.9999876022338867 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.1576137442025356e-05 - rollout_corr/rollout_is_max:1.7245110273361206 - rollout_corr/rollout_is_min:0.2930092215538025 - rollout_corr/rollout_is_std:0.03913719579577446 - rollout_corr/rollout_is_eff_sample_size:0.9984703847272242 - rollout_corr/rollout_is_seq_mean:0.9999865293502808 - rollout_corr/rollout_is_seq_std:0.002267190022394061 - rollout_corr/rollout_is_seq_max:1.0080516338348389 - rollout_corr/rollout_is_seq_min:0.9936971664428711 - rollout_corr/rollout_is_seq_max_deviation:0.008051633834838867 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5080496412701905) - rollout_corr/training_log_ppl:np.float64(0.4042311883531511) - rollout_corr/kl:np.float64(0.0007036857559299392) - rollout_corr/k3_kl:np.float64(0.0009411578781737262) - rollout_corr/rollout_ppl:np.float64(1.5069640525616705) - rollout_corr/rollout_log_ppl:np.float64(0.40352750167949125) - rollout_corr/log_ppl_diff:np.float64(0.0007036866736598313) - rollout_corr/log_ppl_abs_diff:np.float64(0.001983315742108971) - rollout_corr/log_ppl_diff_max:np.float64(0.0076701343059539795) - rollout_corr/log_ppl_diff_min:np.float64(-0.006415665149688721) - rollout_corr/ppl_ratio:np.float64(1.0007067790720612) - rollout_corr/chi2_token:np.float64(0.002371277194470167) - rollout_corr/chi2_seq:np.float64(0.9433175600133836) - actor/pg_loss:np.float64(0.010519832605496049) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009025154786286294) - actor/ppo_kl:np.float64(-0.00013643282175568316) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(12954.285722455475) - self_distillation/token_reweight_w_std:np.float64(227959.35449937586) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0025726119056344) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14800434719654731) - self_distillation/empty_target_batch:np.float64(0.0390625) - actor/grad_norm:np.float64(0.45579010248184204) - perf/mfu/actor:np.float64(0.04795526011784892) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.25069046020508) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:0 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0013215383514761925 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0013215383514761925 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/
(TaskRunner pid=2255397) 
Training Progress:  42%|████▏     | 42/100 [39:16<1:01:21, 63.48s/it]
(TaskRunner pid=2255397) step:43 - global_seqlen/min:18065 - global_seqlen/max:23096 - global_seqlen/minmax_diff:5031 - global_seqlen/balanced_min:20756 - global_seqlen/balanced_max:20762 - global_seqlen/mean:20760.875 - actor/entropy:0.42775002121925354 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3424738645553589 - training/rollout_probs_diff_mean:0.005239918828010559 - training/rollout_probs_diff_std:0.01176416501402855 - training/rollout_actor_probs_pearson_corr:0.999097466468811 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.79296875 - self_distillation/correct_sample_fraction:0.61328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.79296875 - rollout_corr/rollout_is_mean:0.9999403357505798 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1294968317088205e-05 - rollout_corr/rollout_is_max:1.6902360916137695 - rollout_corr/rollout_is_min:0.39213547110557556 - rollout_corr/rollout_is_std:0.038980014622211456 - rollout_corr/rollout_is_eff_sample_size:0.9984827447477223 - rollout_corr/rollout_is_seq_mean:0.9998909831047058 - rollout_corr/rollout_is_seq_std:0.002513354877009988 - rollout_corr/rollout_is_seq_max:1.0098495483398438 - rollout_corr/rollout_is_seq_min:0.9915899038314819 - rollout_corr/rollout_is_seq_max_deviation:0.00984954833984375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5651745153591037) - rollout_corr/training_log_ppl:np.float64(0.4374735167075414) - rollout_corr/kl:np.float64(0.0007840867953743391) - rollout_corr/k3_kl:np.float64(0.0008964727625198066) - rollout_corr/rollout_ppl:np.float64(1.5639647683128715) - rollout_corr/rollout_log_ppl:np.float64(0.4366894279664848) - rollout_corr/log_ppl_diff:np.float64(0.0007840887410566211) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020498010562732816) - rollout_corr/log_ppl_diff_max:np.float64(0.009105801582336426) - rollout_corr/log_ppl_diff_min:np.float64(-0.00906282663345337) - rollout_corr/ppl_ratio:np.float64(1.0007877822499722) - rollout_corr/chi2_token:np.float64(0.00206923414953053) - rollout_corr/chi2_seq:np.float64(1.3589389722328633) - actor/pg_loss:np.float64(0.004318759543821216) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004535638540801301) - actor/ppo_kl:np.float64(-0.00012510620860073374) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.79296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.79296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.61328125) - self_distillation/token_reweight_w_mean:np.float64(39451.46745206439) - self_distillation/token_reweight_w_std:np.float64(461544.07824653096) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995274562388659) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09244409290840849) - self_distillation/empty_target_batch:np.float64(0.20703125) - actor/grad_norm:np.float64(0.32581663131713867) - perf/mfu/actor:np.float64(0.05044233796581919) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.0129623413086) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:0 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0011210255324840546 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0011210255324840546 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_leng
(TaskRunner pid=2255397) 
Training Progress:  43%|████▎     | 43/100 [39:56<53:26, 56.25s/it]  
(TaskRunner pid=2255397) step:44 - global_seqlen/min:19213 - global_seqlen/max:22622 - global_seqlen/minmax_diff:3409 - global_seqlen/balanced_min:20879 - global_seqlen/balanced_max:20890 - global_seqlen/mean:20886.75 - actor/entropy:0.43929049372673035 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22547024488449097 - training/rollout_probs_diff_mean:0.0053957682102918625 - training/rollout_probs_diff_std:0.011915630660951138 - training/rollout_actor_probs_pearson_corr:0.9990841150283813 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.000036358833313 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9659661054611206 - rollout_corr/rollout_is_min:0.5844541192054749 - rollout_corr/rollout_is_std:0.039155468344688416 - rollout_corr/rollout_is_eff_sample_size:0.9984691962798411 - rollout_corr/rollout_is_seq_mean:1.0000959634780884 - rollout_corr/rollout_is_seq_std:0.0024589113891124725 - rollout_corr/rollout_is_seq_max:1.0087720155715942 - rollout_corr/rollout_is_seq_min:0.9921355843544006 - rollout_corr/rollout_is_seq_max_deviation:0.008772015571594238 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6047232481651008) - rollout_corr/training_log_ppl:np.float64(0.46260377258295193) - rollout_corr/kl:np.float64(0.0009372506496678756) - rollout_corr/k3_kl:np.float64(0.000976888309367041) - rollout_corr/rollout_ppl:np.float64(1.6032404340803623) - rollout_corr/rollout_log_ppl:np.float64(0.4616665196372196) - rollout_corr/log_ppl_diff:np.float64(0.0009372529457323253) - rollout_corr/log_ppl_abs_diff:np.float64(0.002075349271763116) - rollout_corr/log_ppl_diff_max:np.float64(0.013183057308197021) - rollout_corr/log_ppl_diff_min:np.float64(-0.006263077259063721) - rollout_corr/ppl_ratio:np.float64(1.000940943369642) - rollout_corr/chi2_token:np.float64(0.0020958164241164923) - rollout_corr/chi2_seq:np.float64(0.34359696647152305) - actor/pg_loss:np.float64(0.005748242023400962) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005829084911965765) - actor/ppo_kl:np.float64(0.0002053698056716513) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(14803.191471951548) - self_distillation/token_reweight_w_std:np.float64(246306.37441055127) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9985546870157123) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10855769729823805) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.36420872062444687) - perf/mfu/actor:np.float64(0.05008791245123715) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.9407615661621) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:0 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00279244058765471 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00279244058765471 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:341.1484375
(TaskRunner pid=2255397) 
Training Progress:  44%|████▍     | 44/100 [40:37<48:25, 51.89s/it]
(TaskRunner pid=2255397) step:45 - global_seqlen/min:18835 - global_seqlen/max:26317 - global_seqlen/minmax_diff:7482 - global_seqlen/balanced_min:21440 - global_seqlen/balanced_max:21451 - global_seqlen/mean:21447.875 - actor/entropy:0.4101407527923584 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28218671679496765 - training/rollout_probs_diff_mean:0.004841715097427368 - training/rollout_probs_diff_std:0.01116796862334013 - training/rollout_actor_probs_pearson_corr:0.9992083311080933 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73046875 - self_distillation/correct_sample_fraction:0.5390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73046875 - rollout_corr/rollout_is_mean:0.9999707937240601 - rollout_corr/rollout_is_ratio_fraction_high:1.0986233974108472e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.0986233974108472e-05 - rollout_corr/rollout_is_max:2.0019893646240234 - rollout_corr/rollout_is_min:0.25255540013313293 - rollout_corr/rollout_is_std:0.036969371140003204 - rollout_corr/rollout_is_eff_sample_size:0.998635012024894 - rollout_corr/rollout_is_seq_mean:1.0000436305999756 - rollout_corr/rollout_is_seq_std:0.002479569287970662 - rollout_corr/rollout_is_seq_max:1.0116991996765137 - rollout_corr/rollout_is_seq_min:0.9924298524856567 - rollout_corr/rollout_is_seq_max_deviation:0.011699199676513672 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5578296538442373) - rollout_corr/training_log_ppl:np.float64(0.432097623241134) - rollout_corr/kl:np.float64(0.0007768278248150295) - rollout_corr/k3_kl:np.float64(0.0008250089271086836) - rollout_corr/rollout_ppl:np.float64(1.5565427541732788) - rollout_corr/rollout_log_ppl:np.float64(0.4313207929953933) - rollout_corr/log_ppl_diff:np.float64(0.0007768302457407117) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019471754785627127) - rollout_corr/log_ppl_diff_max:np.float64(0.011698126792907715) - rollout_corr/log_ppl_diff_min:np.float64(-0.010318875312805176) - rollout_corr/ppl_ratio:np.float64(1.0007803542539477) - rollout_corr/chi2_token:np.float64(0.0017588012851774693) - rollout_corr/chi2_seq:np.float64(0.6141417354810983) - actor/pg_loss:np.float64(0.0026712483959272504) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039884855823402177) - actor/ppo_kl:np.float64(6.892802709934642e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5390625) - self_distillation/token_reweight_w_mean:np.float64(38907.24203802785) - self_distillation/token_reweight_w_std:np.float64(568287.8935087718) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006591160781682) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08028327141073532) - self_distillation/empty_target_batch:np.float64(0.26953125) - actor/grad_norm:np.float64(0.28873957321047783) - perf/mfu/actor:np.float64(0.05157141247510403) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.14898681640625) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:0 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005405227188020945 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005405227188020945 - critic/returns/max:0.875 - critic/returns/min:-0.875 -
(TaskRunner pid=2255397) 
Training Progress:  45%|████▌     | 45/100 [41:17<44:07, 48.13s/it]
(TaskRunner pid=2255397) step:46 - global_seqlen/min:17315 - global_seqlen/max:21947 - global_seqlen/minmax_diff:4632 - global_seqlen/balanced_min:19767 - global_seqlen/balanced_max:19776 - global_seqlen/mean:19773.25 - actor/entropy:0.4300471842288971 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5463374257087708 - training/rollout_probs_diff_mean:0.0050852419808506966 - training/rollout_probs_diff_std:0.011606025509536266 - training/rollout_actor_probs_pearson_corr:0.9991208910942078 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8203125 - self_distillation/correct_sample_fraction:0.5546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8203125 - rollout_corr/rollout_is_mean:0.9999197125434875 - rollout_corr/rollout_is_ratio_fraction_high:1.1397570233384613e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.2795140466769226e-05 - rollout_corr/rollout_is_max:2.4014859199523926 - rollout_corr/rollout_is_min:0.26859816908836365 - rollout_corr/rollout_is_std:0.03753577917814255 - rollout_corr/rollout_is_eff_sample_size:0.9985928099624436 - rollout_corr/rollout_is_seq_mean:1.0000156164169312 - rollout_corr/rollout_is_seq_std:0.002315930090844631 - rollout_corr/rollout_is_seq_max:1.0097026824951172 - rollout_corr/rollout_is_seq_min:0.9926007986068726 - rollout_corr/rollout_is_seq_max_deviation:0.009702682495117188 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5882981927134097) - rollout_corr/training_log_ppl:np.float64(0.44962009048322216) - rollout_corr/kl:np.float64(0.0011486389046666545) - rollout_corr/k3_kl:np.float64(0.0009466409361493788) - rollout_corr/rollout_ppl:np.float64(1.5864842752926052) - rollout_corr/rollout_log_ppl:np.float64(0.4484714485006407) - rollout_corr/log_ppl_diff:np.float64(0.0011486419825814664) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022174433688633144) - rollout_corr/log_ppl_diff_max:np.float64(0.015058279037475586) - rollout_corr/log_ppl_diff_min:np.float64(-0.00858527421951294) - rollout_corr/ppl_ratio:np.float64(1.0011532127391547) - rollout_corr/chi2_token:np.float64(0.0015598325990140438) - rollout_corr/chi2_seq:np.float64(0.6058120087254792) - actor/pg_loss:np.float64(0.005358678288757801) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006527249729515461) - actor/ppo_kl:np.float64(0.0004008172310321356) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5546875) - self_distillation/token_reweight_w_mean:np.float64(13853.629367629066) - self_distillation/token_reweight_w_std:np.float64(223926.18253548152) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000649391906336) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12465728094684891) - self_distillation/empty_target_batch:np.float64(0.1796875) - actor/grad_norm:np.float64(0.37411294877529144) - perf/mfu/actor:np.float64(0.04759140321566382) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.96075439453125) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:0 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004386639688163996 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004386639688163996 - critic/returns/max:0.875 - critic/returns/min:-0.875 - res
(TaskRunner pid=2255397) 
Training Progress:  46%|████▌     | 46/100 [41:56<40:59, 45.54s/it]
(TaskRunner pid=2255397) step:47 - global_seqlen/min:16338 - global_seqlen/max:23481 - global_seqlen/minmax_diff:7143 - global_seqlen/balanced_min:20960 - global_seqlen/balanced_max:20969 - global_seqlen/mean:20966.0 - actor/entropy:0.4562165141105652 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135512113571167 - training/rollout_probs_diff_mean:0.004941638559103012 - training/rollout_probs_diff_std:0.01093968003988266 - training/rollout_actor_probs_pearson_corr:0.9992673993110657 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:1.0000667572021484 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9514927864074707 - rollout_corr/rollout_is_min:0.5686209797859192 - rollout_corr/rollout_is_std:0.03698226809501648 - rollout_corr/rollout_is_eff_sample_size:0.998634417604882 - rollout_corr/rollout_is_seq_mean:1.0001049041748047 - rollout_corr/rollout_is_seq_std:0.0021275379694998264 - rollout_corr/rollout_is_seq_max:1.0070573091506958 - rollout_corr/rollout_is_seq_min:0.9932472705841064 - rollout_corr/rollout_is_seq_max_deviation:0.007057309150695801 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6164640178903937) - rollout_corr/training_log_ppl:np.float64(0.46599303945549764) - rollout_corr/kl:np.float64(0.0011402188001117253) - rollout_corr/k3_kl:np.float64(0.0008741923521711215) - rollout_corr/rollout_ppl:np.float64(1.6145951971411705) - rollout_corr/rollout_log_ppl:np.float64(0.46485281962668523) - rollout_corr/log_ppl_diff:np.float64(0.0011402198288124055) - rollout_corr/log_ppl_abs_diff:np.float64(0.001965065923286602) - rollout_corr/log_ppl_diff_max:np.float64(0.008194833993911743) - rollout_corr/log_ppl_diff_min:np.float64(-0.005479037761688232) - rollout_corr/ppl_ratio:np.float64(1.0011436399072409) - rollout_corr/chi2_token:np.float64(0.0012568566016852856) - rollout_corr/chi2_seq:np.float64(0.40265263244509697) - actor/pg_loss:np.float64(0.006021094392053783) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005619370990643802) - actor/ppo_kl:np.float64(0.0005166811963022155) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(12667.028565393994) - self_distillation/token_reweight_w_std:np.float64(223391.28332038334) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007588975131512) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12597810916486196) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.35431264340877533) - perf/mfu/actor:np.float64(0.050087304585172576) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.9956817626953) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005755655933171511 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005755655933171511 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:361.90625 - respon
(TaskRunner pid=2255397) 
Training Progress:  47%|████▋     | 47/100 [42:36<38:37, 43.73s/it]
(TaskRunner pid=2255397) step:48 - global_seqlen/min:16138 - global_seqlen/max:20539 - global_seqlen/minmax_diff:4401 - global_seqlen/balanced_min:19360 - global_seqlen/balanced_max:19367 - global_seqlen/mean:19365.625 - actor/entropy:0.45748016238212585 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.423759400844574 - training/rollout_probs_diff_mean:0.00517674908041954 - training/rollout_probs_diff_std:0.011389809660613537 - training/rollout_actor_probs_pearson_corr:0.9992020130157471 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.0000624656677246 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.4522043279139325e-05 - rollout_corr/rollout_is_max:1.9825339317321777 - rollout_corr/rollout_is_min:0.29720374941825867 - rollout_corr/rollout_is_std:0.038425594568252563 - rollout_corr/rollout_is_eff_sample_size:0.9985258883712781 - rollout_corr/rollout_is_seq_mean:1.0000547170639038 - rollout_corr/rollout_is_seq_std:0.0022232853807508945 - rollout_corr/rollout_is_seq_max:1.0069233179092407 - rollout_corr/rollout_is_seq_min:0.9934024810791016 - rollout_corr/rollout_is_seq_max_deviation:0.006923317909240723 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6189399659633636) - rollout_corr/training_log_ppl:np.float64(0.47103061771485955) - rollout_corr/kl:np.float64(0.0008533685957203829) - rollout_corr/k3_kl:np.float64(0.0008472814761262271) - rollout_corr/rollout_ppl:np.float64(1.6175275961868465) - rollout_corr/rollout_log_ppl:np.float64(0.47017724718898535) - rollout_corr/log_ppl_diff:np.float64(0.0008533705258741975) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018438339466229081) - rollout_corr/log_ppl_diff_max:np.float64(0.009492874145507812) - rollout_corr/log_ppl_diff_min:np.float64(-0.004758328199386597) - rollout_corr/ppl_ratio:np.float64(1.000856165541336) - rollout_corr/chi2_token:np.float64(0.0016905255615711212) - rollout_corr/chi2_seq:np.float64(0.5319139340426773) - actor/pg_loss:np.float64(0.004102788749150932) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002375331782786816) - actor/ppo_kl:np.float64(0.00012016280814464153) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.671875) - self_distillation/token_reweight_w_mean:np.float64(1381.5336034777574) - self_distillation/token_reweight_w_std:np.float64(20333.09212728584) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9975523005705327) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11302358307875693) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.2960100993514061) - perf/mfu/actor:np.float64(0.04606089601890587) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.1104965209961) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:0 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0013794433325529099 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0013794433325529099 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mea
(TaskRunner pid=2255397) 
Training Progress:  48%|████▊     | 48/100 [43:15<36:44, 42.40s/it]
(TaskRunner pid=2255397) step:49 - global_seqlen/min:19946 - global_seqlen/max:22692 - global_seqlen/minmax_diff:2746 - global_seqlen/balanced_min:21730 - global_seqlen/balanced_max:21733 - global_seqlen/mean:21731.75 - actor/entropy:0.4979252517223358 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43991324305534363 - training/rollout_probs_diff_mean:0.004984842147678137 - training/rollout_probs_diff_std:0.01066806074231863 - training/rollout_actor_probs_pearson_corr:0.9993179440498352 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:0.9998939037322998 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.758192391018383e-06 - rollout_corr/rollout_is_max:1.9085274934768677 - rollout_corr/rollout_is_min:0.12017349898815155 - rollout_corr/rollout_is_std:0.03726325184106827 - rollout_corr/rollout_is_eff_sample_size:0.9986130189557251 - rollout_corr/rollout_is_seq_mean:0.9998945593833923 - rollout_corr/rollout_is_seq_std:0.001821837155148387 - rollout_corr/rollout_is_seq_max:1.005088448524475 - rollout_corr/rollout_is_seq_min:0.9940054416656494 - rollout_corr/rollout_is_seq_max_deviation:0.005994558334350586 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6754766842350364) - rollout_corr/training_log_ppl:np.float64(0.5048167807399295) - rollout_corr/kl:np.float64(0.0009508746554258707) - rollout_corr/k3_kl:np.float64(0.0007735003869697721) - rollout_corr/rollout_ppl:np.float64(1.6738910162821412) - rollout_corr/rollout_log_ppl:np.float64(0.503865905397106) - rollout_corr/log_ppl_diff:np.float64(0.000950875342823565) - rollout_corr/log_ppl_abs_diff:np.float64(0.001617251429706812) - rollout_corr/log_ppl_diff_max:np.float64(0.008522868156433105) - rollout_corr/log_ppl_diff_min:np.float64(-0.00493285059928894) - rollout_corr/ppl_ratio:np.float64(1.0009531062096357) - rollout_corr/chi2_token:np.float64(0.0011934449430555105) - rollout_corr/chi2_seq:np.float64(0.1550909043289721) - actor/pg_loss:np.float64(0.005771090276539326) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002795115901790268) - actor/ppo_kl:np.float64(0.00010383528175317291) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(4803.391529331682) - self_distillation/token_reweight_w_std:np.float64(100370.09618178767) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001389476004988) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10803735570516437) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.3376929610967636) - perf/mfu/actor:np.float64(0.05224311765341433) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.75490951538086) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:0 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0023309881798923016 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0023309881798923016 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:400.304687
(TaskRunner pid=2255397) 
Training Progress:  49%|████▉     | 49/100 [43:55<35:21, 41.59s/it]
(TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50}
(TaskRunner pid=2255397) validation generation end
(TaskRunner pid=2255397) [prompt] system
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2255397) <reasoning>
(TaskRunner pid=2255397) ...
(TaskRunner pid=2255397) </reasoning>
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) ...
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) A
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) user
(TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2255397) Please reason step by step.
(TaskRunner pid=2255397) assistant
(TaskRunner pid=2255397) <think>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) </think>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) [response] <reasoning>
(TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to examine the structure of the product and compare it with the structures of the given options.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) The product has a nitrogen-containing ring with a substituent group: "Cc2ccc(Cl)cc2C(F)(F)F". This suggests that the molecule contains a chlorinated aromatic ring attached to a trifluoromethyl group, which is connected to a nitrogen-containing ring.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Now, let's evaluate the options:
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) - **Option A**: Contains a structure with a nitrogen ring and a chlorinated aromatic ring connected to a trifluoromethyl group. The structure is very similar to the product, with the correct substituents and connectivity.
(TaskRunner pid=2255397) - **Option B**: Contains a chlorinated nitrogen ring but lacks the trifluoromethyl group and the correct aromatic ring structure.
(TaskRunner pid=2255397) - **Option C**: Contains a hydroxyl group and a different substituent, not matching the product.
(TaskRunner pid=2255397) - **Option D**: Contains a different ring structure and lacks the trifluoromethyl group and the correct aromatic ring.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Only **Option A** matches the structure and substituents of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1".
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) </reasoning>
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) A
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) [ground_truth] A
(TaskRunner pid=2255397) [score] 1.0
(TaskRunner pid=2255397) [acc] 1.0
(TaskRunner pid=2255397) [pred] A
(TaskRunner pid=2255397) [incorrect_format] 1
(TaskRunner pid=2255397) [feedback] 
(TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50
(TaskRunner pid=2255397) step:50 - global_seqlen/min:19045 - global_seqlen/max:30156 - global_seqlen/minmax_diff:11111 - global_seqlen/balanced_min:21698 - global_seqlen/balanced_max:21699 - global_seqlen/mean:21698.5 - actor/entropy:0.39573362469673157 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24491804838180542 - training/rollout_probs_diff_mean:0.004589053802192211 - training/rollout_probs_diff_std:0.01072707585990429 - training/rollout_actor_probs_pearson_corr:0.9992213845252991 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.0000442266464233 - rollout_corr/rollout_is_ratio_fraction_high:9.806613888940774e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.004254102706909 - rollout_corr/rollout_is_min:0.6065316796302795 - rollout_corr/rollout_is_std:0.03603039309382439 - rollout_corr/rollout_is_eff_sample_size:0.9987034939469002 - rollout_corr/rollout_is_seq_mean:1.000090479850769 - rollout_corr/rollout_is_seq_std:0.0021530245430767536 - rollout_corr/rollout_is_seq_max:1.0064139366149902 - rollout_corr/rollout_is_seq_min:0.991096556186676 - rollout_corr/rollout_is_seq_max_deviation:0.008903443813323975 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.524656287394464) - rollout_corr/training_log_ppl:np.float64(0.4108033778902609) - rollout_corr/kl:np.float64(0.0007370334822254243) - rollout_corr/k3_kl:np.float64(0.0008599899813646061) - rollout_corr/rollout_ppl:np.float64(1.5234609101898968) - rollout_corr/rollout_log_ppl:np.float64(0.41006634262157604) - rollout_corr/log_ppl_diff:np.float64(0.0007370352686848491) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018268449639435858) - rollout_corr/log_ppl_diff_max:np.float64(0.00987979769706726) - rollout_corr/log_ppl_diff_min:np.float64(-0.00624692440032959) - rollout_corr/ppl_ratio:np.float64(1.000740017509088) - rollout_corr/chi2_token:np.float64(0.0019624873530119658) - rollout_corr/chi2_seq:np.float64(0.6737326441798359) - actor/pg_loss:np.float64(0.00585430464707315) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007581320141980541) - actor/ppo_kl:np.float64(0.00011956099752552518) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(4577.494679181604) - self_distillation/token_reweight_w_std:np.float64(75540.59034832189) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9987221618648618) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11705390218412504) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.3779703937470913) - perf/mfu/actor:np.float64(0.0522593230068327) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.6965789794922) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.66875) - val-aux/sciknoweval/reward/std@16:np.float64(0.17551919057486814) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7372904761904763) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13660825245257055) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6003428571428571) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15818906699634022) - val-aux/sciknoweval/rew
(TaskRunner pid=2255397) 
Training Progress:  50%|█████     | 50/100 [47:02<1:11:07, 85.35s/it]
(TaskRunner pid=2255397) step:51 - global_seqlen/min:19988 - global_seqlen/max:24236 - global_seqlen/minmax_diff:4248 - global_seqlen/balanced_min:21993 - global_seqlen/balanced_max:22000 - global_seqlen/mean:21997.625 - actor/entropy:0.47643905878067017 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6292527914047241 - training/rollout_probs_diff_mean:0.004974652547389269 - training/rollout_probs_diff_std:0.010949385352432728 - training/rollout_actor_probs_pearson_corr:0.9992679953575134 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.5546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.000044584274292 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0419596947031096e-05 - rollout_corr/rollout_is_max:1.6338132619857788 - rollout_corr/rollout_is_min:0.19046327471733093 - rollout_corr/rollout_is_std:0.03751512989401817 - rollout_corr/rollout_is_eff_sample_size:0.998594711950472 - rollout_corr/rollout_is_seq_mean:0.9999472498893738 - rollout_corr/rollout_is_seq_std:0.002286979230120778 - rollout_corr/rollout_is_seq_max:1.0055614709854126 - rollout_corr/rollout_is_seq_min:0.9931483864784241 - rollout_corr/rollout_is_seq_max_deviation:0.006851613521575928 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6695182812400162) - rollout_corr/training_log_ppl:np.float64(0.49964700039708987) - rollout_corr/kl:np.float64(0.0010526214147859037) - rollout_corr/k3_kl:np.float64(0.0008514240415706809) - rollout_corr/rollout_ppl:np.float64(1.6677322825416923) - rollout_corr/rollout_log_ppl:np.float64(0.49859437631675974) - rollout_corr/log_ppl_diff:np.float64(0.0010526240803301334) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019851572578772902) - rollout_corr/log_ppl_diff_max:np.float64(0.008082211017608643) - rollout_corr/log_ppl_diff_min:np.float64(-0.008058786392211914) - rollout_corr/ppl_ratio:np.float64(1.0010559679940343) - rollout_corr/chi2_token:np.float64(0.0013342886231839657) - rollout_corr/chi2_seq:np.float64(1.1409833203069866) - actor/pg_loss:np.float64(0.005735161481425166) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006951400860089052) - actor/ppo_kl:np.float64(0.00024209431599508946) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5546875) - self_distillation/token_reweight_w_mean:np.float64(7385.579081017524) - self_distillation/token_reweight_w_std:np.float64(136267.17127367936) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0020049593877047) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11803711034008302) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.3889169320464134) - perf/mfu/actor:np.float64(0.053154140926217126) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.41073989868164) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:0 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006512248073704541 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006512248073704541 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_leng
(TaskRunner pid=2255397) 
Training Progress:  51%|█████     | 51/100 [47:42<58:37, 71.78s/it]  
(TaskRunner pid=2255397) step:52 - global_seqlen/min:16418 - global_seqlen/max:23760 - global_seqlen/minmax_diff:7342 - global_seqlen/balanced_min:20391 - global_seqlen/balanced_max:20400 - global_seqlen/mean:20398.0 - actor/entropy:0.4327685832977295 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4141288995742798 - training/rollout_probs_diff_mean:0.004852033220231533 - training/rollout_probs_diff_std:0.011078495532274246 - training/rollout_actor_probs_pearson_corr:0.9992164969444275 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.61328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:0.999881386756897 - rollout_corr/rollout_is_ratio_fraction_high:1.0429704161651898e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.107421636581421 - rollout_corr/rollout_is_min:0.5139061212539673 - rollout_corr/rollout_is_std:0.03718959912657738 - rollout_corr/rollout_is_eff_sample_size:0.9986184874121644 - rollout_corr/rollout_is_seq_mean:0.9998266696929932 - rollout_corr/rollout_is_seq_std:0.0023117384407669306 - rollout_corr/rollout_is_seq_max:1.010014533996582 - rollout_corr/rollout_is_seq_min:0.9893986582756042 - rollout_corr/rollout_is_seq_max_deviation:0.010601341724395752 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5747977024875581) - rollout_corr/training_log_ppl:np.float64(0.44207731005735695) - rollout_corr/kl:np.float64(0.0007742779266513011) - rollout_corr/k3_kl:np.float64(0.0008074032141394127) - rollout_corr/rollout_ppl:np.float64(1.5735519244335592) - rollout_corr/rollout_log_ppl:np.float64(0.44130302700796165) - rollout_corr/log_ppl_diff:np.float64(0.0007742830493953079) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017560763808432966) - rollout_corr/log_ppl_diff_max:np.float64(0.011528313159942627) - rollout_corr/log_ppl_diff_min:np.float64(-0.008214741945266724) - rollout_corr/ppl_ratio:np.float64(1.0007772489916533) - rollout_corr/chi2_token:np.float64(0.0016987936105579138) - rollout_corr/chi2_seq:np.float64(0.6159643346909434) - actor/pg_loss:np.float64(0.004326079739257693) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003785648150369525) - actor/ppo_kl:np.float64(-0.00013583274305517534) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.61328125) - self_distillation/token_reweight_w_mean:np.float64(23087.692809279542) - self_distillation/token_reweight_w_std:np.float64(411971.67904203804) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0013474761508405) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10722107632318512) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.31624583154916763) - perf/mfu/actor:np.float64(0.04994189212775807) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.2727928161621) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:0 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00683667091652751 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00683667091652751 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:374.5
(TaskRunner pid=2255397) 
Training Progress:  52%|█████▏    | 52/100 [48:21<49:26, 61.79s/it]
(TaskRunner pid=2255397) step:53 - global_seqlen/min:19241 - global_seqlen/max:23446 - global_seqlen/minmax_diff:4205 - global_seqlen/balanced_min:21257 - global_seqlen/balanced_max:21274 - global_seqlen/mean:21261.625 - actor/entropy:0.4501931071281433 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2294929027557373 - training/rollout_probs_diff_mean:0.0048794192261993885 - training/rollout_probs_diff_std:0.010888302698731422 - training/rollout_actor_probs_pearson_corr:0.9992465376853943 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.62109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:0.9998956918716431 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.1179934265092015e-05 - rollout_corr/rollout_is_max:1.6338140964508057 - rollout_corr/rollout_is_min:0.37676042318344116 - rollout_corr/rollout_is_std:0.036510247737169266 - rollout_corr/rollout_is_eff_sample_size:0.9986683006749928 - rollout_corr/rollout_is_seq_mean:0.9998424053192139 - rollout_corr/rollout_is_seq_std:0.0020514947827905416 - rollout_corr/rollout_is_seq_max:1.0057796239852905 - rollout_corr/rollout_is_seq_min:0.9921916127204895 - rollout_corr/rollout_is_seq_max_deviation:0.007808387279510498 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6163378930650651) - rollout_corr/training_log_ppl:np.float64(0.4686294782150071) - rollout_corr/kl:np.float64(0.000762243592463463) - rollout_corr/k3_kl:np.float64(0.000810159646334796) - rollout_corr/rollout_ppl:np.float64(1.6150716617703438) - rollout_corr/rollout_log_ppl:np.float64(0.46786723379045725) - rollout_corr/log_ppl_diff:np.float64(0.0007622444245498627) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019291590724606067) - rollout_corr/log_ppl_diff_max:np.float64(0.008153051137924194) - rollout_corr/log_ppl_diff_min:np.float64(-0.005449235439300537) - rollout_corr/ppl_ratio:np.float64(1.000765364151448) - rollout_corr/chi2_token:np.float64(0.001715675462037325) - rollout_corr/chi2_seq:np.float64(0.6910448384005576) - actor/pg_loss:np.float64(0.0023863104870542884) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002726298084780865) - actor/ppo_kl:np.float64(-0.00011610987491650349) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_w_mean:np.float64(1507.4308137190528) - self_distillation/token_reweight_w_std:np.float64(33027.9989340055) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995625410228968) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06903543486259878) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.21907289698719978) - perf/mfu/actor:np.float64(0.051939957351732395) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.24185180664062) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:0 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003284213598817587 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003284213598817587 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_le
(TaskRunner pid=2255397) 
Training Progress:  53%|█████▎    | 53/100 [49:01<43:26, 55.45s/it]
(TaskRunner pid=2255397) step:54 - global_seqlen/min:16771 - global_seqlen/max:23434 - global_seqlen/minmax_diff:6663 - global_seqlen/balanced_min:20121 - global_seqlen/balanced_max:20134 - global_seqlen/mean:20129.375 - actor/entropy:0.39255428314208984 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2144627422094345 - training/rollout_probs_diff_mean:0.004672019276767969 - training/rollout_probs_diff_std:0.011115854606032372 - training/rollout_actor_probs_pearson_corr:0.9991840720176697 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.5546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9999427795410156 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.969622254371643 - rollout_corr/rollout_is_min:0.5503029227256775 - rollout_corr/rollout_is_std:0.03688544034957886 - rollout_corr/rollout_is_eff_sample_size:0.998641194034968 - rollout_corr/rollout_is_seq_mean:0.9999832510948181 - rollout_corr/rollout_is_seq_std:0.002377620432525873 - rollout_corr/rollout_is_seq_max:1.0109039545059204 - rollout_corr/rollout_is_seq_min:0.9921522736549377 - rollout_corr/rollout_is_seq_max_deviation:0.01090395450592041 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5304169207811356) - rollout_corr/training_log_ppl:np.float64(0.4121860411542002) - rollout_corr/kl:np.float64(0.0008754907883350427) - rollout_corr/k3_kl:np.float64(0.0008680712330715323) - rollout_corr/rollout_ppl:np.float64(1.5290372776798904) - rollout_corr/rollout_log_ppl:np.float64(0.411310548952315) - rollout_corr/log_ppl_diff:np.float64(0.0008754922018852085) - rollout_corr/log_ppl_abs_diff:np.float64(0.002070396934868768) - rollout_corr/log_ppl_diff_max:np.float64(0.016439110040664673) - rollout_corr/log_ppl_diff_min:np.float64(-0.00867973268032074) - rollout_corr/ppl_ratio:np.float64(1.0008794867899269) - rollout_corr/chi2_token:np.float64(0.0017218261491507292) - rollout_corr/chi2_seq:np.float64(0.5381895105820149) - actor/pg_loss:np.float64(0.006413624971173704) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005543413315081125) - actor/ppo_kl:np.float64(9.862682850325655e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5546875) - self_distillation/token_reweight_w_mean:np.float64(21560.225262402324) - self_distillation/token_reweight_w_std:np.float64(340430.1742267248) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004913297016174) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10845586232608184) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.3543579652905464) - perf/mfu/actor:np.float64(0.04868076222396018) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.25546646118164) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:0 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011497573927044868 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011497573927044868 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:358.26171875 - response_le
(TaskRunner pid=2255397) 
Training Progress:  54%|█████▍    | 54/100 [49:42<39:13, 51.16s/it]
(TaskRunner pid=2255397) step:55 - global_seqlen/min:19138 - global_seqlen/max:21905 - global_seqlen/minmax_diff:2767 - global_seqlen/balanced_min:20379 - global_seqlen/balanced_max:20392 - global_seqlen/mean:20387.375 - actor/entropy:0.46607786417007446 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44003328680992126 - training/rollout_probs_diff_mean:0.005262596998363733 - training/rollout_probs_diff_std:0.011458019725978374 - training/rollout_actor_probs_pearson_corr:0.9992002844810486 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.0000497102737427 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.523318446241319e-05 - rollout_corr/rollout_is_max:1.6244064569473267 - rollout_corr/rollout_is_min:0.396142840385437 - rollout_corr/rollout_is_std:0.03780318796634674 - rollout_corr/rollout_is_eff_sample_size:0.99857307726417 - rollout_corr/rollout_is_seq_mean:0.9999815225601196 - rollout_corr/rollout_is_seq_std:0.002393631497398019 - rollout_corr/rollout_is_seq_max:1.0126123428344727 - rollout_corr/rollout_is_seq_min:0.9941840171813965 - rollout_corr/rollout_is_seq_max_deviation:0.012612342834472656 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.630201779305935) - rollout_corr/training_log_ppl:np.float64(0.47879417467629537) - rollout_corr/kl:np.float64(0.0010309481362873463) - rollout_corr/k3_kl:np.float64(0.0008061554526079817) - rollout_corr/rollout_ppl:np.float64(1.6284840498119593) - rollout_corr/rollout_log_ppl:np.float64(0.4777632247423753) - rollout_corr/log_ppl_diff:np.float64(0.0010309499339200556) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020579289994202554) - rollout_corr/log_ppl_diff_max:np.float64(0.007941722869873047) - rollout_corr/log_ppl_diff_min:np.float64(-0.0057179927825927734) - rollout_corr/ppl_ratio:np.float64(1.0010342469904572) - rollout_corr/chi2_token:np.float64(0.00116553227417171) - rollout_corr/chi2_seq:np.float64(0.674807739444077) - actor/pg_loss:np.float64(0.004637886886484921) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022305224865704076) - actor/ppo_kl:np.float64(0.0002447084712073888) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(5661.525033625308) - self_distillation/token_reweight_w_std:np.float64(108029.05970195204) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004851918201894) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08942344659590162) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.34092599898576736) - perf/mfu/actor:np.float64(0.04916186108901947) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.24573516845703) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:0 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0027775494381785393 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0027775494381785393 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:332.6054
(TaskRunner pid=2255397) 
Training Progress:  55%|█████▌    | 55/100 [50:22<35:47, 47.71s/it]
(TaskRunner pid=2255397) step:56 - global_seqlen/min:16165 - global_seqlen/max:23611 - global_seqlen/minmax_diff:7446 - global_seqlen/balanced_min:19603 - global_seqlen/balanced_max:19611 - global_seqlen/mean:19608.875 - actor/entropy:0.42714759707450867 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2066904902458191 - training/rollout_probs_diff_mean:0.004840976092964411 - training/rollout_probs_diff_std:0.011059464886784554 - training/rollout_actor_probs_pearson_corr:0.9992184638977051 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.6484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:1.0001850128173828 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7041244506835938 - rollout_corr/rollout_is_min:0.5909921526908875 - rollout_corr/rollout_is_std:0.03720882534980774 - rollout_corr/rollout_is_eff_sample_size:0.9986178930118242 - rollout_corr/rollout_is_seq_mean:1.0002026557922363 - rollout_corr/rollout_is_seq_std:0.0023563196882605553 - rollout_corr/rollout_is_seq_max:1.0108153820037842 - rollout_corr/rollout_is_seq_min:0.9901830554008484 - rollout_corr/rollout_is_seq_max_deviation:0.01081538200378418 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5864489898085594) - rollout_corr/training_log_ppl:np.float64(0.4489033415738959) - rollout_corr/kl:np.float64(0.0005826077157036202) - rollout_corr/k3_kl:np.float64(0.0008374885150033151) - rollout_corr/rollout_ppl:np.float64(1.5854855361394584) - rollout_corr/rollout_log_ppl:np.float64(0.4483207310840953) - rollout_corr/log_ppl_diff:np.float64(0.0005826104898005724) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019630271708592772) - rollout_corr/log_ppl_diff_max:np.float64(0.011552989482879639) - rollout_corr/log_ppl_diff_min:np.float64(-0.00930735468864441) - rollout_corr/ppl_ratio:np.float64(1.0005861730314791) - rollout_corr/chi2_token:np.float64(0.0022083548828959465) - rollout_corr/chi2_seq:np.float64(1.1513643299695104) - actor/pg_loss:np.float64(0.0053109838627278805) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004679971348195977) - actor/ppo_kl:np.float64(4.2931282393965375e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_w_mean:np.float64(133.22875495348126) - self_distillation/token_reweight_w_std:np.float64(2217.535897645983) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0020651405211538) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08932297839783132) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.3138376846909523) - perf/mfu/actor:np.float64(0.04696252700818672) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.17254257202148) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:0 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004433032590895891 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.004433032590895891 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:347.62109375 - re
(TaskRunner pid=2255397) 
Training Progress:  56%|█████▌    | 56/100 [51:02<33:15, 45.35s/it]
(TaskRunner pid=2255397) step:57 - global_seqlen/min:17187 - global_seqlen/max:22734 - global_seqlen/minmax_diff:5547 - global_seqlen/balanced_min:19426 - global_seqlen/balanced_max:19436 - global_seqlen/mean:19433.875 - actor/entropy:0.419066846370697 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45647868514060974 - training/rollout_probs_diff_mean:0.005257493816316128 - training/rollout_probs_diff_std:0.012042389251291752 - training/rollout_actor_probs_pearson_corr:0.9990561008453369 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9998766779899597 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.874779187957756e-05 - rollout_corr/rollout_is_max:1.9517626762390137 - rollout_corr/rollout_is_min:0.412737101316452 - rollout_corr/rollout_is_std:0.039478376507759094 - rollout_corr/rollout_is_eff_sample_size:0.9984436453453855 - rollout_corr/rollout_is_seq_mean:0.9998682141304016 - rollout_corr/rollout_is_seq_std:0.002758133690804243 - rollout_corr/rollout_is_seq_max:1.010321855545044 - rollout_corr/rollout_is_seq_min:0.9932553768157959 - rollout_corr/rollout_is_seq_max_deviation:0.010321855545043945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5485623660497367) - rollout_corr/training_log_ppl:np.float64(0.4282529861666262) - rollout_corr/kl:np.float64(0.0008480568455153659) - rollout_corr/k3_kl:np.float64(0.0008880625406391118) - rollout_corr/rollout_ppl:np.float64(1.5472203940153122) - rollout_corr/rollout_log_ppl:np.float64(0.42740493212477304) - rollout_corr/log_ppl_diff:np.float64(0.0008480540418531746) - rollout_corr/log_ppl_abs_diff:np.float64(0.002164474077289924) - rollout_corr/log_ppl_diff_max:np.float64(0.007681757211685181) - rollout_corr/log_ppl_diff_min:np.float64(-0.008211076259613037) - rollout_corr/ppl_ratio:np.float64(1.0008519866969436) - rollout_corr/chi2_token:np.float64(0.0018828820902854204) - rollout_corr/chi2_seq:np.float64(1.0425616323482245) - actor/pg_loss:np.float64(0.002906394307501614) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00029778121393064794) - actor/ppo_kl:np.float64(-0.00014163734993388744) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(3090.494025749387) - self_distillation/token_reweight_w_std:np.float64(77747.68742029642) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994963144417852) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06993347342358902) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.22349200583994389) - perf/mfu/actor:np.float64(0.04662792309159809) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.20368194580078) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:0 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0013542745728045702 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0013542745728045702 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_l
(TaskRunner pid=2255397) 
Training Progress:  57%|█████▋    | 57/100 [51:42<31:24, 43.83s/it]
(TaskRunner pid=2255397) step:58 - global_seqlen/min:17591 - global_seqlen/max:22208 - global_seqlen/minmax_diff:4617 - global_seqlen/balanced_min:19936 - global_seqlen/balanced_max:19980 - global_seqlen/mean:19944.0 - actor/entropy:0.39805784821510315 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5443886518478394 - training/rollout_probs_diff_mean:0.0047958018258214 - training/rollout_probs_diff_std:0.011325978673994541 - training/rollout_actor_probs_pearson_corr:0.9991068243980408 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999314546585083 - rollout_corr/rollout_is_ratio_fraction_high:1.1276500117674004e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1276500117674004e-05 - rollout_corr/rollout_is_max:2.245145320892334 - rollout_corr/rollout_is_min:0.3792571425437927 - rollout_corr/rollout_is_std:0.037710048258304596 - rollout_corr/rollout_is_eff_sample_size:0.9985797339908784 - rollout_corr/rollout_is_seq_mean:0.9998851418495178 - rollout_corr/rollout_is_seq_std:0.002318696118891239 - rollout_corr/rollout_is_seq_max:1.007555365562439 - rollout_corr/rollout_is_seq_min:0.9894842505455017 - rollout_corr/rollout_is_seq_max_deviation:0.010515749454498291 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5350726060569286) - rollout_corr/training_log_ppl:np.float64(0.4149510355782695) - rollout_corr/kl:np.float64(0.0008157491595808253) - rollout_corr/k3_kl:np.float64(0.0008106242991630097) - rollout_corr/rollout_ppl:np.float64(1.533764130435884) - rollout_corr/rollout_log_ppl:np.float64(0.41413528152043) - rollout_corr/log_ppl_diff:np.float64(0.0008157540578395128) - rollout_corr/log_ppl_abs_diff:np.float64(0.00186522479634732) - rollout_corr/log_ppl_diff_max:np.float64(0.012087315320968628) - rollout_corr/log_ppl_diff_min:np.float64(-0.005733311176300049) - rollout_corr/ppl_ratio:np.float64(1.0008189915679395) - rollout_corr/chi2_token:np.float64(0.0016144579276442528) - rollout_corr/chi2_seq:np.float64(0.5739311659708619) - actor/pg_loss:np.float64(0.0032648365013301373) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039790487971913535) - actor/ppo_kl:np.float64(-6.927822899172043e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_w_mean:np.float64(20.40596836898476) - self_distillation/token_reweight_w_std:np.float64(398.28047909680754) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010924299713224) - self_distillation/token_reweight_w_clip_frac:np.float64(0.060014044371200725) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.22483210638165474) - perf/mfu/actor:np.float64(0.048109296920814684) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.0834503173828) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:0 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00522806728258729 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00522806728258729 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/m
(TaskRunner pid=2255397) 
Training Progress:  58%|█████▊    | 58/100 [52:22<29:54, 42.72s/it]
(TaskRunner pid=2255397) step:59 - global_seqlen/min:16516 - global_seqlen/max:23158 - global_seqlen/minmax_diff:6642 - global_seqlen/balanced_min:19285 - global_seqlen/balanced_max:19288 - global_seqlen/mean:19286.5 - actor/entropy:0.39392659068107605 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105931282043457 - training/rollout_probs_diff_mean:0.004885942675173283 - training/rollout_probs_diff_std:0.01138182170689106 - training/rollout_actor_probs_pearson_corr:0.9991214275360107 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65234375 - self_distillation/correct_sample_fraction:0.5390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65234375 - rollout_corr/rollout_is_mean:0.9998358488082886 - rollout_corr/rollout_is_ratio_fraction_high:1.1690437531797215e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1690437531797215e-05 - rollout_corr/rollout_is_max:3.892078161239624 - rollout_corr/rollout_is_min:0.4614630937576294 - rollout_corr/rollout_is_std:0.03779766708612442 - rollout_corr/rollout_is_eff_sample_size:0.9985728395255719 - rollout_corr/rollout_is_seq_mean:0.999951183795929 - rollout_corr/rollout_is_seq_std:0.002800993388518691 - rollout_corr/rollout_is_seq_max:1.0143204927444458 - rollout_corr/rollout_is_seq_min:0.9928278923034668 - rollout_corr/rollout_is_seq_max_deviation:0.0143204927444458 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5359033830463886) - rollout_corr/training_log_ppl:np.float64(0.4180586190195754) - rollout_corr/kl:np.float64(0.0011108192077600165) - rollout_corr/k3_kl:np.float64(0.0008730394356462057) - rollout_corr/rollout_ppl:np.float64(1.5341403023339808) - rollout_corr/rollout_log_ppl:np.float64(0.41694779935642146) - rollout_corr/log_ppl_diff:np.float64(0.0011108196631539613) - rollout_corr/log_ppl_abs_diff:np.float64(0.002291953394887969) - rollout_corr/log_ppl_diff_max:np.float64(0.008277744054794312) - rollout_corr/log_ppl_diff_min:np.float64(-0.007644861936569214) - rollout_corr/ppl_ratio:np.float64(1.0011151337530464) - rollout_corr/chi2_token:np.float64(0.0013092008884996176) - rollout_corr/chi2_seq:np.float64(0.4192327775526792) - actor/pg_loss:np.float64(0.0025457896990701556) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003396774759494292) - actor/ppo_kl:np.float64(0.0002558892238049282) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5390625) - self_distillation/token_reweight_w_mean:np.float64(9165.766619424336) - self_distillation/token_reweight_w_std:np.float64(181822.19756039116) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998559767846018) - self_distillation/token_reweight_w_clip_frac:np.float64(0.057435350317973644) - self_distillation/empty_target_batch:np.float64(0.34765625) - actor/grad_norm:np.float64(0.21796750277280807) - perf/mfu/actor:np.float64(0.0472734036832577) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(198.940327167511) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:0 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0026581131387501955 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0026581131387501955 - critic/returns/max:0.875 - critic/returns/min:-0.875 - r
(TaskRunner pid=2255397) 
Training Progress:  59%|█████▉    | 59/100 [53:02<28:35, 41.85s/it]
(TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60}
(TaskRunner pid=2255397) validation generation end
(TaskRunner pid=2255397) [prompt] system
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2255397) <reasoning>
(TaskRunner pid=2255397) ...
(TaskRunner pid=2255397) </reasoning>
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) ...
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) A
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) user
(TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2255397) Please reason step by step.
(TaskRunner pid=2255397) assistant
(TaskRunner pid=2255397) <think>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) </think>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) [response] <reasoning>
(TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to analyze the structure of the product and compare it with the structures of the given options.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) The product contains a nitrogen-containing ring with a substituent group: a chlorine-substituted benzene ring connected to a trifluoromethyl group (C(F)(F)F). This suggests that the correct reactant must contain a similar structure, specifically a nitrogen-containing ring with the same substituents.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Option A: "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" contains a nitrogen-containing ring with the exact substituents: a chlorine-substituted benzene ring and a trifluoromethyl group. This matches the structure of the product exactly.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Option B: Contains a nitrogen but not the correct substituents.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Option C: Contains a hydroxyl group and a different substituent, not matching the product.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Option D: Contains a different structure with a carbon chain and a fluorinated group, not matching the product.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Therefore, the only correct reactant is Option A.
(TaskRunner pid=2255397) </reasoning>
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) A
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) [ground_truth] A
(TaskRunner pid=2255397) [score] 1.0
(TaskRunner pid=2255397) [acc] 1.0
(TaskRunner pid=2255397) [pred] A
(TaskRunner pid=2255397) [incorrect_format] 1
(TaskRunner pid=2255397) [feedback] 
(TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60
(TaskRunner pid=2255397) step:60 - global_seqlen/min:15969 - global_seqlen/max:20753 - global_seqlen/minmax_diff:4784 - global_seqlen/balanced_min:19004 - global_seqlen/balanced_max:19008 - global_seqlen/mean:19006.625 - actor/entropy:0.40371620655059814 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8045270442962646 - training/rollout_probs_diff_mean:0.005058089271187782 - training/rollout_probs_diff_std:0.01212889701128006 - training/rollout_actor_probs_pearson_corr:0.9990420341491699 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.59375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:0.9997034668922424 - rollout_corr/rollout_is_ratio_fraction_high:1.1908589840459172e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.763435936183669e-05 - rollout_corr/rollout_is_max:2.487139940261841 - rollout_corr/rollout_is_min:0.0640438050031662 - rollout_corr/rollout_is_std:0.03801152855157852 - rollout_corr/rollout_is_eff_sample_size:0.9985564358356739 - rollout_corr/rollout_is_seq_mean:0.9996296167373657 - rollout_corr/rollout_is_seq_std:0.0025690472684800625 - rollout_corr/rollout_is_seq_max:1.0076467990875244 - rollout_corr/rollout_is_seq_min:0.9901750683784485 - rollout_corr/rollout_is_seq_max_deviation:0.009824931621551514 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.534271144773811) - rollout_corr/training_log_ppl:np.float64(0.41600528117851354) - rollout_corr/kl:np.float64(0.001141798264093552) - rollout_corr/k3_kl:np.float64(0.000918725037195145) - rollout_corr/rollout_ppl:np.float64(1.5324818124063313) - rollout_corr/rollout_log_ppl:np.float64(0.4148634799348656) - rollout_corr/log_ppl_diff:np.float64(0.001141801243647933) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023173787049017847) - rollout_corr/log_ppl_diff_max:np.float64(0.009830474853515625) - rollout_corr/log_ppl_diff_min:np.float64(-0.006625443696975708) - rollout_corr/ppl_ratio:np.float64(1.0011462338734418) - rollout_corr/chi2_token:np.float64(0.0013863001950085163) - rollout_corr/chi2_seq:np.float64(0.6778646670281887) - actor/pg_loss:np.float64(0.005434742895886302) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011875206650984182) - actor/ppo_kl:np.float64(-4.705816970229648e-05) - actor/pg_clipfrac_lower:np.float64(2.3182492441264912e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59375) - self_distillation/token_reweight_w_mean:np.float64(18888.49168556533) - self_distillation/token_reweight_w_std:np.float64(311593.95773308317) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0030431225895882) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10766072731348686) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.35879671573638916) - perf/mfu/actor:np.float64(0.045344773814958786) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.29048919677734) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6875) - val-aux/sciknoweval/reward/std@16:np.float64(0.16029546205916068) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7511857142857143) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13365319867771744) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6229333333333333) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1386095264154
(TaskRunner pid=2255397) 
Training Progress:  60%|██████    | 60/100 [56:05<56:00, 84.02s/it]
(TaskRunner pid=2255397) step:61 - global_seqlen/min:15598 - global_seqlen/max:23498 - global_seqlen/minmax_diff:7900 - global_seqlen/balanced_min:19661 - global_seqlen/balanced_max:19666 - global_seqlen/mean:19664.0 - actor/entropy:0.375679075717926 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102344214916229 - training/rollout_probs_diff_mean:0.00489032082259655 - training/rollout_probs_diff_std:0.011647011153399944 - training/rollout_actor_probs_pearson_corr:0.9990482330322266 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.66015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.00008225440979 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1082543096563313e-05 - rollout_corr/rollout_is_max:1.639464259147644 - rollout_corr/rollout_is_min:0.3959808051586151 - rollout_corr/rollout_is_std:0.03841473534703255 - rollout_corr/rollout_is_eff_sample_size:0.9985268392369275 - rollout_corr/rollout_is_seq_mean:1.000138759613037 - rollout_corr/rollout_is_seq_std:0.002528289332985878 - rollout_corr/rollout_is_seq_max:1.007035493850708 - rollout_corr/rollout_is_seq_min:0.993572473526001 - rollout_corr/rollout_is_seq_max_deviation:0.007035493850708008 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4903184147551656) - rollout_corr/training_log_ppl:np.float64(0.3903155774460174) - rollout_corr/kl:np.float64(0.0006912411112880434) - rollout_corr/k3_kl:np.float64(0.0009254292584728319) - rollout_corr/rollout_ppl:np.float64(1.4893131870776415) - rollout_corr/rollout_log_ppl:np.float64(0.38962433370761573) - rollout_corr/log_ppl_diff:np.float64(0.0006912437384016812) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020783296204172075) - rollout_corr/log_ppl_diff_max:np.float64(0.008693814277648926) - rollout_corr/log_ppl_diff_min:np.float64(-0.006119638681411743) - rollout_corr/ppl_ratio:np.float64(1.0006948560476303) - rollout_corr/chi2_token:np.float64(0.002362349070608616) - rollout_corr/chi2_seq:np.float64(0.9650198624003679) - actor/pg_loss:np.float64(0.005624711047858) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006064838689781027) - actor/ppo_kl:np.float64(1.730111737785478e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66015625) - self_distillation/token_reweight_w_mean:np.float64(5193.730784527026) - self_distillation/token_reweight_w_std:np.float64(104429.90018021135) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993646144866943) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08162092015845701) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.354709729552269) - perf/mfu/actor:np.float64(0.04738440538771457) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.02718353271484) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:1 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004777961410582066 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004777961410582066 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:352.46
(TaskRunner pid=2255397) 
Training Progress:  61%|██████    | 61/100 [56:44<45:50, 70.52s/it]
(TaskRunner pid=2255397) step:62 - global_seqlen/min:17175 - global_seqlen/max:22516 - global_seqlen/minmax_diff:5341 - global_seqlen/balanced_min:19524 - global_seqlen/balanced_max:19535 - global_seqlen/mean:19531.875 - actor/entropy:0.4294959306716919 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.20105724036693573 - training/rollout_probs_diff_mean:0.005539817735552788 - training/rollout_probs_diff_std:0.012213723734021187 - training/rollout_actor_probs_pearson_corr:0.9990189075469971 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0000041723251343 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9759001731872559 - rollout_corr/rollout_is_min:0.5684848427772522 - rollout_corr/rollout_is_std:0.04096618667244911 - rollout_corr/rollout_is_eff_sample_size:0.9983247021636537 - rollout_corr/rollout_is_seq_mean:0.9999959468841553 - rollout_corr/rollout_is_seq_std:0.0028718190733343363 - rollout_corr/rollout_is_seq_max:1.008870244026184 - rollout_corr/rollout_is_seq_min:0.987141489982605 - rollout_corr/rollout_is_seq_max_deviation:0.01285851001739502 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.584152389317751) - rollout_corr/training_log_ppl:np.float64(0.4482707906863652) - rollout_corr/kl:np.float64(0.001287838817653153) - rollout_corr/k3_kl:np.float64(0.0013142074097913792) - rollout_corr/rollout_ppl:np.float64(1.5821097679436207) - rollout_corr/rollout_log_ppl:np.float64(0.4469829489826225) - rollout_corr/log_ppl_diff:np.float64(0.0012878417037427425) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026649965438991785) - rollout_corr/log_ppl_diff_max:np.float64(0.014283061027526855) - rollout_corr/log_ppl_diff_min:np.float64(-0.010016441345214844) - rollout_corr/ppl_ratio:np.float64(1.0012943353503942) - rollout_corr/chi2_token:np.float64(0.002664147410541773) - rollout_corr/chi2_seq:np.float64(0.6478703722823411) - actor/pg_loss:np.float64(0.005159451276995242) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001237628416674852) - actor/ppo_kl:np.float64(0.0003923675134416271) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(9608.619164069882) - self_distillation/token_reweight_w_std:np.float64(147490.825859931) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000067490618676) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10762355217593722) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.41205330193042755) - perf/mfu/actor:np.float64(0.04771627973118822) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.9746551513672) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:1 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012106545269489288 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012106545269489288 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:311.12109375 - respo
(TaskRunner pid=2255397) 
Training Progress:  62%|██████▏   | 62/100 [57:23<38:49, 61.31s/it]
(TaskRunner pid=2255397) step:63 - global_seqlen/min:15986 - global_seqlen/max:20551 - global_seqlen/minmax_diff:4565 - global_seqlen/balanced_min:18758 - global_seqlen/balanced_max:18761 - global_seqlen/mean:18759.625 - actor/entropy:0.35515502095222473 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24133962392807007 - training/rollout_probs_diff_mean:0.004971614107489586 - training/rollout_probs_diff_std:0.012004957534372807 - training/rollout_actor_probs_pearson_corr:0.9989368915557861 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.53125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.0001026391983032 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.3019985999562778e-05 - rollout_corr/rollout_is_max:1.8590831756591797 - rollout_corr/rollout_is_min:0.4984112083911896 - rollout_corr/rollout_is_std:0.03878994286060333 - rollout_corr/rollout_is_eff_sample_size:0.998497719797901 - rollout_corr/rollout_is_seq_mean:1.000045895576477 - rollout_corr/rollout_is_seq_std:0.002819458954036236 - rollout_corr/rollout_is_seq_max:1.0079857110977173 - rollout_corr/rollout_is_seq_min:0.9885396957397461 - rollout_corr/rollout_is_seq_max_deviation:0.011460304260253906 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4885337855666876) - rollout_corr/training_log_ppl:np.float64(0.38630677238688804) - rollout_corr/kl:np.float64(0.0008763675811707117) - rollout_corr/k3_kl:np.float64(0.0011061657141908654) - rollout_corr/rollout_ppl:np.float64(1.4872424700297415) - rollout_corr/rollout_log_ppl:np.float64(0.3854304051492363) - rollout_corr/log_ppl_diff:np.float64(0.0008763672376517206) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025414098927285522) - rollout_corr/log_ppl_diff_max:np.float64(0.016067981719970703) - rollout_corr/log_ppl_diff_min:np.float64(-0.010530859231948853) - rollout_corr/ppl_ratio:np.float64(1.0008826348930597) - rollout_corr/chi2_token:np.float64(0.002648768713697791) - rollout_corr/chi2_seq:np.float64(1.621614676201716) - actor/pg_loss:np.float64(0.004201351199299097) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006837096566414402) - actor/ppo_kl:np.float64(9.11460165120559e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53125) - self_distillation/token_reweight_w_mean:np.float64(328.2442679554224) - self_distillation/token_reweight_w_std:np.float64(5757.5124940559035) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0031547287944704) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0877059873600956) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.3450646325945854) - perf/mfu/actor:np.float64(0.04625351205637495) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.83617782592773) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:1 - critic/score/mean:0.53125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001894407905638218 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001894407905638218 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:300.01953125 - response_len
(TaskRunner pid=2255397) 
Training Progress:  63%|██████▎   | 63/100 [58:02<33:33, 54.41s/it]
(TaskRunner pid=2255397) step:64 - global_seqlen/min:15100 - global_seqlen/max:20599 - global_seqlen/minmax_diff:5499 - global_seqlen/balanced_min:18082 - global_seqlen/balanced_max:18090 - global_seqlen/mean:18088.125 - actor/entropy:0.3758540451526642 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24836724996566772 - training/rollout_probs_diff_mean:0.005264882929623127 - training/rollout_probs_diff_std:0.012344393879175186 - training/rollout_actor_probs_pearson_corr:0.9989309310913086 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.0000416040420532 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.8669313905993477e-05 - rollout_corr/rollout_is_max:1.78631591796875 - rollout_corr/rollout_is_min:0.3966720402240753 - rollout_corr/rollout_is_std:0.04027950391173363 - rollout_corr/rollout_is_eff_sample_size:0.9983804272587137 - rollout_corr/rollout_is_seq_mean:1.0000438690185547 - rollout_corr/rollout_is_seq_std:0.0027673090808093548 - rollout_corr/rollout_is_seq_max:1.0073356628417969 - rollout_corr/rollout_is_seq_min:0.9913079738616943 - rollout_corr/rollout_is_seq_max_deviation:0.008692026138305664 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.488935824483633) - rollout_corr/training_log_ppl:np.float64(0.3868299189489335) - rollout_corr/kl:np.float64(0.0007672079092806428) - rollout_corr/k3_kl:np.float64(0.0009736951585637144) - rollout_corr/rollout_ppl:np.float64(1.4878136315383017) - rollout_corr/rollout_log_ppl:np.float64(0.38606271165190265) - rollout_corr/log_ppl_diff:np.float64(0.0007672072970308363) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023564034490846097) - rollout_corr/log_ppl_diff_max:np.float64(0.011275023221969604) - rollout_corr/log_ppl_diff_min:np.float64(-0.008923828601837158) - rollout_corr/ppl_ratio:np.float64(1.000771946972236) - rollout_corr/chi2_token:np.float64(0.002396456664428115) - rollout_corr/chi2_seq:np.float64(0.5857332549057901) - actor/pg_loss:np.float64(0.0037766702007502317) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00045624542644873145) - actor/ppo_kl:np.float64(-8.097473682511236e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_w_mean:np.float64(34646.944526499836) - self_distillation/token_reweight_w_std:np.float64(444681.0361109813) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998654320370406) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06305540914763696) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.2998654544353485) - perf/mfu/actor:np.float64(0.04428300784041782) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.82562017440796) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:1 - critic/score/mean:0.765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009668726474046707 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009668726474046707 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:272.
(TaskRunner pid=2255397) 
Training Progress:  64%|██████▍   | 64/100 [58:41<29:58, 49.97s/it]
(TaskRunner pid=2255397) step:65 - global_seqlen/min:15051 - global_seqlen/max:21236 - global_seqlen/minmax_diff:6185 - global_seqlen/balanced_min:18004 - global_seqlen/balanced_max:18008 - global_seqlen/mean:18006.625 - actor/entropy:0.3955051898956299 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1947164535522461 - training/rollout_probs_diff_mean:0.0055443341843783855 - training/rollout_probs_diff_std:0.012429744005203247 - training/rollout_actor_probs_pearson_corr:0.9989457726478577 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.71484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:0.9999876618385315 - rollout_corr/rollout_is_ratio_fraction_high:1.3995213521411642e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.031123399734497 - rollout_corr/rollout_is_min:0.504069447517395 - rollout_corr/rollout_is_std:0.041325442492961884 - rollout_corr/rollout_is_eff_sample_size:0.9982950004968502 - rollout_corr/rollout_is_seq_mean:0.9999707341194153 - rollout_corr/rollout_is_seq_std:0.002948605688288808 - rollout_corr/rollout_is_seq_max:1.009615182876587 - rollout_corr/rollout_is_seq_min:0.9913778305053711 - rollout_corr/rollout_is_seq_max_deviation:0.009615182876586914 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.525312386918813) - rollout_corr/training_log_ppl:np.float64(0.4125790528487414) - rollout_corr/kl:np.float64(0.00100548441882653) - rollout_corr/k3_kl:np.float64(0.001056205478960237) - rollout_corr/rollout_ppl:np.float64(1.5237079304642975) - rollout_corr/rollout_log_ppl:np.float64(0.4115735680970829) - rollout_corr/log_ppl_diff:np.float64(0.0010054847516585141) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024253172159660608) - rollout_corr/log_ppl_diff_max:np.float64(0.012961804866790771) - rollout_corr/log_ppl_diff_min:np.float64(-0.007540702819824219) - rollout_corr/ppl_ratio:np.float64(1.0010107702109963) - rollout_corr/chi2_token:np.float64(0.0022052584681659937) - rollout_corr/chi2_seq:np.float64(0.9272577015217394) - actor/pg_loss:np.float64(0.003355647437274456) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000539568421572767) - actor/ppo_kl:np.float64(7.455754089491506e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_w_mean:np.float64(178.37819157890044) - self_distillation/token_reweight_w_std:np.float64(3052.0747698339983) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9989271517843008) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07668937157723121) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.31669436022639275) - perf/mfu/actor:np.float64(0.04449943785630652) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.69052124023438) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:1 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013841266743838787 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013841266743838787 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mea
(TaskRunner pid=2255397) 
Training Progress:  65%|██████▌   | 65/100 [59:19<26:59, 46.27s/it]
(TaskRunner pid=2255397) step:66 - global_seqlen/min:16554 - global_seqlen/max:19900 - global_seqlen/minmax_diff:3346 - global_seqlen/balanced_min:18653 - global_seqlen/balanced_max:18704 - global_seqlen/mean:18662.625 - actor/entropy:0.3692823350429535 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24191009998321533 - training/rollout_probs_diff_mean:0.005031523294746876 - training/rollout_probs_diff_std:0.011805592104792595 - training/rollout_actor_probs_pearson_corr:0.998999834060669 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:0.9999994039535522 - rollout_corr/rollout_is_ratio_fraction_high:1.3333866263565142e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.2511892318725586 - rollout_corr/rollout_is_min:0.5517903566360474 - rollout_corr/rollout_is_std:0.03924669697880745 - rollout_corr/rollout_is_eff_sample_size:0.998462065654954 - rollout_corr/rollout_is_seq_mean:1.0000253915786743 - rollout_corr/rollout_is_seq_std:0.002698667347431183 - rollout_corr/rollout_is_seq_max:1.0073356628417969 - rollout_corr/rollout_is_seq_min:0.992901086807251 - rollout_corr/rollout_is_seq_max_deviation:0.007335662841796875 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5085271298885345) - rollout_corr/training_log_ppl:np.float64(0.4001465483015636) - rollout_corr/kl:np.float64(0.0009092299301867168) - rollout_corr/k3_kl:np.float64(0.001055825624348472) - rollout_corr/rollout_ppl:np.float64(1.5071750050410628) - rollout_corr/rollout_log_ppl:np.float64(0.3992373197543202) - rollout_corr/log_ppl_diff:np.float64(0.0009092285472434014) - rollout_corr/log_ppl_abs_diff:np.float64(0.002248107542982325) - rollout_corr/log_ppl_diff_max:np.float64(0.008349627256393433) - rollout_corr/log_ppl_diff_min:np.float64(-0.008058488368988037) - rollout_corr/ppl_ratio:np.float64(1.0009135268628597) - rollout_corr/chi2_token:np.float64(0.0025282949209213257) - rollout_corr/chi2_seq:np.float64(0.6103038967121392) - actor/pg_loss:np.float64(0.002379831741563976) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00033230165354325436) - actor/ppo_kl:np.float64(8.482311253033004e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(14.794726703781635) - self_distillation/token_reweight_w_std:np.float64(315.56329661473865) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9989614612422884) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04611051009851508) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.24986116960644722) - perf/mfu/actor:np.float64(0.044863735441974) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.69231033325195) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0023150925990194082 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0023150925990194082 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:29
(TaskRunner pid=2255397) 
Training Progress:  66%|██████▌   | 66/100 [1:00:00<25:17, 44.63s/it]
(TaskRunner pid=2255397) step:67 - global_seqlen/min:14936 - global_seqlen/max:21018 - global_seqlen/minmax_diff:6082 - global_seqlen/balanced_min:18548 - global_seqlen/balanced_max:18548 - global_seqlen/mean:18548.0 - actor/entropy:0.4010680615901947 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18490827083587646 - training/rollout_probs_diff_mean:0.005276753567159176 - training/rollout_probs_diff_std:0.011970635503530502 - training/rollout_actor_probs_pearson_corr:0.9990053772926331 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.6796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0001569986343384 - rollout_corr/rollout_is_ratio_fraction_high:1.357367818854982e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.357367818854982e-05 - rollout_corr/rollout_is_max:2.239859104156494 - rollout_corr/rollout_is_min:0.4357939660549164 - rollout_corr/rollout_is_std:0.03935437649488449 - rollout_corr/rollout_is_eff_sample_size:0.9984541032441931 - rollout_corr/rollout_is_seq_mean:1.0001627206802368 - rollout_corr/rollout_is_seq_std:0.002580940490588546 - rollout_corr/rollout_is_seq_max:1.0100871324539185 - rollout_corr/rollout_is_seq_min:0.9937511682510376 - rollout_corr/rollout_is_seq_max_deviation:0.010087132453918457 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5274818786419928) - rollout_corr/training_log_ppl:np.float64(0.41380137618398294) - rollout_corr/kl:np.float64(0.0010146739747582956) - rollout_corr/k3_kl:np.float64(0.0009835226737777703) - rollout_corr/rollout_ppl:np.float64(1.5259358463808894) - rollout_corr/rollout_log_ppl:np.float64(0.41278669840539806) - rollout_corr/log_ppl_diff:np.float64(0.0010146777785848826) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021784571872558445) - rollout_corr/log_ppl_diff_max:np.float64(0.014455139636993408) - rollout_corr/log_ppl_diff_min:np.float64(-0.0060721635818481445) - rollout_corr/ppl_ratio:np.float64(1.0010192152112722) - rollout_corr/chi2_token:np.float64(0.0019758078269660473) - rollout_corr/chi2_seq:np.float64(0.4387206360697746) - actor/pg_loss:np.float64(0.0029250025982037187) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039720970880807727) - actor/ppo_kl:np.float64(0.0003566578742226767) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_w_mean:np.float64(547.347474707989) - self_distillation/token_reweight_w_std:np.float64(8326.106724163285) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9982913499698043) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06155556399608031) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.2857639491558075) - perf/mfu/actor:np.float64(0.04548089224569089) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.682861328125) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:1 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0016373249236494303 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0016373249236494303 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:287.78125 -
(TaskRunner pid=2255397) 
Training Progress:  67%|██████▋   | 67/100 [1:00:37<23:20, 42.45s/it]
(TaskRunner pid=2255397) step:68 - global_seqlen/min:15394 - global_seqlen/max:21848 - global_seqlen/minmax_diff:6454 - global_seqlen/balanced_min:19111 - global_seqlen/balanced_max:19113 - global_seqlen/mean:19112.0 - actor/entropy:0.454579621553421 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2294895052909851 - training/rollout_probs_diff_mean:0.005558677017688751 - training/rollout_probs_diff_std:0.011987541802227497 - training/rollout_actor_probs_pearson_corr:0.999093234539032 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.6484375 - self_distillation/correct_sample_fraction:0.46875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6484375 - rollout_corr/rollout_is_mean:0.999990701675415 - rollout_corr/rollout_is_ratio_fraction_high:2.770083119685296e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.329737901687622 - rollout_corr/rollout_is_min:0.5801910758018494 - rollout_corr/rollout_is_std:0.04091522842645645 - rollout_corr/rollout_is_eff_sample_size:0.9983287417268669 - rollout_corr/rollout_is_seq_mean:0.9999210834503174 - rollout_corr/rollout_is_seq_std:0.0029238073620945215 - rollout_corr/rollout_is_seq_max:1.010511040687561 - rollout_corr/rollout_is_seq_min:0.9896543025970459 - rollout_corr/rollout_is_seq_max_deviation:0.010511040687561035 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6329940822906792) - rollout_corr/training_log_ppl:np.float64(0.4769620407023467) - rollout_corr/kl:np.float64(0.0010583482470849503) - rollout_corr/k3_kl:np.float64(0.0009939358607766735) - rollout_corr/rollout_ppl:np.float64(1.631263299845159) - rollout_corr/rollout_log_ppl:np.float64(0.47590369096724316) - rollout_corr/log_ppl_diff:np.float64(0.0010583497351035476) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024261822691187263) - rollout_corr/log_ppl_diff_max:np.float64(0.012225866317749023) - rollout_corr/log_ppl_diff_min:np.float64(-0.008922696113586426) - rollout_corr/ppl_ratio:np.float64(1.0010634786449373) - rollout_corr/chi2_token:np.float64(0.0018958586733788252) - rollout_corr/chi2_seq:np.float64(0.7360206483863294) - actor/pg_loss:np.float64(0.0039513889933004975) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004996858078811783) - actor/ppo_kl:np.float64(9.023317768352968e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.46875) - self_distillation/token_reweight_w_mean:np.float64(16211.263574679615) - self_distillation/token_reweight_w_std:np.float64(247760.0435439156) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0013059759512544) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09038829631754197) - self_distillation/empty_target_batch:np.float64(0.3515625) - actor/grad_norm:np.float64(0.35276055335998535) - perf/mfu/actor:np.float64(0.04664085001944063) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.4664192199707) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:1 - critic/score/mean:0.46875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0022922437638044357 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0022922437638044357 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:282.03125 - res
(TaskRunner pid=2255397) 
Training Progress:  68%|██████▊   | 68/100 [1:01:16<22:00, 41.26s/it]
(TaskRunner pid=2255397) step:69 - global_seqlen/min:17581 - global_seqlen/max:21958 - global_seqlen/minmax_diff:4377 - global_seqlen/balanced_min:20257 - global_seqlen/balanced_max:20260 - global_seqlen/mean:20258.25 - actor/entropy:0.46467649936676025 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135446548461914 - training/rollout_probs_diff_mean:0.00523096090182662 - training/rollout_probs_diff_std:0.011185083538293839 - training/rollout_actor_probs_pearson_corr:0.999234676361084 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.6796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.0000218152999878 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6324684619903564 - rollout_corr/rollout_is_min:0.5706769227981567 - rollout_corr/rollout_is_std:0.03825926035642624 - rollout_corr/rollout_is_eff_sample_size:0.998538606349212 - rollout_corr/rollout_is_seq_mean:1.0000791549682617 - rollout_corr/rollout_is_seq_std:0.002675000810995698 - rollout_corr/rollout_is_seq_max:1.0090395212173462 - rollout_corr/rollout_is_seq_min:0.989310622215271 - rollout_corr/rollout_is_seq_max_deviation:0.010689377784729004 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.646835464052856) - rollout_corr/training_log_ppl:np.float64(0.4856712990440428) - rollout_corr/kl:np.float64(0.0008948077715515979) - rollout_corr/k3_kl:np.float64(0.0010960273730233894) - rollout_corr/rollout_ppl:np.float64(1.6454020515084267) - rollout_corr/rollout_log_ppl:np.float64(0.48477648955304176) - rollout_corr/log_ppl_diff:np.float64(0.0008948094910010695) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023694130359217525) - rollout_corr/log_ppl_diff_max:np.float64(0.012593626976013184) - rollout_corr/log_ppl_diff_min:np.float64(-0.010994911193847656) - rollout_corr/ppl_ratio:np.float64(1.0008999034762383) - rollout_corr/chi2_token:np.float64(0.0027323926333338022) - rollout_corr/chi2_seq:np.float64(1.6403908263891935) - actor/pg_loss:np.float64(0.004149097250774503) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006782121126889251) - actor/ppo_kl:np.float64(0.0001744897412141455) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_w_mean:np.float64(112.03770920354873) - self_distillation/token_reweight_w_std:np.float64(1537.8677468675305) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9990846202708781) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10416805295972154) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.3852573037147522) - perf/mfu/actor:np.float64(0.04884632323523585) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.50916624069214) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:1 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001025630161166191 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001025630161166191 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:323.2578125 - response_l
(TaskRunner pid=2255397) 
Training Progress:  69%|██████▉   | 69/100 [1:01:55<21:00, 40.65s/it]
(TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70}
(TaskRunner pid=2255397) validation generation end
(TaskRunner pid=2255397) [prompt] system
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2255397) <reasoning>
(TaskRunner pid=2255397) ...
(TaskRunner pid=2255397) </reasoning>
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) ...
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) A
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) user
(TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2255397) Please reason step by step.
(TaskRunner pid=2255397) assistant
(TaskRunner pid=2255397) <think>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) </think>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) [response] <reasoning>
(TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we must analyze the structure and functional groups present in both the product and the options.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) The product contains:
(TaskRunner pid=2255397) - A nitrogen-containing ring (likely a pyrrole or similar heterocycle).
(TaskRunner pid=2255397) - A chlorinated aromatic ring (–Cl group).
(TaskRunner pid=2255397) - A trifluoromethyl group (–C(F)(F)F).
(TaskRunner pid=2255397) - A nitrogen atom bonded to a carbon chain that includes the chlorinated aromatic ring.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Now, let's examine the options:
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option A**: Contains a chlorinated aromatic ring, a trifluoromethyl group, and a nitrogen-containing ring with a nitrogen atom bonded to a carbon chain that includes the chlorinated aromatic ring. It closely matches the structure of the product and is a strong candidate.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option B**: Contains a chlorinated aromatic ring but lacks the trifluoromethyl group and the correct nitrogen-containing ring structure. It is not a match.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option C**: Contains a chlorinated aromatic ring and a trifluoromethyl group, but the nitrogen-containing ring is not correctly structured. It does not match the product.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option D**: Contains a trifluoromethyl group and a carbon chain with a chlorinated aromatic ring, but the nitrogen-containing ring is not correctly structured. It also does not match the product.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Thus, **Option A** is the only one that contains all the necessary functional groups and structural features required to synthesize the product.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) </reasoning>
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) A
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) [ground_truth] A
(TaskRunner pid=2255397) [score] 1.0
(TaskRunner pid=2255397) [acc] 1.0
(TaskRunner pid=2255397) [pred] A
(TaskRunner pid=2255397) [incorrect_format] 1
(TaskRunner pid=2255397) [feedback] 
(TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70
(TaskRunner pid=2255397) step:70 - global_seqlen/min:14508 - global_seqlen/max:21779 - global_seqlen/minmax_diff:7271 - global_seqlen/balanced_min:19295 - global_seqlen/balanced_max:19376 - global_seqlen/mean:19309.25 - actor/entropy:0.48265349864959717 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2931375205516815 - training/rollout_probs_diff_mean:0.005450577940791845 - training/rollout_probs_diff_std:0.011540403589606285 - training/rollout_actor_probs_pearson_corr:0.9992024302482605 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.796875 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.796875 - rollout_corr/rollout_is_mean:1.0000245571136475 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6136518716812134 - rollout_corr/rollout_is_min:0.5989248752593994 - rollout_corr/rollout_is_std:0.03922086954116821 - rollout_corr/rollout_is_eff_sample_size:0.9984642048317262 - rollout_corr/rollout_is_seq_mean:1.0000163316726685 - rollout_corr/rollout_is_seq_std:0.002710202941671014 - rollout_corr/rollout_is_seq_max:1.0095508098602295 - rollout_corr/rollout_is_seq_min:0.9921322464942932 - rollout_corr/rollout_is_seq_max_deviation:0.009550809860229492 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6673680869862437) - rollout_corr/training_log_ppl:np.float64(0.4960062170866877) - rollout_corr/kl:np.float64(0.0012155248211556113) - rollout_corr/k3_kl:np.float64(0.0012827583032049006) - rollout_corr/rollout_ppl:np.float64(1.6654691863805056) - rollout_corr/rollout_log_ppl:np.float64(0.4947906881570816) - rollout_corr/log_ppl_diff:np.float64(0.00121552892960608) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026022420497611165) - rollout_corr/log_ppl_diff_max:np.float64(0.016035795211791992) - rollout_corr/log_ppl_diff_min:np.float64(-0.00719606876373291) - rollout_corr/ppl_ratio:np.float64(1.0012216775212437) - rollout_corr/chi2_token:np.float64(0.002924366621300578) - rollout_corr/chi2_seq:np.float64(1.3708638157695532) - actor/pg_loss:np.float64(0.0026991143822669983) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006329814200398687) - actor/ppo_kl:np.float64(0.0004067609955740181) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(1099.6124461798463) - self_distillation/token_reweight_w_std:np.float64(17668.276791150623) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005380015354604) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08962540709762834) - self_distillation/empty_target_batch:np.float64(0.203125) - actor/grad_norm:np.float64(0.30823376774787903) - perf/mfu/actor:np.float64(0.04660907978159935) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.58329391479492) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6967261904761904) - val-aux/sciknoweval/reward/std@16:np.float64(0.15836902316671747) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7580857142857141) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.12446919093953822) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6347571428571429) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1415950973799327) - val-aux/sciknoweval/reward/maj@
(TaskRunner pid=2255397) 
Training Progress:  70%|███████   | 70/100 [1:04:59<41:54, 83.82s/it]
(TaskRunner pid=2255397) step:71 - global_seqlen/min:19243 - global_seqlen/max:22825 - global_seqlen/minmax_diff:3582 - global_seqlen/balanced_min:21239 - global_seqlen/balanced_max:21249 - global_seqlen/mean:21244.625 - actor/entropy:0.48243123292922974 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1782420575618744 - training/rollout_probs_diff_mean:0.004945038352161646 - training/rollout_probs_diff_std:0.010490105487406254 - training/rollout_actor_probs_pearson_corr:0.9993298649787903 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.58984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:1.0001386404037476 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.646081805229187 - rollout_corr/rollout_is_min:0.6092609167098999 - rollout_corr/rollout_is_std:0.036472681909799576 - rollout_corr/rollout_is_eff_sample_size:0.998671986340567 - rollout_corr/rollout_is_seq_mean:1.00009286403656 - rollout_corr/rollout_is_seq_std:0.0023478141520172358 - rollout_corr/rollout_is_seq_max:1.0064475536346436 - rollout_corr/rollout_is_seq_min:0.9918783903121948 - rollout_corr/rollout_is_seq_max_deviation:0.008121609687805176 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6814591218717396) - rollout_corr/training_log_ppl:np.float64(0.5044128161389381) - rollout_corr/kl:np.float64(0.0009273746180333831) - rollout_corr/k3_kl:np.float64(0.0009558818163668548) - rollout_corr/rollout_ppl:np.float64(1.679951566271484) - rollout_corr/rollout_log_ppl:np.float64(0.5034854415280279) - rollout_corr/log_ppl_diff:np.float64(0.0009273746109101921) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021467784827109426) - rollout_corr/log_ppl_diff_max:np.float64(0.009796887636184692) - rollout_corr/log_ppl_diff_min:np.float64(-0.007009267807006836) - rollout_corr/ppl_ratio:np.float64(1.0009312871843576) - rollout_corr/chi2_token:np.float64(0.002058735117316246) - rollout_corr/chi2_seq:np.float64(0.9684193118009716) - actor/pg_loss:np.float64(0.002505662152543664) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036506839978756034) - actor/ppo_kl:np.float64(0.0002983230811750559) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_w_mean:np.float64(1927.8832940307911) - self_distillation/token_reweight_w_std:np.float64(37339.10144847393) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009577083401382) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08062945469282568) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.5023428723216057) - perf/mfu/actor:np.float64(0.051698067264255104) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(201.09897232055664) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:1 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0013479345943778753 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0013479345943778753 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:361.51953125 - 
(TaskRunner pid=2255397) 
Training Progress:  71%|███████   | 71/100 [1:05:39<34:06, 70.57s/it]
(TaskRunner pid=2255397) step:72 - global_seqlen/min:16380 - global_seqlen/max:21611 - global_seqlen/minmax_diff:5231 - global_seqlen/balanced_min:19858 - global_seqlen/balanced_max:19861 - global_seqlen/mean:19860.0 - actor/entropy:0.5429559946060181 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.15169164538383484 - training/rollout_probs_diff_mean:0.005359958857297897 - training/rollout_probs_diff_std:0.010805261321365833 - training/rollout_actor_probs_pearson_corr:0.9993027448654175 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0000475645065308 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.644222617149353 - rollout_corr/rollout_is_min:0.6008924245834351 - rollout_corr/rollout_is_std:0.03811253234744072 - rollout_corr/rollout_is_eff_sample_size:0.9985496605558143 - rollout_corr/rollout_is_seq_mean:0.9999592304229736 - rollout_corr/rollout_is_seq_std:0.0024577102158218622 - rollout_corr/rollout_is_seq_max:1.0064175128936768 - rollout_corr/rollout_is_seq_min:0.9917982220649719 - rollout_corr/rollout_is_seq_max_deviation:0.008201777935028076 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7471187515184283) - rollout_corr/training_log_ppl:np.float64(0.5459236325696111) - rollout_corr/kl:np.float64(0.0010623894131924772) - rollout_corr/k3_kl:np.float64(0.0008694290827975237) - rollout_corr/rollout_ppl:np.float64(1.745131317526102) - rollout_corr/rollout_log_ppl:np.float64(0.5448612435720861) - rollout_corr/log_ppl_diff:np.float64(0.0010623889975249767) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021515846019610763) - rollout_corr/log_ppl_diff_max:np.float64(0.010035932064056396) - rollout_corr/log_ppl_diff_min:np.float64(-0.00523415207862854) - rollout_corr/ppl_ratio:np.float64(1.0010662653949112) - rollout_corr/chi2_token:np.float64(0.001382074085995555) - rollout_corr/chi2_seq:np.float64(0.6654008245095611) - actor/pg_loss:np.float64(0.005195157486014068) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003912408333235362) - actor/ppo_kl:np.float64(0.0002666896097824889) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.671875) - self_distillation/token_reweight_w_mean:np.float64(7515.255449632183) - self_distillation/token_reweight_w_std:np.float64(120734.30576296372) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9992036235053092) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1153781593311578) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.3598603457212448) - perf/mfu/actor:np.float64(0.048463804802359345) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.67909240722656) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:1 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006963299238123 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006963299238123 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:333.78125 - response_length/m
(TaskRunner pid=2255397) 
Training Progress:  72%|███████▏  | 72/100 [1:06:19<28:36, 61.30s/it]
(TaskRunner pid=2255397) step:73 - global_seqlen/min:19647 - global_seqlen/max:22925 - global_seqlen/minmax_diff:3278 - global_seqlen/balanced_min:21192 - global_seqlen/balanced_max:21202 - global_seqlen/mean:21199.0 - actor/entropy:0.5374614596366882 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.301607221364975 - training/rollout_probs_diff_mean:0.004932303912937641 - training/rollout_probs_diff_std:0.010089674964547157 - training/rollout_actor_probs_pearson_corr:0.9993661642074585 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:0.9999767541885376 - rollout_corr/rollout_is_ratio_fraction_high:1.021909702103585e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.1998536586761475 - rollout_corr/rollout_is_min:0.6301840543746948 - rollout_corr/rollout_is_std:0.03611796349287033 - rollout_corr/rollout_is_eff_sample_size:0.9986969544688866 - rollout_corr/rollout_is_seq_mean:1.0000014305114746 - rollout_corr/rollout_is_seq_std:0.0019163991091772914 - rollout_corr/rollout_is_seq_max:1.0062246322631836 - rollout_corr/rollout_is_seq_min:0.9927696585655212 - rollout_corr/rollout_is_seq_max_deviation:0.00723034143447876 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7496894244104624) - rollout_corr/training_log_ppl:np.float64(0.5468992951209657) - rollout_corr/kl:np.float64(0.0007367991016540287) - rollout_corr/k3_kl:np.float64(0.0007361613289731395) - rollout_corr/rollout_ppl:np.float64(1.7483539679087698) - rollout_corr/rollout_log_ppl:np.float64(0.5461624904419295) - rollout_corr/log_ppl_diff:np.float64(0.0007368046790361404) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016733732772991061) - rollout_corr/log_ppl_diff_max:np.float64(0.009091198444366455) - rollout_corr/log_ppl_diff_min:np.float64(-0.005505651235580444) - rollout_corr/ppl_ratio:np.float64(1.0007393239066005) - rollout_corr/chi2_token:np.float64(0.0014872262254357338) - rollout_corr/chi2_seq:np.float64(0.43628761847503483) - actor/pg_loss:np.float64(0.0029139972757548094) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018539705752118607) - actor/ppo_kl:np.float64(6.380892117796577e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(1624.953690484399) - self_distillation/token_reweight_w_std:np.float64(32560.52424334915) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991668974980712) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06964316591620445) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.21447288990020752) - perf/mfu/actor:np.float64(0.050516435918862526) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.0999493598938) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007218515034765005 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007218515034765005 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:382.25 -
(TaskRunner pid=2255397) 
Training Progress:  73%|███████▎  | 73/100 [1:07:00<24:51, 55.23s/it]
(TaskRunner pid=2255397) step:74 - global_seqlen/min:18287 - global_seqlen/max:25091 - global_seqlen/minmax_diff:6804 - global_seqlen/balanced_min:20904 - global_seqlen/balanced_max:20912 - global_seqlen/mean:20910.0 - actor/entropy:0.5127137303352356 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27330654859542847 - training/rollout_probs_diff_mean:0.004899736028164625 - training/rollout_probs_diff_std:0.010343815200030804 - training/rollout_actor_probs_pearson_corr:0.9993671774864197 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.000099539756775 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0278759873472154e-05 - rollout_corr/rollout_is_max:1.7854251861572266 - rollout_corr/rollout_is_min:0.48449382185935974 - rollout_corr/rollout_is_std:0.0365249402821064 - rollout_corr/rollout_is_eff_sample_size:0.9986679439991228 - rollout_corr/rollout_is_seq_mean:1.000190258026123 - rollout_corr/rollout_is_seq_std:0.0022744936868548393 - rollout_corr/rollout_is_seq_max:1.0073591470718384 - rollout_corr/rollout_is_seq_min:0.9917874336242676 - rollout_corr/rollout_is_seq_max_deviation:0.008212566375732422 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7097132932394743) - rollout_corr/training_log_ppl:np.float64(0.5229829574818723) - rollout_corr/kl:np.float64(0.0006870645115757412) - rollout_corr/k3_kl:np.float64(0.0007633993515128168) - rollout_corr/rollout_ppl:np.float64(1.7085823002271354) - rollout_corr/rollout_log_ppl:np.float64(0.5222958918311633) - rollout_corr/log_ppl_diff:np.float64(0.0006870656507089734) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019122406374663115) - rollout_corr/log_ppl_diff_max:np.float64(0.008660197257995605) - rollout_corr/log_ppl_diff_min:np.float64(-0.005268260836601257) - rollout_corr/ppl_ratio:np.float64(1.000690019223839) - rollout_corr/chi2_token:np.float64(0.0016905386000871658) - rollout_corr/chi2_seq:np.float64(1.4013629674445838) - actor/pg_loss:np.float64(0.0029980618273839355) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017913388501256122) - actor/ppo_kl:np.float64(0.00016305825718454514) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_w_mean:np.float64(15.748328928370029) - self_distillation/token_reweight_w_std:np.float64(289.3728471599752) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004254090599716) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06948782474501058) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.2460213266313076) - perf/mfu/actor:np.float64(0.0504502030203242) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.804443359375) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:1 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005830626469105482 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005830626469105482 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:380.0
(TaskRunner pid=2255397) 
Training Progress:  74%|███████▍  | 74/100 [1:07:40<21:56, 50.63s/it]
(TaskRunner pid=2255397) step:75 - global_seqlen/min:16481 - global_seqlen/max:26987 - global_seqlen/minmax_diff:10506 - global_seqlen/balanced_min:22439 - global_seqlen/balanced_max:22447 - global_seqlen/mean:22444.625 - actor/entropy:0.5642927289009094 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2400461733341217 - training/rollout_probs_diff_mean:0.0051166643388569355 - training/rollout_probs_diff_std:0.010140416212379932 - training/rollout_actor_probs_pearson_corr:0.9994109869003296 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.71484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9999061822891235 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0068769370263908e-05 - rollout_corr/rollout_is_max:1.5713963508605957 - rollout_corr/rollout_is_min:0.4978005588054657 - rollout_corr/rollout_is_std:0.03701609745621681 - rollout_corr/rollout_is_eff_sample_size:0.9986314455154365 - rollout_corr/rollout_is_seq_mean:0.999901294708252 - rollout_corr/rollout_is_seq_std:0.0019736075773835182 - rollout_corr/rollout_is_seq_max:1.0060852766036987 - rollout_corr/rollout_is_seq_min:0.9931412935256958 - rollout_corr/rollout_is_seq_max_deviation:0.006858706474304199 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.791621611919254) - rollout_corr/training_log_ppl:np.float64(0.570267406059429) - rollout_corr/kl:np.float64(0.0008821988329259511) - rollout_corr/k3_kl:np.float64(0.0008163573830870519) - rollout_corr/rollout_ppl:np.float64(1.7901061396114528) - rollout_corr/rollout_log_ppl:np.float64(0.56938520638505) - rollout_corr/log_ppl_diff:np.float64(0.0008821996743790805) - rollout_corr/log_ppl_abs_diff:np.float64(0.00185499427607283) - rollout_corr/log_ppl_diff_max:np.float64(0.00786745548248291) - rollout_corr/log_ppl_diff_min:np.float64(-0.005721062421798706) - rollout_corr/ppl_ratio:np.float64(1.000884982291609) - rollout_corr/chi2_token:np.float64(0.001510836649686098) - rollout_corr/chi2_seq:np.float64(0.9038489798549563) - actor/pg_loss:np.float64(0.002076247939839959) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023045666375764995) - actor/ppo_kl:np.float64(5.9166708297198056e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_w_mean:np.float64(6473.88584861136) - self_distillation/token_reweight_w_std:np.float64(110608.01085398125) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9989379325415939) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0619521141925361) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.19757572747766972) - perf/mfu/actor:np.float64(0.053976922098505964) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.7879409790039) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:1 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0026480865199118853 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0026480865199118853 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:38
(TaskRunner pid=2255397) 
Training Progress:  75%|███████▌  | 75/100 [1:08:20<19:47, 47.50s/it]
(TaskRunner pid=2255397) step:76 - global_seqlen/min:20385 - global_seqlen/max:23735 - global_seqlen/minmax_diff:3350 - global_seqlen/balanced_min:22458 - global_seqlen/balanced_max:22461 - global_seqlen/mean:22459.875 - actor/entropy:0.5038673281669617 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17917871475219727 - training/rollout_probs_diff_mean:0.004544694907963276 - training/rollout_probs_diff_std:0.009781488217413425 - training/rollout_actor_probs_pearson_corr:0.9994357228279114 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:0.999882698059082 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.531444787979126 - rollout_corr/rollout_is_min:0.6298530101776123 - rollout_corr/rollout_is_std:0.03470920771360397 - rollout_corr/rollout_is_eff_sample_size:0.9987963637779437 - rollout_corr/rollout_is_seq_mean:0.9998737573623657 - rollout_corr/rollout_is_seq_std:0.001968828495591879 - rollout_corr/rollout_is_seq_max:1.0092756748199463 - rollout_corr/rollout_is_seq_min:0.9928333163261414 - rollout_corr/rollout_is_seq_max_deviation:0.009275674819946289 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7229888113215566) - rollout_corr/training_log_ppl:np.float64(0.5273766612517647) - rollout_corr/kl:np.float64(0.0006758117173214506) - rollout_corr/k3_kl:np.float64(0.0006499772979395857) - rollout_corr/rollout_ppl:np.float64(1.72180867055431) - rollout_corr/rollout_log_ppl:np.float64(0.526700847200118) - rollout_corr/log_ppl_diff:np.float64(0.0006758140516467392) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016295489040203393) - rollout_corr/log_ppl_diff_max:np.float64(0.007842868566513062) - rollout_corr/log_ppl_diff_min:np.float64(-0.005845367908477783) - rollout_corr/ppl_ratio:np.float64(1.0006780109833926) - rollout_corr/chi2_token:np.float64(0.0012538495939224958) - rollout_corr/chi2_seq:np.float64(0.6266895108856261) - actor/pg_loss:np.float64(0.0024852192727848887) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022367794190358836) - actor/ppo_kl:np.float64(-8.031681899822729e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(11.586508150445297) - self_distillation/token_reweight_w_std:np.float64(205.42800806532614) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991163620725274) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07528238961822353) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.21515963971614838) - perf/mfu/actor:np.float64(0.05377171741192728) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.86333084106445) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009749584714882076 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009749584714882076 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:416.18359375 - re
(TaskRunner pid=2255397) 
Training Progress:  76%|███████▌  | 76/100 [1:09:00<18:08, 45.33s/it]
(TaskRunner pid=2255397) step:77 - global_seqlen/min:19528 - global_seqlen/max:26165 - global_seqlen/minmax_diff:6637 - global_seqlen/balanced_min:22384 - global_seqlen/balanced_max:22391 - global_seqlen/mean:22389.375 - actor/entropy:0.5504021048545837 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2723144292831421 - training/rollout_probs_diff_mean:0.005151774734258652 - training/rollout_probs_diff_std:0.01037499401718378 - training/rollout_actor_probs_pearson_corr:0.9993544816970825 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.7421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:0.9999526739120483 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6774260997772217 - rollout_corr/rollout_is_min:0.6362783908843994 - rollout_corr/rollout_is_std:0.037128645926713943 - rollout_corr/rollout_is_eff_sample_size:0.9986231237591031 - rollout_corr/rollout_is_seq_mean:0.9999860525131226 - rollout_corr/rollout_is_seq_std:0.002109500812366605 - rollout_corr/rollout_is_seq_max:1.0075759887695312 - rollout_corr/rollout_is_seq_min:0.993508517742157 - rollout_corr/rollout_is_seq_max_deviation:0.00757598876953125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7856992213055491) - rollout_corr/training_log_ppl:np.float64(0.5679072652128525) - rollout_corr/kl:np.float64(0.0009176925997458341) - rollout_corr/k3_kl:np.float64(0.0007757482836723284) - rollout_corr/rollout_ppl:np.float64(1.784037644509226) - rollout_corr/rollout_log_ppl:np.float64(0.5669895681203343) - rollout_corr/log_ppl_diff:np.float64(0.0009176970925182104) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019323080778121948) - rollout_corr/log_ppl_diff_max:np.float64(0.007754087448120117) - rollout_corr/log_ppl_diff_min:np.float64(-0.00577545166015625) - rollout_corr/ppl_ratio:np.float64(1.00092074717395) - rollout_corr/chi2_token:np.float64(0.00127337034791708) - rollout_corr/chi2_seq:np.float64(0.6404874103609473) - actor/pg_loss:np.float64(0.0034807646879926324) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002253228935842344) - actor/ppo_kl:np.float64(0.00018203446227005315) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_w_mean:np.float64(130.1160973869264) - self_distillation/token_reweight_w_std:np.float64(2325.1732160223182) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002648527733982) - self_distillation/token_reweight_w_clip_frac:np.float64(0.091672543407185) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.2861635349690914) - perf/mfu/actor:np.float64(0.053185240362257795) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.9845733642578) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:1 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0007428093231283128 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0007428093231283128 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:382.57421875 - response_le
(TaskRunner pid=2255397) 
Training Progress:  77%|███████▋  | 77/100 [1:09:40<16:47, 43.82s/it]
(TaskRunner pid=2255397) step:78 - global_seqlen/min:20134 - global_seqlen/max:31492 - global_seqlen/minmax_diff:11358 - global_seqlen/balanced_min:22482 - global_seqlen/balanced_max:29618 - global_seqlen/mean:23385.375 - actor/entropy:0.48590248823165894 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40464895963668823 - training/rollout_probs_diff_mean:0.004600349348038435 - training/rollout_probs_diff_std:0.009926710277795792 - training/rollout_actor_probs_pearson_corr:0.9993968605995178 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.89453125 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.89453125 - rollout_corr/rollout_is_mean:0.9998959302902222 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.656884347146843e-06 - rollout_corr/rollout_is_max:1.8092979192733765 - rollout_corr/rollout_is_min:0.48850229382514954 - rollout_corr/rollout_is_std:0.03525106608867645 - rollout_corr/rollout_is_eff_sample_size:0.9987587856840187 - rollout_corr/rollout_is_seq_mean:0.9998641014099121 - rollout_corr/rollout_is_seq_std:0.0020814789459109306 - rollout_corr/rollout_is_seq_max:1.0097109079360962 - rollout_corr/rollout_is_seq_min:0.9942805171012878 - rollout_corr/rollout_is_seq_max_deviation:0.009710907936096191 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7207122039981186) - rollout_corr/training_log_ppl:np.float64(0.5288956113836321) - rollout_corr/kl:np.float64(0.0008265634318824766) - rollout_corr/k3_kl:np.float64(0.0007540007430009155) - rollout_corr/rollout_ppl:np.float64(1.7192450258880854) - rollout_corr/rollout_log_ppl:np.float64(0.5280690497056639) - rollout_corr/log_ppl_diff:np.float64(0.0008265616779681295) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017743823991622776) - rollout_corr/log_ppl_diff_max:np.float64(0.009331107139587402) - rollout_corr/log_ppl_diff_min:np.float64(-0.004447638988494873) - rollout_corr/ppl_ratio:np.float64(1.0008291078265756) - rollout_corr/chi2_token:np.float64(0.0013703966978937387) - rollout_corr/chi2_seq:np.float64(0.8413228269200772) - actor/pg_loss:np.float64(0.004617663682438433) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030698799764650175) - actor/ppo_kl:np.float64(-3.2745020313029727e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.89453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.89453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(38.89375674771145) - self_distillation/token_reweight_w_std:np.float64(700.807301894878) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001390076475218) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09478579316055402) - self_distillation/empty_target_batch:np.float64(0.10546875) - actor/grad_norm:np.float64(0.28953084722161293) - perf/mfu/actor:np.float64(0.0530806510376341) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.00171661376953) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.025150412693619728 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.025150412693619728 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_
(TaskRunner pid=2255397) 
Training Progress:  78%|███████▊  | 78/100 [1:11:14<21:31, 58.72s/it]
(TaskRunner pid=2255397) step:79 - global_seqlen/min:20379 - global_seqlen/max:23902 - global_seqlen/minmax_diff:3523 - global_seqlen/balanced_min:22483 - global_seqlen/balanced_max:22828 - global_seqlen/mean:22531.875 - actor/entropy:0.47975319623947144 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.999380886554718 - training/rollout_probs_diff_mean:0.004774122033268213 - training/rollout_probs_diff_std:0.010841231793165207 - training/rollout_actor_probs_pearson_corr:0.9993086457252502 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9999167919158936 - rollout_corr/rollout_is_ratio_fraction_high:8.872878424881492e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.7745756849762984e-05 - rollout_corr/rollout_is_max:2.256157159805298 - rollout_corr/rollout_is_min:1.8565188497632334e-07 - rollout_corr/rollout_is_std:0.035090066492557526 - rollout_corr/rollout_is_eff_sample_size:0.9987699636823175 - rollout_corr/rollout_is_seq_mean:0.9999032020568848 - rollout_corr/rollout_is_seq_std:0.002013471210375428 - rollout_corr/rollout_is_seq_max:1.0075592994689941 - rollout_corr/rollout_is_seq_min:0.9934999942779541 - rollout_corr/rollout_is_seq_max_deviation:0.007559299468994141 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6734817675314844) - rollout_corr/training_log_ppl:np.float64(0.5028578019700944) - rollout_corr/kl:np.float64(0.0007867071553491201) - rollout_corr/k3_kl:np.float64(0.0008091878738127889) - rollout_corr/rollout_ppl:np.float64(1.6721355388872325) - rollout_corr/rollout_log_ppl:np.float64(0.5020710933895316) - rollout_corr/log_ppl_diff:np.float64(0.0007867085805628449) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017737957823555917) - rollout_corr/log_ppl_diff_max:np.float64(0.022169530391693115) - rollout_corr/log_ppl_diff_min:np.float64(-0.006566405296325684) - rollout_corr/ppl_ratio:np.float64(1.000790162710473) - rollout_corr/chi2_token:np.float64(0.0015391185879707336) - rollout_corr/chi2_seq:np.float64(1.4496739441528916) - actor/pg_loss:np.float64(0.005306927720084786) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039829218349041184) - actor/ppo_kl:np.float64(-4.736486153222863e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(76.99012057809159) - self_distillation/token_reweight_w_std:np.float64(1541.8618622039212) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995152873452753) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11978079983964562) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.2994599714875221) - perf/mfu/actor:np.float64(0.052974969811297) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.06073379516602) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:1 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009753511287271976 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009753511287271976 - critic/returns/max:0.875 - critic/returns/min:-0.875 - re
(TaskRunner pid=2255397) 
Training Progress:  79%|███████▉  | 79/100 [1:12:00<19:11, 54.82s/it]
(TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80}
(TaskRunner pid=2255397) validation generation end
(TaskRunner pid=2255397) [prompt] system
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2255397) <reasoning>
(TaskRunner pid=2255397) ...
(TaskRunner pid=2255397) </reasoning>
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) ...
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) A
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) user
(TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2255397) Please reason step by step.
(TaskRunner pid=2255397) assistant
(TaskRunner pid=2255397) <think>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) </think>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) [response] <reasoning>
(TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we must analyze the structure and functional groups of the target molecule and compare it to the structures of the given options.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) The target molecule contains:
(TaskRunner pid=2255397) - A nitrogen-containing ring (likely an azole or similar).
(TaskRunner pid=2255397) - A chlorinated aromatic ring (–Cl).
(TaskRunner pid=2255397) - A trifluoromethyl group (–C(F)(F)F).
(TaskRunner pid=2255397) - A nitrogen-containing side chain that is connected to a carbon chain with a chlorinated aromatic group and a trifluoromethyl group.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Now, we analyze the options:
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option A:**  
(TaskRunner pid=2255397) "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"  
(TaskRunner pid=2255397) - Contains a chlorinated aromatic ring (–Cl).
(TaskRunner pid=2255397) - Contains a trifluoromethyl group (–C(F)(F)F).
(TaskRunner pid=2255397) - Contains a nitrogen-containing ring with a side chain that connects to the chlorinated aromatic ring and the trifluoromethyl group.
(TaskRunner pid=2255397) - Matches the structure of the target molecule closely.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option B:**  
(TaskRunner pid=2255397) "O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1"  
(TaskRunner pid=2255397) - Contains a chlorinated ring, but the side chain and functional groups do not match the target molecule.
(TaskRunner pid=2255397) - The nitrogen-containing ring is different and does not include the trifluoromethyl group.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option C:**  
(TaskRunner pid=2255397) "O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1"  
(TaskRunner pid=2255397) - Contains a chlorinated aromatic ring and a trifluoromethyl group.
(TaskRunner pid=2255397) - However, the functional group arrangement and the nitrogen-containing ring do not match the target molecule's structure.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option D:**  
(TaskRunner pid=2255397) "O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F"  
(TaskRunner pid=2255397) - Contains a trifluoromethyl group and a chlorinated ring.
(TaskRunner pid=2255397) - However, the nitrogen-containing ring and the side chain are not structured correctly to match the target molecule.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Conclusion:**  
(TaskRunner pid=2255397) Only **Option A** contains all the necessary functional groups and structural elements that match the target molecule "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1".
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) </reasoning>
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) A
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) [ground_truth] A
(TaskRunner pid=2255397) [score] 1.0
(TaskRunner pid=2255397) [acc] 1.0
(TaskRunner pid=2255397) [pred] A
(TaskRunner pid=2255397) [incorrect_format] 1
(TaskRunner pid=2255397) [feedback] 
(TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80
(TaskRunner pid=2255397) step:80 - global_seqlen/min:21379 - global_seqlen/max:27736 - global_seqlen/minmax_diff:6357 - global_seqlen/balanced_min:24041 - global_seqlen/balanced_max:24046 - global_seqlen/mean:24044.0 - actor/entropy:0.5321111679077148 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22554826736450195 - training/rollout_probs_diff_mean:0.005127668380737305 - training/rollout_probs_diff_std:0.010481650941073895 - training/rollout_actor_probs_pearson_corr:0.999333381652832 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:0.9999201893806458 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.784020185470581 - rollout_corr/rollout_is_min:0.5773587822914124 - rollout_corr/rollout_is_std:0.03691612929105759 - rollout_corr/rollout_is_eff_sample_size:0.9986386974447631 - rollout_corr/rollout_is_seq_mean:0.9999967813491821 - rollout_corr/rollout_is_seq_std:0.002032036194577813 - rollout_corr/rollout_is_seq_max:1.0090333223342896 - rollout_corr/rollout_is_seq_min:0.99375981092453 - rollout_corr/rollout_is_seq_max_deviation:0.00903332233428955 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7447218438610435) - rollout_corr/training_log_ppl:np.float64(0.5451616903301328) - rollout_corr/kl:np.float64(0.0007389033336671069) - rollout_corr/k3_kl:np.float64(0.0007813431365661927) - rollout_corr/rollout_ppl:np.float64(1.743419658858329) - rollout_corr/rollout_log_ppl:np.float64(0.5444227863335982) - rollout_corr/log_ppl_diff:np.float64(0.0007389039965346456) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016596049536019564) - rollout_corr/log_ppl_diff_max:np.float64(0.005907297134399414) - rollout_corr/log_ppl_diff_min:np.float64(-0.0042874813079833984) - rollout_corr/ppl_ratio:np.float64(1.0007411232218146) - rollout_corr/chi2_token:np.float64(0.001654008636251092) - rollout_corr/chi2_seq:np.float64(0.4709114315919578) - actor/pg_loss:np.float64(0.004708616062998772) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004071622186074819) - actor/ppo_kl:np.float64(2.530917598608795e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(13418.10702541261) - self_distillation/token_reweight_w_std:np.float64(249965.62142065133) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001714508980513) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10478754635551013) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.277141023427248) - perf/mfu/actor:np.float64(0.05686537883473178) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.12682485580444) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7098214285714286) - val-aux/sciknoweval/reward/std@16:np.float64(0.14355997006676555) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7664952380952381) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.10898051269342317) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6531333333333332) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.13319323391175897) - val-aux/sciknoweval/reward/m
(TaskRunner pid=2255397) 
Training Progress:  80%|████████  | 80/100 [1:15:09<31:40, 95.02s/it]
(TaskRunner pid=2255397) step:81 - global_seqlen/min:19063 - global_seqlen/max:27921 - global_seqlen/minmax_diff:8858 - global_seqlen/balanced_min:22607 - global_seqlen/balanced_max:22618 - global_seqlen/mean:22615.0 - actor/entropy:0.4837610721588135 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18462881445884705 - training/rollout_probs_diff_mean:0.004895538557320833 - training/rollout_probs_diff_std:0.010500817559659481 - training/rollout_actor_probs_pearson_corr:0.9993140697479248 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:0.9999274611473083 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.790866897674277e-06 - rollout_corr/rollout_is_max:1.9132018089294434 - rollout_corr/rollout_is_min:0.4658448398113251 - rollout_corr/rollout_is_std:0.036682888865470886 - rollout_corr/rollout_is_eff_sample_size:0.998655936060176 - rollout_corr/rollout_is_seq_mean:0.9999848008155823 - rollout_corr/rollout_is_seq_std:0.002200948540121317 - rollout_corr/rollout_is_seq_max:1.0080451965332031 - rollout_corr/rollout_is_seq_min:0.9924934506416321 - rollout_corr/rollout_is_seq_max_deviation:0.008045196533203125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6707927286624908) - rollout_corr/training_log_ppl:np.float64(0.496672905923333) - rollout_corr/kl:np.float64(0.0007028676274636325) - rollout_corr/k3_kl:np.float64(0.0008774436023486487) - rollout_corr/rollout_ppl:np.float64(1.6695120050571859) - rollout_corr/rollout_log_ppl:np.float64(0.4959700393374078) - rollout_corr/log_ppl_diff:np.float64(0.0007028665859252214) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018403253052383661) - rollout_corr/log_ppl_diff_max:np.float64(0.008645951747894287) - rollout_corr/log_ppl_diff_min:np.float64(-0.005466446280479431) - rollout_corr/ppl_ratio:np.float64(1.0007056852336973) - rollout_corr/chi2_token:np.float64(0.002068758476525545) - rollout_corr/chi2_seq:np.float64(0.5465720517095178) - actor/pg_loss:np.float64(0.0029664566973224282) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00035411453472988796) - actor/ppo_kl:np.float64(-4.833459730235745e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(46.35069849877618) - self_distillation/token_reweight_w_std:np.float64(809.1466785562807) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000212090089917) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07078566148993559) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.2487758994102478) - perf/mfu/actor:np.float64(0.0539087676600922) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.76457595825195) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:1 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008355281315743923 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008355281315743923 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:
(TaskRunner pid=2255397) 
Training Progress:  81%|████████  | 81/100 [1:15:50<24:59, 78.90s/it]
(TaskRunner pid=2255397) step:82 - global_seqlen/min:20637 - global_seqlen/max:24388 - global_seqlen/minmax_diff:3751 - global_seqlen/balanced_min:22213 - global_seqlen/balanced_max:22222 - global_seqlen/mean:22220.125 - actor/entropy:0.44993990659713745 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135524034500122 - training/rollout_probs_diff_mean:0.0047300467267632484 - training/rollout_probs_diff_std:0.010585080832242966 - training/rollout_actor_probs_pearson_corr:0.9992892146110535 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.6015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:0.9999090433120728 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.8240019926452078e-05 - rollout_corr/rollout_is_max:1.7895359992980957 - rollout_corr/rollout_is_min:0.000389088352676481 - rollout_corr/rollout_is_std:0.03580636531114578 - rollout_corr/rollout_is_eff_sample_size:0.9987193079476842 - rollout_corr/rollout_is_seq_mean:0.9998470544815063 - rollout_corr/rollout_is_seq_std:0.0019274608930572867 - rollout_corr/rollout_is_seq_max:1.0058091878890991 - rollout_corr/rollout_is_seq_min:0.9948349595069885 - rollout_corr/rollout_is_seq_max_deviation:0.005809187889099121 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6126991645433009) - rollout_corr/training_log_ppl:np.float64(0.4650081274448894) - rollout_corr/kl:np.float64(0.0009749815301454134) - rollout_corr/k3_kl:np.float64(0.0008126581537908351) - rollout_corr/rollout_ppl:np.float64(1.6110614440403879) - rollout_corr/rollout_log_ppl:np.float64(0.46403314505005255) - rollout_corr/log_ppl_diff:np.float64(0.000974982394836843) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017385364044457674) - rollout_corr/log_ppl_diff_max:np.float64(0.01138317584991455) - rollout_corr/log_ppl_diff_min:np.float64(-0.004374325275421143) - rollout_corr/ppl_ratio:np.float64(1.0009776803199202) - rollout_corr/chi2_token:np.float64(0.0012213955633342266) - rollout_corr/chi2_seq:np.float64(0.3682185474317521) - actor/pg_loss:np.float64(0.0047809198731556535) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00044549755830303184) - actor/ppo_kl:np.float64(9.73689185599369e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6015625) - self_distillation/token_reweight_w_mean:np.float64(8763.06257886719) - self_distillation/token_reweight_w_std:np.float64(183308.58723631455) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998282515443861) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10694929157034494) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.27595510706305504) - perf/mfu/actor:np.float64(0.053128164780379857) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.69163131713867) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:1 - critic/score/mean:0.6015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036035440862178802 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0036035440862178802 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean
(TaskRunner pid=2255397) 
Training Progress:  82%|████████▏ | 82/100 [1:16:31<20:13, 67.44s/it]
(TaskRunner pid=2255397) step:83 - global_seqlen/min:17994 - global_seqlen/max:23375 - global_seqlen/minmax_diff:5381 - global_seqlen/balanced_min:21160 - global_seqlen/balanced_max:21170 - global_seqlen/mean:21165.25 - actor/entropy:0.4580913484096527 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1788254976272583 - training/rollout_probs_diff_mean:0.005244065076112747 - training/rollout_probs_diff_std:0.011341297999024391 - training/rollout_actor_probs_pearson_corr:0.9992040395736694 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7578125 - self_distillation/correct_sample_fraction:0.49609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7578125 - rollout_corr/rollout_is_mean:1.0000450611114502 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6948878765106201 - rollout_corr/rollout_is_min:0.5044571757316589 - rollout_corr/rollout_is_std:0.03826860338449478 - rollout_corr/rollout_is_eff_sample_size:0.9985378931829969 - rollout_corr/rollout_is_seq_mean:0.9999917149543762 - rollout_corr/rollout_is_seq_std:0.0024205187801271677 - rollout_corr/rollout_is_seq_max:1.007947325706482 - rollout_corr/rollout_is_seq_min:0.9913325309753418 - rollout_corr/rollout_is_seq_max_deviation:0.008667469024658203 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6229573977179825) - rollout_corr/training_log_ppl:np.float64(0.4732308021048084) - rollout_corr/kl:np.float64(0.0009288024647275961) - rollout_corr/k3_kl:np.float64(0.0008689218968811474) - rollout_corr/rollout_ppl:np.float64(1.6214703912846744) - rollout_corr/rollout_log_ppl:np.float64(0.47230199666228145) - rollout_corr/log_ppl_diff:np.float64(0.0009288054425269365) - rollout_corr/log_ppl_abs_diff:np.float64(0.002043650601990521) - rollout_corr/log_ppl_diff_max:np.float64(0.01018601655960083) - rollout_corr/log_ppl_diff_min:np.float64(-0.0065596699714660645) - rollout_corr/ppl_ratio:np.float64(1.0009324771817774) - rollout_corr/chi2_token:np.float64(0.0016192824114114046) - rollout_corr/chi2_seq:np.float64(0.7770539077464491) - actor/pg_loss:np.float64(0.005605395417660475) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005950954305262712) - actor/ppo_kl:np.float64(0.00014756420982209306) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.49609375) - self_distillation/token_reweight_w_mean:np.float64(43651.46375838155) - self_distillation/token_reweight_w_std:np.float64(663337.3462570815) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001920053968206) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12145704132854007) - self_distillation/empty_target_batch:np.float64(0.2421875) - actor/grad_norm:np.float64(0.37171755731105804) - perf/mfu/actor:np.float64(0.051307628909952094) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.40789794921875) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:1 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006963750929571688 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006963750929571688 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:362.5078125 - 
(TaskRunner pid=2255397) 
Training Progress:  83%|████████▎ | 83/100 [1:17:10<16:46, 59.20s/it]
(TaskRunner pid=2255397) step:84 - global_seqlen/min:16326 - global_seqlen/max:23634 - global_seqlen/minmax_diff:7308 - global_seqlen/balanced_min:19956 - global_seqlen/balanced_max:19966 - global_seqlen/mean:19962.0 - actor/entropy:0.4355461001396179 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105639219284058 - training/rollout_probs_diff_mean:0.0052682701498270035 - training/rollout_probs_diff_std:0.011572654359042645 - training/rollout_actor_probs_pearson_corr:0.9991452097892761 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.6875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999979138374329 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7088184356689453 - rollout_corr/rollout_is_min:0.6109715700149536 - rollout_corr/rollout_is_std:0.038198452442884445 - rollout_corr/rollout_is_eff_sample_size:0.9985430042300492 - rollout_corr/rollout_is_seq_mean:0.999947190284729 - rollout_corr/rollout_is_seq_std:0.0022522625513374805 - rollout_corr/rollout_is_seq_max:1.0068590641021729 - rollout_corr/rollout_is_seq_min:0.9928377866744995 - rollout_corr/rollout_is_seq_max_deviation:0.007162213325500488 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5662654708139598) - rollout_corr/training_log_ppl:np.float64(0.4385948544368148) - rollout_corr/kl:np.float64(0.0010095660052041877) - rollout_corr/k3_kl:np.float64(0.0008661396312845682) - rollout_corr/rollout_ppl:np.float64(1.5646833986975253) - rollout_corr/rollout_log_ppl:np.float64(0.4375852865341585) - rollout_corr/log_ppl_diff:np.float64(0.001009567902656272) - rollout_corr/log_ppl_abs_diff:np.float64(0.002128819382051006) - rollout_corr/log_ppl_diff_max:np.float64(0.010426610708236694) - rollout_corr/log_ppl_diff_min:np.float64(-0.0056244730949401855) - rollout_corr/ppl_ratio:np.float64(1.0010133925825357) - rollout_corr/chi2_token:np.float64(0.0014718861784785986) - rollout_corr/chi2_seq:np.float64(0.8280399544164538) - actor/pg_loss:np.float64(0.005005670711398125) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003053363407161669) - actor/ppo_kl:np.float64(0.00018772196818872544) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_w_mean:np.float64(1562.93544895784) - self_distillation/token_reweight_w_std:np.float64(29233.69878470525) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9984272990841419) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09582373814191669) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.27201734483242035) - perf/mfu/actor:np.float64(0.04843784064306516) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.41874313354492) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:1 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0011461590183898807 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0011461590183898807 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:369.78125 - response_length/max:
(TaskRunner pid=2255397) 
Training Progress:  84%|████████▍ | 84/100 [1:17:50<14:13, 53.36s/it]
(TaskRunner pid=2255397) step:85 - global_seqlen/min:18150 - global_seqlen/max:26353 - global_seqlen/minmax_diff:8203 - global_seqlen/balanced_min:22089 - global_seqlen/balanced_max:22097 - global_seqlen/mean:22095.125 - actor/entropy:0.5015257596969604 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22547149658203125 - training/rollout_probs_diff_mean:0.005411901045590639 - training/rollout_probs_diff_std:0.011227205395698547 - training/rollout_actor_probs_pearson_corr:0.9992418885231018 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.71484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0000245571136475 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9998200514237396e-05 - rollout_corr/rollout_is_max:1.8690942525863647 - rollout_corr/rollout_is_min:0.3950064182281494 - rollout_corr/rollout_is_std:0.038822196424007416 - rollout_corr/rollout_is_eff_sample_size:0.9984952239250111 - rollout_corr/rollout_is_seq_mean:1.0000362396240234 - rollout_corr/rollout_is_seq_std:0.0021999897435307503 - rollout_corr/rollout_is_seq_max:1.0113379955291748 - rollout_corr/rollout_is_seq_min:0.9936926364898682 - rollout_corr/rollout_is_seq_max_deviation:0.011337995529174805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6718262596987188) - rollout_corr/training_log_ppl:np.float64(0.5036372784525156) - rollout_corr/kl:np.float64(0.0008671445817096668) - rollout_corr/k3_kl:np.float64(0.0008546028941509576) - rollout_corr/rollout_ppl:np.float64(1.670389695558697) - rollout_corr/rollout_log_ppl:np.float64(0.5027701300568879) - rollout_corr/log_ppl_diff:np.float64(0.000867148395627737) - rollout_corr/log_ppl_abs_diff:np.float64(0.00184359191916883) - rollout_corr/log_ppl_diff_max:np.float64(0.007470607757568359) - rollout_corr/log_ppl_diff_min:np.float64(-0.005036354064941406) - rollout_corr/ppl_ratio:np.float64(1.000870036194101) - rollout_corr/chi2_token:np.float64(0.001706940121948719) - rollout_corr/chi2_seq:np.float64(0.803199248155579) - actor/pg_loss:np.float64(0.0017579543637111783) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.799221642315388e-05) - actor/ppo_kl:np.float64(0.00011437771347289072) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_w_mean:np.float64(531.5800662350375) - self_distillation/token_reweight_w_std:np.float64(6761.905646110186) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.998828696552664) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02915754570858553) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.16114498674869537) - perf/mfu/actor:np.float64(0.052797853671240076) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.15685653686523) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:1 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009824115550145507 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009824115550145507 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:390.66015625 - response_lengt
(TaskRunner pid=2255397) 
Training Progress:  85%|████████▌ | 85/100 [1:18:31<12:22, 49.52s/it]
(TaskRunner pid=2255397) step:86 - global_seqlen/min:19172 - global_seqlen/max:23707 - global_seqlen/minmax_diff:4535 - global_seqlen/balanced_min:21178 - global_seqlen/balanced_max:21185 - global_seqlen/mean:21182.625 - actor/entropy:0.4417911767959595 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2772989869117737 - training/rollout_probs_diff_mean:0.004951337352395058 - training/rollout_probs_diff_std:0.011015241034328938 - training/rollout_actor_probs_pearson_corr:0.999230682849884 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0001946687698364 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0157130418519955e-05 - rollout_corr/rollout_is_max:1.913230538368225 - rollout_corr/rollout_is_min:0.48568272590637207 - rollout_corr/rollout_is_std:0.03722003847360611 - rollout_corr/rollout_is_eff_sample_size:0.9986170608525367 - rollout_corr/rollout_is_seq_mean:1.000307321548462 - rollout_corr/rollout_is_seq_std:0.0024071792140603065 - rollout_corr/rollout_is_seq_max:1.0088176727294922 - rollout_corr/rollout_is_seq_min:0.9944450259208679 - rollout_corr/rollout_is_seq_max_deviation:0.008817672729492188 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.608684377744794) - rollout_corr/training_log_ppl:np.float64(0.4590825633786153) - rollout_corr/kl:np.float64(0.0005738776003347823) - rollout_corr/k3_kl:np.float64(0.0008272058389025005) - rollout_corr/rollout_ppl:np.float64(1.6077406979165971) - rollout_corr/rollout_log_ppl:np.float64(0.45850868528941646) - rollout_corr/log_ppl_diff:np.float64(0.0005738780891988426) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020339481707196683) - rollout_corr/log_ppl_diff_max:np.float64(0.008264422416687012) - rollout_corr/log_ppl_diff_min:np.float64(-0.009043440222740173) - rollout_corr/ppl_ratio:np.float64(1.00057721324265) - rollout_corr/chi2_token:np.float64(0.0023612575605511665) - rollout_corr/chi2_seq:np.float64(1.4658026506658643) - actor/pg_loss:np.float64(0.00446285936050117) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004066334154231299) - actor/ppo_kl:np.float64(0.00014794760728875644) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(132.06133578903973) - self_distillation/token_reweight_w_std:np.float64(2865.135968480725) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988646688871086) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1047386541031301) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.297544926404953) - perf/mfu/actor:np.float64(0.05125659080467514) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.21656799316406) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:1 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011623566970229149 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011623566970229149 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:384.58
(TaskRunner pid=2255397) 
Training Progress:  86%|████████▌ | 86/100 [1:19:11<10:56, 46.87s/it]
(TaskRunner pid=2255397) step:87 - global_seqlen/min:19881 - global_seqlen/max:22805 - global_seqlen/minmax_diff:2924 - global_seqlen/balanced_min:21579 - global_seqlen/balanced_max:21581 - global_seqlen/mean:21580.5 - actor/entropy:0.4654737114906311 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3751887083053589 - training/rollout_probs_diff_mean:0.0050046672113239765 - training/rollout_probs_diff_std:0.010888464748859406 - training/rollout_actor_probs_pearson_corr:0.9992407560348511 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.62109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.0000450611114502 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0308853234164417e-05 - rollout_corr/rollout_is_max:1.854941487312317 - rollout_corr/rollout_is_min:0.3242693543434143 - rollout_corr/rollout_is_std:0.03734952583909035 - rollout_corr/rollout_is_eff_sample_size:0.998607075049263 - rollout_corr/rollout_is_seq_mean:1.0000066757202148 - rollout_corr/rollout_is_seq_std:0.00233814911916852 - rollout_corr/rollout_is_seq_max:1.0092549324035645 - rollout_corr/rollout_is_seq_min:0.9934799671173096 - rollout_corr/rollout_is_seq_max_deviation:0.009254932403564453 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6343936491757631) - rollout_corr/training_log_ppl:np.float64(0.47929042315809056) - rollout_corr/kl:np.float64(0.0010372451878630784) - rollout_corr/k3_kl:np.float64(0.00085206899655077) - rollout_corr/rollout_ppl:np.float64(1.6326408791355789) - rollout_corr/rollout_log_ppl:np.float64(0.4782531792880036) - rollout_corr/log_ppl_diff:np.float64(0.001037243870086968) - rollout_corr/log_ppl_abs_diff:np.float64(0.002030018251389265) - rollout_corr/log_ppl_diff_max:np.float64(0.011031702160835266) - rollout_corr/log_ppl_diff_min:np.float64(-0.006544649600982666) - rollout_corr/ppl_ratio:np.float64(1.0010407408699393) - rollout_corr/chi2_token:np.float64(0.0013177578803151846) - rollout_corr/chi2_seq:np.float64(0.7552365991286933) - actor/pg_loss:np.float64(0.006197207723744214) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000917987231787265) - actor/ppo_kl:np.float64(0.00031247709554804715) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_w_mean:np.float64(8798.193575591547) - self_distillation/token_reweight_w_std:np.float64(137248.53065139498) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011895855423063) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1116473093861714) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.3543427586555481) - perf/mfu/actor:np.float64(0.052530135237403436) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.04389572143555) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:1 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004306523595005274 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004306523595005274 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:378
(TaskRunner pid=2255397) 
Training Progress:  87%|████████▋ | 87/100 [1:19:51<09:41, 44.71s/it]
(TaskRunner pid=2255397) step:88 - global_seqlen/min:18137 - global_seqlen/max:23978 - global_seqlen/minmax_diff:5841 - global_seqlen/balanced_min:21618 - global_seqlen/balanced_max:21626 - global_seqlen/mean:21623.75 - actor/entropy:0.4558359980583191 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135497212409973 - training/rollout_probs_diff_mean:0.005039794836193323 - training/rollout_probs_diff_std:0.01103699766099453 - training/rollout_actor_probs_pearson_corr:0.9992441534996033 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.84765625 - self_distillation/correct_sample_fraction:0.58984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84765625 - rollout_corr/rollout_is_mean:0.9999773502349854 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.879797284549568e-05 - rollout_corr/rollout_is_max:1.6429718732833862 - rollout_corr/rollout_is_min:0.34945860505104065 - rollout_corr/rollout_is_std:0.03734952583909035 - rollout_corr/rollout_is_eff_sample_size:0.9986068372944764 - rollout_corr/rollout_is_seq_mean:0.9999490976333618 - rollout_corr/rollout_is_seq_std:0.00212614587508142 - rollout_corr/rollout_is_seq_max:1.007338047027588 - rollout_corr/rollout_is_seq_min:0.9925395846366882 - rollout_corr/rollout_is_seq_max_deviation:0.007460415363311768 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6032785079441965) - rollout_corr/training_log_ppl:np.float64(0.46080948115559295) - rollout_corr/kl:np.float64(0.0011248015301781322) - rollout_corr/k3_kl:np.float64(0.0008873714502897201) - rollout_corr/rollout_ppl:np.float64(1.601451831869781) - rollout_corr/rollout_log_ppl:np.float64(0.4596846750937402) - rollout_corr/log_ppl_diff:np.float64(0.0011248060618527234) - rollout_corr/log_ppl_abs_diff:np.float64(0.002022003580350429) - rollout_corr/log_ppl_diff_max:np.float64(0.012501776218414307) - rollout_corr/log_ppl_diff_min:np.float64(-0.007802844047546387) - rollout_corr/ppl_ratio:np.float64(1.0011287301313132) - rollout_corr/chi2_token:np.float64(0.0012870903592556715) - rollout_corr/chi2_seq:np.float64(0.6421570053789765) - actor/pg_loss:np.float64(0.004205821780487895) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004930502600473119) - actor/ppo_kl:np.float64(0.00033077471513121015) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84765625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84765625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_w_mean:np.float64(21107.031607169192) - self_distillation/token_reweight_w_std:np.float64(309542.6647549386) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0036525861360133) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08997728512622416) - self_distillation/empty_target_batch:np.float64(0.15234375) - actor/grad_norm:np.float64(0.29456714540719986) - perf/mfu/actor:np.float64(0.05240681242673272) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.06198120117188) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:1 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007283487357199192 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0007283487357199192 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/m
(TaskRunner pid=2255397) 
Training Progress:  88%|████████▊ | 88/100 [1:20:32<08:41, 43.45s/it]
(TaskRunner pid=2255397) step:89 - global_seqlen/min:20192 - global_seqlen/max:26260 - global_seqlen/minmax_diff:6068 - global_seqlen/balanced_min:23178 - global_seqlen/balanced_max:23186 - global_seqlen/mean:23183.75 - actor/entropy:0.4386472702026367 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7019415497779846 - training/rollout_probs_diff_mean:0.0047370451502501965 - training/rollout_probs_diff_std:0.010689225047826767 - training/rollout_actor_probs_pearson_corr:0.999261736869812 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:0.9999846816062927 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.8036542314803228e-05 - rollout_corr/rollout_is_max:1.5554907321929932 - rollout_corr/rollout_is_min:0.03433068096637726 - rollout_corr/rollout_is_std:0.0357397198677063 - rollout_corr/rollout_is_eff_sample_size:0.9987241830413971 - rollout_corr/rollout_is_seq_mean:0.9999765157699585 - rollout_corr/rollout_is_seq_std:0.0020081165712326765 - rollout_corr/rollout_is_seq_max:1.0105236768722534 - rollout_corr/rollout_is_seq_min:0.9940011501312256 - rollout_corr/rollout_is_seq_max_deviation:0.010523676872253418 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6203870358876884) - rollout_corr/training_log_ppl:np.float64(0.47031728448928334) - rollout_corr/kl:np.float64(0.0010279387461658018) - rollout_corr/k3_kl:np.float64(0.0008464122184363987) - rollout_corr/rollout_ppl:np.float64(1.6186228357255459) - rollout_corr/rollout_log_ppl:np.float64(0.4692893385363277) - rollout_corr/log_ppl_diff:np.float64(0.0010279459529556334) - rollout_corr/log_ppl_abs_diff:np.float64(0.00189788214629516) - rollout_corr/log_ppl_diff_max:np.float64(0.0077957212924957275) - rollout_corr/log_ppl_diff_min:np.float64(-0.004509687423706055) - rollout_corr/ppl_ratio:np.float64(1.0010310297366232) - rollout_corr/chi2_token:np.float64(0.0013277053367346525) - rollout_corr/chi2_seq:np.float64(0.5938672970514745) - actor/pg_loss:np.float64(0.0077381625305861235) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008514890976130118) - actor/ppo_kl:np.float64(0.0002891868470378256) - actor/pg_clipfrac_lower:np.float64(8.172070920409169e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(47.27642107871361) - self_distillation/token_reweight_w_std:np.float64(801.2714838348329) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000362384133041) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13113477459410205) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.3602307364344597) - perf/mfu/actor:np.float64(0.05579510012189607) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.08685684204102) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:1 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0041562956757843494 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0041562956757843494 - critic/returns/max:0.875 - critic/returns/min:-0.87
(TaskRunner pid=2255397) 
Training Progress:  89%|████████▉ | 89/100 [1:21:13<07:52, 42.96s/it]
(TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90}
(TaskRunner pid=2255397) validation generation end
(TaskRunner pid=2255397) [prompt] system
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2255397) <reasoning>
(TaskRunner pid=2255397) ...
(TaskRunner pid=2255397) </reasoning>
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) ...
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) A
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) user
(TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2255397) Please reason step by step.
(TaskRunner pid=2255397) assistant
(TaskRunner pid=2255397) <think>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) </think>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) [response] <reasoning>
(TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"**, we need to analyze the structure of the product and compare it with the options provided.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) The product has the following key features:
(TaskRunner pid=2255397) - A nitrogen-containing ring (likely a pyrrole or similar structure).
(TaskRunner pid=2255397) - A **chlorine-substituted benzene ring** (from the **Cl** in **"ccc(Cl)"**).
(TaskRunner pid=2255397) - A **trifluoromethyl group** (from **"C(F)(F)F"**).
(TaskRunner pid=2255397) - A **nitrogen-containing bridge** connecting the ring to the rest of the molecule.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Now, let’s evaluate each option:
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option A**:  
(TaskRunner pid=2255397) `O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1`  
(TaskRunner pid=2255397) - Contains a **chlorine-substituted benzene ring**.
(TaskRunner pid=2255397) - Contains a **trifluoromethyl group**.
(TaskRunner pid=2255397) - Contains a **nitrogen-containing bridge** with the correct connectivity.
(TaskRunner pid=2255397) - This matches the structure of the product and appears to be a direct precursor.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option B**:  
(TaskRunner pid=2255397) `O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1`  
(TaskRunner pid=2255397) - Contains **chlorine atoms**, but the **trifluoromethyl group** is missing.
(TaskRunner pid=2255397) - The **nitrogen-containing bridge** is not properly connected to the **trifluoromethyl** group.
(TaskRunner pid=2255397) - This does not match the product structure.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option C**:  
(TaskRunner pid=2255397) `O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1`  
(TaskRunner pid=2255397) - Contains a **chlorine-substituted benzene ring** and a **trifluoromethyl group**.
(TaskRunner pid=2255397) - However, the **nitrogen-containing bridge** is not properly positioned and is connected via an **oxygen**, which is not part of the product.
(TaskRunner pid=2255397) - This is not a correct match.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option D**:  
(TaskRunner pid=2255397) `O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F`  
(TaskRunner pid=2255397) - Contains a **trifluoromethyl group**.
(TaskRunner pid=2255397) - However, it **lacks the chlorine-substituted benzene ring** and the **correct nitrogen-containing bridge**.
(TaskRunner pid=2255397) - This also does not match the product structure.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Conclusion**:  
(TaskRunner pid=2255397) Only **Option A** contains all the necessary functional groups and the correct connectivity to serve as the only correct reactant in the synthesis of the given product.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) </reasoning>
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) A
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) [ground_truth] A
(TaskRunner pid=2255397) [score] 1.0
(TaskRunner pid=2255397) [acc] 1.0
(TaskRunner pid=2255397) [pred] A
(TaskRunner pid=2255397) [incorrect_format] 1
(TaskRunner pid=2255397) [feedback] 
(TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90
(TaskRunner pid=2255397) step:90 - global_seqlen/min:18865 - global_seqlen/max:24932 - global_seqlen/minmax_diff:6067 - global_seqlen/balanced_min:22815 - global_seqlen/balanced_max:22831 - global_seqlen/mean:22826.625 - actor/entropy:0.40077415108680725 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5775288939476013 - training/rollout_probs_diff_mean:0.0045609320513904095 - training/rollout_probs_diff_std:0.010716044344007969 - training/rollout_actor_probs_pearson_corr:0.9992610216140747 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.85546875 - self_distillation/correct_sample_fraction:0.62109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.85546875 - rollout_corr/rollout_is_mean:0.9999971985816956 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.970459930424113e-06 - rollout_corr/rollout_is_max:1.8681867122650146 - rollout_corr/rollout_is_min:0.2344389706850052 - rollout_corr/rollout_is_std:0.03548699617385864 - rollout_corr/rollout_is_eff_sample_size:0.9987421380652595 - rollout_corr/rollout_is_seq_mean:1.0000590085983276 - rollout_corr/rollout_is_seq_std:0.0020531753543764353 - rollout_corr/rollout_is_seq_max:1.0109810829162598 - rollout_corr/rollout_is_seq_min:0.9949836730957031 - rollout_corr/rollout_is_seq_max_deviation:0.010981082916259766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5583982807584107) - rollout_corr/training_log_ppl:np.float64(0.43078241116018035) - rollout_corr/kl:np.float64(0.0008912494752149058) - rollout_corr/k3_kl:np.float64(0.000791456644748223) - rollout_corr/rollout_ppl:np.float64(1.5569715378805995) - rollout_corr/rollout_log_ppl:np.float64(0.42989116214448586) - rollout_corr/log_ppl_diff:np.float64(0.0008912490156944841) - rollout_corr/log_ppl_abs_diff:np.float64(0.001720420696074143) - rollout_corr/log_ppl_diff_max:np.float64(0.01338726282119751) - rollout_corr/log_ppl_diff_min:np.float64(-0.00902646780014038) - rollout_corr/ppl_ratio:np.float64(1.0008942331187427) - rollout_corr/chi2_token:np.float64(0.0014041787944734097) - rollout_corr/chi2_seq:np.float64(0.752974865725264) - actor/pg_loss:np.float64(0.004781530937179923) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005209917633237637) - actor/ppo_kl:np.float64(0.0002642208678480529) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.85546875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.85546875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_w_mean:np.float64(619.1751045542769) - self_distillation/token_reweight_w_std:np.float64(7680.31531262747) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999175232835114) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1011948958330322) - self_distillation/empty_target_batch:np.float64(0.14453125) - actor/grad_norm:np.float64(0.260063748806715) - perf/mfu/actor:np.float64(0.05435728539450956) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.9617691040039) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7092261904761905) - val-aux/sciknoweval/reward/std@16:np.float64(0.1437505073441559) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7665238095238094) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.11153257282472556) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6511952380952379) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.13026803458061142) - val-aux/s
(TaskRunner pid=2255397) 
Training Progress:  90%|█████████ | 90/100 [1:24:30<14:51, 89.15s/it]
(TaskRunner pid=2255397) step:91 - global_seqlen/min:20110 - global_seqlen/max:28143 - global_seqlen/minmax_diff:8033 - global_seqlen/balanced_min:22954 - global_seqlen/balanced_max:22958 - global_seqlen/mean:22955.625 - actor/entropy:0.4345111846923828 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.318805456161499 - training/rollout_probs_diff_mean:0.00483555207028985 - training/rollout_probs_diff_std:0.010917664505541325 - training/rollout_actor_probs_pearson_corr:0.9992064833641052 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.0000874996185303 - rollout_corr/rollout_is_ratio_fraction_high:9.058217074198183e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.8116434148396365e-05 - rollout_corr/rollout_is_max:2.0618367195129395 - rollout_corr/rollout_is_min:0.27237245440483093 - rollout_corr/rollout_is_std:0.037339948117733 - rollout_corr/rollout_is_eff_sample_size:0.9986079071919081 - rollout_corr/rollout_is_seq_mean:1.0000126361846924 - rollout_corr/rollout_is_seq_std:0.0018744589760899544 - rollout_corr/rollout_is_seq_max:1.0062435865402222 - rollout_corr/rollout_is_seq_min:0.9926995038986206 - rollout_corr/rollout_is_seq_max_deviation:0.0073004961013793945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5809713881462812) - rollout_corr/training_log_ppl:np.float64(0.4482057260465808) - rollout_corr/kl:np.float64(0.0007782714664905654) - rollout_corr/k3_kl:np.float64(0.0007742295367734187) - rollout_corr/rollout_ppl:np.float64(1.5796857569366693) - rollout_corr/rollout_log_ppl:np.float64(0.4474274521926418) - rollout_corr/log_ppl_diff:np.float64(0.0007782738539390266) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016364274197258055) - rollout_corr/log_ppl_diff_max:np.float64(0.007638216018676758) - rollout_corr/log_ppl_diff_min:np.float64(-0.005470573902130127) - rollout_corr/ppl_ratio:np.float64(1.0007805426139385) - rollout_corr/chi2_token:np.float64(0.001570558873936534) - rollout_corr/chi2_seq:np.float64(0.9185826345346868) - actor/pg_loss:np.float64(0.004598198342137039) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000316593827392353) - actor/ppo_kl:np.float64(5.939174411295767e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(7563.319094766863) - self_distillation/token_reweight_w_std:np.float64(150023.32301567192) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999886890873313) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08767912839539349) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.2757617086172104) - perf/mfu/actor:np.float64(0.054246744251757004) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(201.1460475921631) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0008684565545991063 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0008684565545991063 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_le
(TaskRunner pid=2255397) 
Training Progress:  91%|█████████ | 91/100 [1:25:12<11:13, 74.81s/it]
(TaskRunner pid=2255397) step:92 - global_seqlen/min:20808 - global_seqlen/max:26065 - global_seqlen/minmax_diff:5257 - global_seqlen/balanced_min:23344 - global_seqlen/balanced_max:23359 - global_seqlen/mean:23353.875 - actor/entropy:0.43259748816490173 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2583375573158264 - training/rollout_probs_diff_mean:0.004716021008789539 - training/rollout_probs_diff_std:0.010604353621602058 - training/rollout_actor_probs_pearson_corr:0.9992642402648926 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:1.0000042915344238 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.831503691908438e-06 - rollout_corr/rollout_is_max:1.6297050714492798 - rollout_corr/rollout_is_min:0.3157358169555664 - rollout_corr/rollout_is_std:0.03597741574048996 - rollout_corr/rollout_is_eff_sample_size:0.9987072987735173 - rollout_corr/rollout_is_seq_mean:0.9999913573265076 - rollout_corr/rollout_is_seq_std:0.0019688971806317568 - rollout_corr/rollout_is_seq_max:1.0057662725448608 - rollout_corr/rollout_is_seq_min:0.9920455813407898 - rollout_corr/rollout_is_seq_max_deviation:0.007954418659210205 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5944940163753927) - rollout_corr/training_log_ppl:np.float64(0.4538156447233632) - rollout_corr/kl:np.float64(0.0008374510896800302) - rollout_corr/k3_kl:np.float64(0.000751384576176406) - rollout_corr/rollout_ppl:np.float64(1.5930784032680094) - rollout_corr/rollout_log_ppl:np.float64(0.45297819224651903) - rollout_corr/log_ppl_diff:np.float64(0.0008374524768441916) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017089889151975513) - rollout_corr/log_ppl_diff_max:np.float64(0.008830234408378601) - rollout_corr/log_ppl_diff_min:np.float64(-0.005124390125274658) - rollout_corr/ppl_ratio:np.float64(1.0008400096558034) - rollout_corr/chi2_token:np.float64(0.0013158204965293407) - rollout_corr/chi2_seq:np.float64(0.8615647666156292) - actor/pg_loss:np.float64(0.00415497412905097) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000329681670109494) - actor/ppo_kl:np.float64(0.000143634846812013) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(2081.352001107065) - self_distillation/token_reweight_w_std:np.float64(46560.20298913785) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010203949641436) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09130151267163455) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.25993651896715164) - perf/mfu/actor:np.float64(0.05560204307595709) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.7599754333496) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:1 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011590245179831982 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011590245179831982 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:442.30859375 - 
(TaskRunner pid=2255397) 
Training Progress:  92%|█████████▏| 92/100 [1:25:54<08:39, 64.99s/it]
(TaskRunner pid=2255397) step:93 - global_seqlen/min:20104 - global_seqlen/max:26536 - global_seqlen/minmax_diff:6432 - global_seqlen/balanced_min:22165 - global_seqlen/balanced_max:22172 - global_seqlen/mean:22170.375 - actor/entropy:0.4773474633693695 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18463870882987976 - training/rollout_probs_diff_mean:0.005192984826862812 - training/rollout_probs_diff_std:0.01093792449682951 - training/rollout_actor_probs_pearson_corr:0.9992780685424805 - self_distillation/success_group_fraction:1.0 - self_distillation/success_sample_fraction:0.9921875 - self_distillation/correct_sample_fraction:0.83203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9921875 - rollout_corr/rollout_is_mean:1.0000450611114502 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.661463737487793 - rollout_corr/rollout_is_min:0.516041100025177 - rollout_corr/rollout_is_std:0.037531014531850815 - rollout_corr/rollout_is_eff_sample_size:0.9985935232071051 - rollout_corr/rollout_is_seq_mean:1.0000596046447754 - rollout_corr/rollout_is_seq_std:0.0020964141003787518 - rollout_corr/rollout_is_seq_max:1.0067566633224487 - rollout_corr/rollout_is_seq_min:0.9933486580848694 - rollout_corr/rollout_is_seq_max_deviation:0.0067566633224487305 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6435860991477966) - rollout_corr/training_log_ppl:np.float64(0.48362978905788623) - rollout_corr/kl:np.float64(0.000619826348660979) - rollout_corr/k3_kl:np.float64(0.0007837708870965798) - rollout_corr/rollout_ppl:np.float64(1.6425714306533337) - rollout_corr/rollout_log_ppl:np.float64(0.48300995968747884) - rollout_corr/log_ppl_diff:np.float64(0.0006198293704073876) - rollout_corr/log_ppl_abs_diff:np.float64(0.001762611762387678) - rollout_corr/log_ppl_diff_max:np.float64(0.009479671716690063) - rollout_corr/log_ppl_diff_min:np.float64(-0.005926311016082764) - rollout_corr/ppl_ratio:np.float64(1.0006224988028407) - rollout_corr/chi2_token:np.float64(0.0019295907113701105) - rollout_corr/chi2_seq:np.float64(1.5755573513451964) - actor/pg_loss:np.float64(0.004148329375311732) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002517454886401538) - actor/ppo_kl:np.float64(-5.375289406117645e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9921875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9921875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_w_mean:np.float64(17544.475337664597) - self_distillation/token_reweight_w_std:np.float64(333058.533660894) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001345220953226) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08209344942588359) - self_distillation/empty_target_batch:np.float64(0.0078125) - actor/grad_norm:np.float64(0.2505064271390438) - perf/mfu/actor:np.float64(0.05288764926923402) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.60185623168945) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:1 - critic/score/mean:0.83203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.83203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00024851804482750595 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00024851804482750595 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:426.35546875 - re
(TaskRunner pid=2255397) 
Training Progress:  93%|█████████▎| 93/100 [1:26:35<06:44, 57.82s/it]
(TaskRunner pid=2255397) step:94 - global_seqlen/min:20450 - global_seqlen/max:26778 - global_seqlen/minmax_diff:6328 - global_seqlen/balanced_min:23452 - global_seqlen/balanced_max:23461 - global_seqlen/mean:23457.625 - actor/entropy:0.45124438405036926 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.47588256001472473 - training/rollout_probs_diff_mean:0.004685731139034033 - training/rollout_probs_diff_std:0.010418391786515713 - training/rollout_actor_probs_pearson_corr:0.999316930770874 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.76171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:0.9999880194664001 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.4401948394370265e-05 - rollout_corr/rollout_is_max:1.8073612451553345 - rollout_corr/rollout_is_min:0.04583345353603363 - rollout_corr/rollout_is_std:0.03505437821149826 - rollout_corr/rollout_is_eff_sample_size:0.9987726987625546 - rollout_corr/rollout_is_seq_mean:0.9999796152114868 - rollout_corr/rollout_is_seq_std:0.0017362802755087614 - rollout_corr/rollout_is_seq_max:1.0060876607894897 - rollout_corr/rollout_is_seq_min:0.9951460361480713 - rollout_corr/rollout_is_seq_max_deviation:0.006087660789489746 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.600627651438117) - rollout_corr/training_log_ppl:np.float64(0.4611963724019006) - rollout_corr/kl:np.float64(0.0007944866175790821) - rollout_corr/k3_kl:np.float64(0.000699135587751698) - rollout_corr/rollout_ppl:np.float64(1.599346871022135) - rollout_corr/rollout_log_ppl:np.float64(0.46040188503684476) - rollout_corr/log_ppl_diff:np.float64(0.0007944873650558293) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015184025396592915) - rollout_corr/log_ppl_diff_max:np.float64(0.007160961627960205) - rollout_corr/log_ppl_diff_min:np.float64(-0.004008620977401733) - rollout_corr/ppl_ratio:np.float64(1.0007963897660375) - rollout_corr/chi2_token:np.float64(0.0012088804505765438) - rollout_corr/chi2_seq:np.float64(0.3547051209025085) - actor/pg_loss:np.float64(0.0030129868537187576) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016773132483649533) - actor/ppo_kl:np.float64(0.00010856679642934353) - actor/pg_clipfrac_lower:np.float64(7.659314178454224e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_w_mean:np.float64(4465.609730498167) - self_distillation/token_reweight_w_std:np.float64(81083.046648157) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998130949679762) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04856777528766543) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.18460461869835854) - perf/mfu/actor:np.float64(0.05546533287041886) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.40380096435547) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:1 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005895104259252548 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.005895104259252548 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:480
(TaskRunner pid=2255397) 
Training Progress:  94%|█████████▍| 94/100 [1:27:16<05:17, 52.92s/it]
(TaskRunner pid=2255397) step:95 - global_seqlen/min:18176 - global_seqlen/max:26822 - global_seqlen/minmax_diff:8646 - global_seqlen/balanced_min:23320 - global_seqlen/balanced_max:23330 - global_seqlen/mean:23326.75 - actor/entropy:0.42938148975372314 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102376401424408 - training/rollout_probs_diff_mean:0.0046544442884624004 - training/rollout_probs_diff_std:0.010648963041603565 - training/rollout_actor_probs_pearson_corr:0.9992654323577881 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.0000568628311157 - rollout_corr/rollout_is_ratio_fraction_high:1.6773175957496278e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.386587978748139e-06 - rollout_corr/rollout_is_max:2.396075487136841 - rollout_corr/rollout_is_min:0.3959798514842987 - rollout_corr/rollout_is_std:0.035107050091028214 - rollout_corr/rollout_is_eff_sample_size:0.9987691312695665 - rollout_corr/rollout_is_seq_mean:1.0000056028366089 - rollout_corr/rollout_is_seq_std:0.0018295958871021867 - rollout_corr/rollout_is_seq_max:1.0062776803970337 - rollout_corr/rollout_is_seq_min:0.9954047203063965 - rollout_corr/rollout_is_seq_max_deviation:0.006277680397033691 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5648417291231453) - rollout_corr/training_log_ppl:np.float64(0.4370654333033599) - rollout_corr/kl:np.float64(0.0006150017093133897) - rollout_corr/k3_kl:np.float64(0.0007087535628897967) - rollout_corr/rollout_ppl:np.float64(1.5638857381418347) - rollout_corr/rollout_log_ppl:np.float64(0.43645042926073074) - rollout_corr/log_ppl_diff:np.float64(0.0006150040426291525) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015464308089576662) - rollout_corr/log_ppl_diff_max:np.float64(0.005993485450744629) - rollout_corr/log_ppl_diff_min:np.float64(-0.004650503396987915) - rollout_corr/ppl_ratio:np.float64(1.000617001671344) - rollout_corr/chi2_token:np.float64(0.0016058182809501886) - rollout_corr/chi2_seq:np.float64(0.5998094449751079) - actor/pg_loss:np.float64(0.005259391153231263) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024053280617408745) - actor/ppo_kl:np.float64(-4.200763549278008e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(142.92776992870495) - self_distillation/token_reweight_w_std:np.float64(3127.1786301988177) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005555916577578) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0809866230993066) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.24522369727492332) - perf/mfu/actor:np.float64(0.05459364258201367) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.3312644958496) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017624415457248688 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.017624415457248688 - critic/returns/max:0.875 - critic/returns/min:-0.75 
(TaskRunner pid=2255397) 
Training Progress:  95%|█████████▌| 95/100 [1:27:59<04:09, 49.88s/it]
(TaskRunner pid=2255397) step:96 - global_seqlen/min:20301 - global_seqlen/max:28288 - global_seqlen/minmax_diff:7987 - global_seqlen/balanced_min:23030 - global_seqlen/balanced_max:23033 - global_seqlen/mean:23031.375 - actor/entropy:0.44385668635368347 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2772989869117737 - training/rollout_probs_diff_mean:0.004754771012812853 - training/rollout_probs_diff_std:0.010701586492359638 - training/rollout_actor_probs_pearson_corr:0.99928879737854 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.67578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.000027060508728 - rollout_corr/rollout_is_ratio_fraction_high:8.62165597936837e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.084261417388916 - rollout_corr/rollout_is_min:0.5124618411064148 - rollout_corr/rollout_is_std:0.036241523921489716 - rollout_corr/rollout_is_eff_sample_size:0.9986881560335953 - rollout_corr/rollout_is_seq_mean:1.0000017881393433 - rollout_corr/rollout_is_seq_std:0.0018420576816424727 - rollout_corr/rollout_is_seq_max:1.0063093900680542 - rollout_corr/rollout_is_seq_min:0.9944624304771423 - rollout_corr/rollout_is_seq_max_deviation:0.006309390068054199 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.598726301919669) - rollout_corr/training_log_ppl:np.float64(0.45704233669675887) - rollout_corr/kl:np.float64(0.0006188384301958338) - rollout_corr/k3_kl:np.float64(0.0007431540022935224) - rollout_corr/rollout_ppl:np.float64(1.5976339941844344) - rollout_corr/rollout_log_ppl:np.float64(0.45642349659465253) - rollout_corr/log_ppl_diff:np.float64(0.0006188401021063328) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015413255896419287) - rollout_corr/log_ppl_diff_max:np.float64(0.0064958930015563965) - rollout_corr/log_ppl_diff_min:np.float64(-0.0055953264236450195) - rollout_corr/ppl_ratio:np.float64(1.0006208247505128) - rollout_corr/chi2_token:np.float64(0.0018268234562128782) - rollout_corr/chi2_seq:np.float64(1.2716746137011796) - actor/pg_loss:np.float64(0.004606917384080589) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021080635951875593) - actor/ppo_kl:np.float64(-7.161125205357166e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_w_mean:np.float64(99.39291401347145) - self_distillation/token_reweight_w_std:np.float64(2908.174926977954) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999677997548133) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08357357300701551) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.23777736723423004) - perf/mfu/actor:np.float64(0.05523982050510799) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.2464942932129) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:1 - critic/score/mean:0.67578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.67578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003513324772939086 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003513324772939086 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/m
(TaskRunner pid=2255397) 
Training Progress:  96%|█████████▌| 96/100 [1:28:41<03:10, 47.53s/it]
(TaskRunner pid=2255397) step:97 - global_seqlen/min:21184 - global_seqlen/max:28019 - global_seqlen/minmax_diff:6835 - global_seqlen/balanced_min:24523 - global_seqlen/balanced_max:24538 - global_seqlen/mean:24532.875 - actor/entropy:0.43073078989982605 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5830099582672119 - training/rollout_probs_diff_mean:0.004653396084904671 - training/rollout_probs_diff_std:0.010732254013419151 - training/rollout_actor_probs_pearson_corr:0.9992837905883789 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.765625 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.765625 - rollout_corr/rollout_is_mean:0.999879002571106 - rollout_corr/rollout_is_ratio_fraction_high:8.271640581369866e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.654328116273973e-05 - rollout_corr/rollout_is_max:2.069552421569824 - rollout_corr/rollout_is_min:0.06639717519283295 - rollout_corr/rollout_is_std:0.03590112552046776 - rollout_corr/rollout_is_eff_sample_size:0.9987124115549366 - rollout_corr/rollout_is_seq_mean:0.9998928308486938 - rollout_corr/rollout_is_seq_std:0.001641205046325922 - rollout_corr/rollout_is_seq_max:1.008506178855896 - rollout_corr/rollout_is_seq_min:0.9953399300575256 - rollout_corr/rollout_is_seq_max_deviation:0.008506178855895996 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5909629054367542) - rollout_corr/training_log_ppl:np.float64(0.45360937030636705) - rollout_corr/kl:np.float64(0.0007517823871872054) - rollout_corr/k3_kl:np.float64(0.0007286852571155578) - rollout_corr/rollout_ppl:np.float64(1.5898262090049684) - rollout_corr/rollout_log_ppl:np.float64(0.45285758542013355) - rollout_corr/log_ppl_diff:np.float64(0.0007517848862335086) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015345667488873005) - rollout_corr/log_ppl_diff_max:np.float64(0.00910806655883789) - rollout_corr/log_ppl_diff_min:np.float64(-0.007346630096435547) - rollout_corr/ppl_ratio:np.float64(1.0007539454381913) - rollout_corr/chi2_token:np.float64(0.00142996059730649) - rollout_corr/chi2_seq:np.float64(0.9373617491219193) - actor/pg_loss:np.float64(0.0030622705817222595) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003035150989489921) - actor/ppo_kl:np.float64(-6.880035599365897e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.765625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(482.64709699410014) - self_distillation/token_reweight_w_std:np.float64(7838.689288595924) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0027755422051996) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06985412852372974) - self_distillation/empty_target_batch:np.float64(0.234375) - actor/grad_norm:np.float64(0.2353973463177681) - perf/mfu/actor:np.float64(0.058007253459953285) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.27971267700195) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:1 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036426237784326077 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0036426237784326077 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:
(TaskRunner pid=2255397) 
Training Progress:  97%|█████████▋| 97/100 [1:29:24<02:18, 46.23s/it]
(TaskRunner pid=2255397) step:98 - global_seqlen/min:21081 - global_seqlen/max:28015 - global_seqlen/minmax_diff:6934 - global_seqlen/balanced_min:24041 - global_seqlen/balanced_max:24240 - global_seqlen/mean:24067.875 - actor/entropy:0.4481050968170166 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3950059413909912 - training/rollout_probs_diff_mean:0.0048210457898676395 - training/rollout_probs_diff_std:0.010794337838888168 - training/rollout_actor_probs_pearson_corr:0.9992629289627075 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:1.000078797340393 - rollout_corr/rollout_is_ratio_fraction_high:3.446166738285683e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.7230833691428415e-05 - rollout_corr/rollout_is_max:4.591516971588135 - rollout_corr/rollout_is_min:0.39532703161239624 - rollout_corr/rollout_is_std:0.03653962165117264 - rollout_corr/rollout_is_eff_sample_size:0.9986668739730418 - rollout_corr/rollout_is_seq_mean:1.000028133392334 - rollout_corr/rollout_is_seq_std:0.0019362398888915777 - rollout_corr/rollout_is_seq_max:1.0084097385406494 - rollout_corr/rollout_is_seq_min:0.9942269325256348 - rollout_corr/rollout_is_seq_max_deviation:0.008409738540649414 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.616344683803618) - rollout_corr/training_log_ppl:np.float64(0.4696293716551736) - rollout_corr/kl:np.float64(0.0008239057563663721) - rollout_corr/k3_kl:np.float64(0.000780692749458467) - rollout_corr/rollout_ppl:np.float64(1.6150271990336478) - rollout_corr/rollout_log_ppl:np.float64(0.46880546608008444) - rollout_corr/log_ppl_diff:np.float64(0.0008239055750891566) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015701879747211933) - rollout_corr/log_ppl_diff_max:np.float64(0.007200688123703003) - rollout_corr/log_ppl_diff_min:np.float64(-0.004303097724914551) - rollout_corr/ppl_ratio:np.float64(1.0008261122275144) - rollout_corr/chi2_token:np.float64(0.001507394015789032) - rollout_corr/chi2_seq:np.float64(0.24129036953672767) - actor/pg_loss:np.float64(0.003733934834599495) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000350107799476973) - actor/ppo_kl:np.float64(0.00011479198082042785) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(2963.129227875499) - self_distillation/token_reweight_w_std:np.float64(42459.57496579323) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0034989113919437) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09955895229359157) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.2265518605709076) - perf/mfu/actor:np.float64(0.057423452397621935) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.2193260192871) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:1 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008424799889326096 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008424799889326096 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:453.40234
(TaskRunner pid=2255397) 
Training Progress:  98%|█████████▊| 98/100 [1:30:08<01:30, 45.33s/it]
(TaskRunner pid=2255397) step:99 - global_seqlen/min:19692 - global_seqlen/max:25383 - global_seqlen/minmax_diff:5691 - global_seqlen/balanced_min:23698 - global_seqlen/balanced_max:23716 - global_seqlen/mean:23712.375 - actor/entropy:0.4742501676082611 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4091079831123352 - training/rollout_probs_diff_mean:0.004931129980832338 - training/rollout_probs_diff_std:0.010619550943374634 - training/rollout_actor_probs_pearson_corr:0.9992989301681519 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.765625 - self_distillation/correct_sample_fraction:0.578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.765625 - rollout_corr/rollout_is_mean:0.9998912215232849 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.113195119425654e-06 - rollout_corr/rollout_is_max:1.7866209745407104 - rollout_corr/rollout_is_min:0.43442291021347046 - rollout_corr/rollout_is_std:0.03628097102046013 - rollout_corr/rollout_is_eff_sample_size:0.9986851836242739 - rollout_corr/rollout_is_seq_mean:0.9998824596405029 - rollout_corr/rollout_is_seq_std:0.00225405883975327 - rollout_corr/rollout_is_seq_max:1.0066941976547241 - rollout_corr/rollout_is_seq_min:0.9925881028175354 - rollout_corr/rollout_is_seq_max_deviation:0.0074118971824646 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6540217595174909) - rollout_corr/training_log_ppl:np.float64(0.49091246334137395) - rollout_corr/kl:np.float64(0.0009071645049343324) - rollout_corr/k3_kl:np.float64(0.0007826964022683569) - rollout_corr/rollout_ppl:np.float64(1.6525658327154815) - rollout_corr/rollout_log_ppl:np.float64(0.4900052981101908) - rollout_corr/log_ppl_diff:np.float64(0.0009071652311831713) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018514589173719287) - rollout_corr/log_ppl_diff_max:np.float64(0.008251607418060303) - rollout_corr/log_ppl_diff_min:np.float64(-0.005766570568084717) - rollout_corr/ppl_ratio:np.float64(1.0009100551251322) - rollout_corr/chi2_token:np.float64(0.001357766566798091) - rollout_corr/chi2_seq:np.float64(1.0302318546455353) - actor/pg_loss:np.float64(0.004117259988561273) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030947469667808036) - actor/ppo_kl:np.float64(9.188319657127408e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.765625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.578125) - self_distillation/token_reweight_w_mean:np.float64(190.36794193601236) - self_distillation/token_reweight_w_std:np.float64(4398.700665255601) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010327426716685) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09465871116844937) - self_distillation/empty_target_batch:np.float64(0.234375) - actor/grad_norm:np.float64(0.2390700727701187) - perf/mfu/actor:np.float64(0.055725058096870825) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.29999542236328) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:1 - critic/score/mean:0.578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006170772016048431 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006170772016048431 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:428.63671875 
(TaskRunner pid=2255397) 
Training Progress:  99%|█████████▉| 99/100 [1:30:51<00:44, 44.60s/it]
(TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100}
(TaskRunner pid=2255397) validation generation end
(TaskRunner pid=2255397) [prompt] system
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2255397) <reasoning>
(TaskRunner pid=2255397) ...
(TaskRunner pid=2255397) </reasoning>
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) ...
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) A
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) user
(TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2255397) Please reason step by step.
(TaskRunner pid=2255397) assistant
(TaskRunner pid=2255397) <think>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) </think>
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) [response] <reasoning>
(TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"**, we need to analyze the structure of the product and compare it with the structures of the given options.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) The product contains:
(TaskRunner pid=2255397) - A **nitrogen-containing ring** (likely an imine or azole ring),
(TaskRunner pid=2255397) - A **chlorine atom** attached to a benzene ring,
(TaskRunner pid=2255397) - A **trifluoromethyl group** (C(F)(F)F),
(TaskRunner pid=2255397) - A **nitrogen atom** connected to a carbon chain containing a **chlorinated benzene ring** and a **fluorinated group**.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Now, let’s examine the options:
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option A:**
(TaskRunner pid=2255397) - Contains a **nitrogen-containing ring** (likely a pyrrole or similar),
(TaskRunner pid=2255397) - A **chlorinated benzene ring** (with Cl),
(TaskRunner pid=2255397) - A **trifluoromethyl group** (C(F)(F)F),
(TaskRunner pid=2255397) - A **nitrogen atom** connected to a carbon chain that includes the **chlorinated benzene** and **fluorinated group**.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) This matches the structure of the target product very closely, including all the key functional groups and their positions.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option B:**
(TaskRunner pid=2255397) - Contains a **chlorine** and **nitrogen**, but the **fluorinated group** is not present in the correct position, and the **nitrogen ring** is not properly structured.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option C:**
(TaskRunner pid=2255397) - Contains a **fluorinated group**, but the **chlorine** is attached to an **oxygen**, not directly to the benzene ring. The **fluorinated group** is in a different position than in the product.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) **Option D:**
(TaskRunner pid=2255397) - Contains a **fluorinated group** and a **chlorinated benzene ring**, but the **nitrogen-containing ring** is not present in the correct structure.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) Only **Option A** contains all the required functional groups in the correct positions to serve as the **only correct reactant** for the synthesis of the given product.
(TaskRunner pid=2255397) 
(TaskRunner pid=2255397) </reasoning>
(TaskRunner pid=2255397) <answer>
(TaskRunner pid=2255397) A
(TaskRunner pid=2255397) </answer>
(TaskRunner pid=2255397) [ground_truth] A
(TaskRunner pid=2255397) [score] 1.0
(TaskRunner pid=2255397) [acc] 1.0
(TaskRunner pid=2255397) [pred] A
(TaskRunner pid=2255397) [incorrect_format] 1
(TaskRunner pid=2255397) [feedback] 
(TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_100
(TaskRunner pid=2255397) step:100 - global_seqlen/min:23741 - global_seqlen/max:31057 - global_seqlen/minmax_diff:7316 - global_seqlen/balanced_min:24739 - global_seqlen/balanced_max:31757 - global_seqlen/mean:25617.625 - actor/entropy:0.4436495304107666 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24008721113204956 - training/rollout_probs_diff_mean:0.004401609301567078 - training/rollout_probs_diff_std:0.009758861735463142 - training/rollout_actor_probs_pearson_corr:0.9993872046470642 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.91796875 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.91796875 - rollout_corr/rollout_is_mean:1.0000593662261963 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7518070936203003 - rollout_corr/rollout_is_min:0.5589723587036133 - rollout_corr/rollout_is_std:0.03378680348396301 - rollout_corr/rollout_is_eff_sample_size:0.9988599913647978 - rollout_corr/rollout_is_seq_mean:1.0000724792480469 - rollout_corr/rollout_is_seq_std:0.0018252491718158126 - rollout_corr/rollout_is_seq_max:1.0057400465011597 - rollout_corr/rollout_is_seq_min:0.9935486912727356 - rollout_corr/rollout_is_seq_max_deviation:0.006451308727264404 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6480820523574948) - rollout_corr/training_log_ppl:np.float64(0.4857633382116546) - rollout_corr/kl:np.float64(0.0007351505751955756) - rollout_corr/k3_kl:np.float64(0.0006855184932135217) - rollout_corr/rollout_ppl:np.float64(1.6468097120523453) - rollout_corr/rollout_log_ppl:np.float64(0.48502818830229444) - rollout_corr/log_ppl_diff:np.float64(0.0007351499093601888) - rollout_corr/log_ppl_abs_diff:np.float64(0.001427958528893214) - rollout_corr/log_ppl_diff_max:np.float64(0.007762610912322998) - rollout_corr/log_ppl_diff_min:np.float64(-0.004005610942840576) - rollout_corr/ppl_ratio:np.float64(1.0007370123639703) - rollout_corr/chi2_token:np.float64(0.0012815699446946383) - rollout_corr/chi2_seq:np.float64(0.31913869245909154) - actor/pg_loss:np.float64(0.003367550321854651) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003017752374034899) - actor/ppo_kl:np.float64(0.00018170115079385596) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.91796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.91796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(1570.6249493323267) - self_distillation/token_reweight_w_std:np.float64(40484.064735737746) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001023317920044) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0961508322507143) - self_distillation/empty_target_batch:np.float64(0.08203125) - actor/grad_norm:np.float64(0.2478880025446415) - perf/mfu/actor:np.float64(0.05716674172282474) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.12733459472656) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7065476190476191) - val-aux/sciknoweval/reward/std@16:np.float64(0.14963967195739616) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7672380952380953) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.12522806043858783) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6460952380952382) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.12793485078945213) - val-aux/sciknow
(TaskRunner pid=2255397) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': "
(TaskRunner pid=2255397)  "np.float64(0.7065476190476191), 'val-aux/sciknoweval/reward/std@16': "
(TaskRunner pid=2255397)  "np.float64(0.14963967195739616), 'val-aux/sciknoweval/reward/best@2/mean': "
(TaskRunner pid=2255397)  "np.float64(0.7672380952380953), 'val-aux/sciknoweval/reward/best@2/std': "
(TaskRunner pid=2255397)  "np.float64(0.12522806043858783), 'val-aux/sciknoweval/reward/worst@2/mean': "
(TaskRunner pid=2255397)  "np.float64(0.6460952380952382), 'val-aux/sciknoweval/reward/worst@2/std': "
(TaskRunner pid=2255397)  "np.float64(0.12793485078945213), 'val-aux/sciknoweval/reward/maj@2/mean': "
(TaskRunner pid=2255397)  "np.float64(0.7068333333333333), 'val-aux/sciknoweval/reward/maj@2/std': "
(TaskRunner pid=2255397)  "np.float64(0.14978365629442147), 'val-aux/sciknoweval/reward/best@4/mean': "
(TaskRunner pid=2255397)  "np.float64(0.8158333333333333), 'val-aux/sciknoweval/reward/best@4/std': "
(TaskRunner pid=2255397)  "np.float64(0.09352166658220648), 'val-aux/sciknoweval/reward/worst@4/mean': "
(TaskRunner pid=2255397)  "np.float64(0.5964190476190476), 'val-aux/sciknoweval/reward/worst@4/std': "
(TaskRunner pid=2255397)  "np.float64(0.09674390853009297), 'val-aux/sciknoweval/reward/maj@4/mean': "
(TaskRunner pid=2255397)  "np.float64(0.7151904761904762), 'val-aux/sciknoweval/reward/maj@4/std': "
(TaskRunner pid=2255397)  "np.float64(0.1197973344827459), 'val-aux/sciknoweval/reward/best@8/mean': "
(TaskRunner pid=2255397)  "np.float64(0.8520904761904763), 'val-aux/sciknoweval/reward/best@8/std': "
(TaskRunner pid=2255397)  "np.float64(0.06467821428517558), 'val-aux/sciknoweval/reward/worst@8/mean': "
(TaskRunner pid=2255397)  "np.float64(0.5596142857142857), 'val-aux/sciknoweval/reward/worst@8/std': "
(TaskRunner pid=2255397)  "np.float64(0.06871965760347537), 'val-aux/sciknoweval/reward/maj@8/mean': "
(TaskRunner pid=2255397)  "np.float64(0.7202857142857142), 'val-aux/sciknoweval/reward/maj@8/std': "
(TaskRunner pid=2255397)  "np.float64(0.08852864730736695), 'val-aux/sciknoweval/reward/best@16/mean': "
(TaskRunner pid=2255397)  "np.float64(0.8776952380952381), 'val-aux/sciknoweval/reward/best@16/std': "
(TaskRunner pid=2255397)  "np.float64(0.0403808452308748), 'val-aux/sciknoweval/reward/worst@16/mean': "
(TaskRunner pid=2255397)  "np.float64(0.5314904761904762), 'val-aux/sciknoweval/reward/worst@16/std': "
(TaskRunner pid=2255397)  "np.float64(0.04527152807209723), 'val-aux/sciknoweval/reward/maj@16/mean': "
(TaskRunner pid=2255397)  "np.float64(0.7233190476190475), 'val-aux/sciknoweval/reward/maj@16/std': "
(TaskRunner pid=2255397)  "np.float64(0.06487826170016521), 'val-aux/sciknoweval/score/mean@16': "
(TaskRunner pid=2255397)  "np.float64(0.7065476190476191), 'val-aux/sciknoweval/score/std@16': "
(TaskRunner pid=2255397)  "np.float64(0.14963967195739616), 'val-aux/sciknoweval/score/best@2/mean': "
(TaskRunner pid=2255397)  "np.float64(0.7672380952380953), 'val-aux/sciknoweval/score/best@2/std': "
(TaskRunner pid=2255397)  "np.float64(0.12522806043858783), 'val-aux/sciknoweval/score/worst@2/mean': "
(TaskRunner pid=2255397)  "np.float64(0.6460952380952382), 'val-aux/sciknoweval/score/worst@2/std': "
(TaskRunner pid=2255397)  "np.float64(0.12793485078945213), 'val-aux/sciknoweval/score/maj@2/mean': "
(TaskRunner pid=2255397)  "np.float64(0.7068333333333333), 'val-aux/sciknoweval/score/maj@2/std': "
(TaskRunner pid=2255397)  "np.float64(0.14978365629442147), 'val-aux/sciknoweval/score/best@4/mean': "
(TaskRunner pid=2255397)  "np.float64(0.8158333333333333), 'val-aux/sciknoweval/score/best@4/std': "
(TaskRunner pid=2255397)  "np.float64(0.09352166658220648), 'val-aux/sciknoweval/score/worst@4/mean': "
(TaskRunner pid=2255397)  "np.float64(0.5964190476190476), 'val-aux/sciknoweval/score/worst@4/std': "
(TaskRunner pid=2255397)  "np.float64(0.09674390853009297), 'val-aux/sciknoweval/score/maj@4/mean': "
(TaskRunner pid=2255397)  "np.float64(0.7151904761904762), 'val-aux/sciknoweval/score/maj@4/std': "
(TaskRunner pid=2255397)  "np.float64(0.1197973344827459), 'val-aux/sciknoweval/score/best@8/mean': "
(TaskRunner pid=2255397)  "np.float64(0.8520904761904763), 'val-aux/sciknoweval/score/best@8/std': "
(TaskRunner pid=2255397)  "np.float64(0.06467821428517558), 'val-aux/sciknoweval/score/worst@8/mean': "
(TaskRunner pid=2255397)  "np.float64(0.5596142857142857), 'val-aux/sciknoweval/score/worst@8/std': "
(TaskRunner pid=2255397)  "np.float64(0.06871965760347537), 'val-aux/sciknoweval/score/maj@8/mean': "
(TaskRunner pid=2255397)  "np.float64(0.7202857142857142), 'val-aux/sciknoweval/score/maj@8/std': "
(TaskRunner pid=2255397)  "np.float64(0.08852864730736695), 'val-aux/sciknoweval/score/best@16/mean': "
(TaskRunner pid=2255397)  "np.float64(0.8776952380952381), 'val-aux/sciknoweval/score/best@16/std': "
(TaskRunner pid=2255397)  "np.float64(0.0403808452308748), 'val-aux/sciknoweval/score/worst@16/mean': "
(TaskRunner pid=2255397)  "np.float64(0.5314904761904762), 'val-aux/sciknoweval/score/worst@16/std': "
(TaskRunner pid=2255397)  "np.float64(0.04527152807209723), 'val-aux/sciknoweval/score/maj@16/mean': "
(TaskRunner pid=2255397)  "np.float64(0.7233190476190475), 'val-aux/sciknoweval/score/maj@16/std': "
(TaskRunner pid=2255397)  "np.float64(0.06487826170016521), 'val-core/sciknoweval/acc/mean@16': "
(TaskRunner pid=2255397)  "np.float64(0.7065476190476191), 'val-aux/sciknoweval/acc/std@16': "
(TaskRunner pid=2255397)  "np.float64(0.14963967195739616), 'val-aux/sciknoweval/acc/best@2/mean': "
(TaskRunner pid=2255397)  "np.float64(0.7672380952380953), 'val-aux/sciknoweval/acc/best@2/std': "
(TaskRunner pid=2255397)  "np.float64(0.12522806043858783), 'val-aux/sciknoweval/acc/worst@2/mean': "
(TaskRunner pid=2255397)  "np.float64(0.6460952380952382), 'val-aux/sciknoweval/acc/worst@2/std': "
(TaskRunner pid=2255397)  "np.float64(0.12793485078945213), 'val-aux/sciknoweval/acc/maj@2/mean': "
(TaskRunner pid=2255397)  "np.float64(0.7068333333333333), 'val-aux/sciknoweval/acc/maj@2/std': "
(TaskRunner pid=2255397)  "np.float64(0.14978365629442147), 'val-aux/sciknoweval/acc/best@4/mean': "
(TaskRunner pid=2255397)  "np.float64(0.8158333333333333), 'val-aux/sciknoweval/acc/best@4/std': "
(TaskRunner pid=2255397)  "np.float64(0.09352166658220648), 'val-aux/sciknoweval/acc/worst@4/mean': "
(TaskRunner pid=2255397)  "np.float64(0.5964190476190476), 'val-aux/sciknoweval/acc/worst@4/std': "
(TaskRunner pid=2255397)  "np.float64(0.09674390853009297), 'val-aux/sciknoweval/acc/maj@4/mean': "
(TaskRunner pid=2255397)  "np.float64(0.7151904761904762), 'val-aux/sciknoweval/acc/maj@4/std': "
(TaskRunner pid=2255397)  "np.float64(0.1197973344827459), 'val-aux/sciknoweval/acc/best@8/mean': "
(TaskRunner pid=2255397)  "np.float64(0.8520904761904763), 'val-aux/sciknoweval/acc/best@8/std': "
(TaskRunner pid=2255397)  "np.float64(0.06467821428517558), 'val-aux/sciknoweval/acc/worst@8/mean': "
(TaskRunner pid=2255397)  "np.float64(0.5596142857142857), 'val-aux/sciknoweval/acc/worst@8/std': "
(TaskRunner pid=2255397)  "np.float64(0.06871965760347537), 'val-aux/sciknoweval/acc/maj@8/mean': "
(TaskRunner pid=2255397)  "np.float64(0.7202857142857142), 'val-aux/sciknoweval/acc/maj@8/std': "
(TaskRunner pid=2255397)  "np.float64(0.08852864730736695), 'val-core/sciknoweval/acc/best@16/mean': "
(TaskRunner pid=2255397)  "np.float64(0.8776952380952381), 'val-core/sciknoweval/acc/best@16/std': "
(TaskRunner pid=2255397)  "np.float64(0.0403808452308748), 'val-aux/sciknoweval/acc/worst@16/mean': "
(TaskRunner pid=2255397)  "np.float64(0.5314904761904762), 'val-aux/sciknoweval/acc/worst@16/std': "
(TaskRunner pid=2255397)  "np.float64(0.04527152807209723), 'val-core/sciknoweval/acc/maj@16/mean': "
(TaskRunner pid=2255397)  "np.float64(0.7233190476190475), 'val-core/sciknoweval/acc/maj@16/std': "
(TaskRunner pid=2255397)  'np.float64(0.06487826170016521), '
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/mean@16': "
(TaskRunner pid=2255397)  'np.float64(0.9982142857142857), '
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/std@16': "
(TaskRunner pid=2255397)  'np.float64(0.006916041689656103), '
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/best@2/mean': "
(TaskRunner pid=2255397)  'np.float64(0.999904761904762), '
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/best@2/std': "
(TaskRunner pid=2255397)  'np.float64(0.0016180983939162673), '
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/worst@2/mean': "
(TaskRunner pid=2255397)  'np.float64(0.9966190476190476), '
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/worst@2/std': "
(TaskRunner pid=2255397)  'np.float64(0.009226171602153282), '
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/maj@2/mean': "
(TaskRunner pid=2255397)  'np.float64(0.9982380952380953), '
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/maj@2/std': "
(TaskRunner pid=2255397)  'np.float64(0.006862312067166463), '
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/best@4/mean': np.float64(1.0), "
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/best@4/std': np.float64(0.0), "
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/worst@4/mean': "
(TaskRunner pid=2255397)  'np.float64(0.9934619047619048), '
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/worst@4/std': "
(TaskRunner pid=2255397)  'np.float64(0.012001753208129275), '
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/maj@4/mean': "
(TaskRunner pid=2255397)  'np.float64(0.9993142857142857), '
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/maj@4/std': "
(TaskRunner pid=2255397)  'np.float64(0.0043365672892029085), '
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), "
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), "
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/worst@8/mean': "
(TaskRunner pid=2255397)  'np.float64(0.988647619047619), '
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/worst@8/std': "
(TaskRunner pid=2255397)  'np.float64(0.01397823789672266), '
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/maj@8/mean': "
(TaskRunner pid=2255397)  'np.float64(0.9998380952380953), '
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/maj@8/std': "
(TaskRunner pid=2255397)  'np.float64(0.0019314678388328032), '
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), "
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), "
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/worst@16/mean': "
(TaskRunner pid=2255397)  'np.float64(0.9817619047619048), '
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/worst@16/std': "
(TaskRunner pid=2255397)  'np.float64(0.013705261025670075), '
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/maj@16/mean': "
(TaskRunner pid=2255397)  'np.float64(0.9999714285714286), '
(TaskRunner pid=2255397)  "'val-aux/sciknoweval/incorrect_format/maj@16/std': "
(TaskRunner pid=2255397)  "np.float64(0.0006015850374731164), 'val-aux/num_turns/min': np.int32(2), "
(TaskRunner pid=2255397)  "'val-aux/num_turns/max': np.int32(2), 'val-aux/num_turns/mean': "
(TaskRunner pid=2255397)  'np.float64(2.0)}')
(TaskRunner pid=2255397) 
Training Progress: 100%|██████████| 100/100 [1:35:02<00:00, 106.78s/it]
Training Progress: 100%|██████████| 100/100 [1:35:02<00:00, 57.03s/it] 
(TaskRunner pid=2255397) Traceback (most recent call last):
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/workers/default_worker.py", line 322, in <module>
(TaskRunner pid=2255397)     worker.main_loop()
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py", line 1031, in main_loop
(TaskRunner pid=2255397)     self.core_worker.run_task_loop()
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/utils/data/_utils/signal_handling.py", line 73, in handler
(TaskRunner pid=2255397)     _error_if_any_worker_fails()
(TaskRunner pid=2255397) RuntimeError: DataLoader worker (pid 2270092) is killed by signal: Killed. 
(TaskRunner pid=2255397) Traceback (most recent call last):
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/workers/default_worker.py", line 322, in <module>
(TaskRunner pid=2255397)     worker.main_loop()
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py", line 1031, in main_loop
(TaskRunner pid=2255397)     self.core_worker.run_task_loop()
(TaskRunner pid=2255397) Exception ignored in: <function Tracking.__del__ at 0x7c4b4a93e020>
(TaskRunner pid=2255397) Traceback (most recent call last):
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/verl/utils/tracking.py", line 169, in __del__
(TaskRunner pid=2255397)     self.logger["wandb"].finish(exit_code=0)
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 4097, in finish
(TaskRunner pid=2255397)     wandb.run.finish(exit_code=exit_code, quiet=quiet)
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 400, in wrapper
(TaskRunner pid=2255397)     return func(self, *args, **kwargs)
(TaskRunner pid=2255397)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 445, in wrapper
(TaskRunner pid=2255397)     return func(self, *args, **kwargs)
(TaskRunner pid=2255397)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 2275, in finish
(TaskRunner pid=2255397)     return self._finish(exit_code)
(TaskRunner pid=2255397)            ^^^^^^^^^^^^^^^^^^^^^^^
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 400, in wrapper
(TaskRunner pid=2255397)     return func(self, *args, **kwargs)
(TaskRunner pid=2255397)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 2288, in _finish
(TaskRunner pid=2255397)     with telemetry.context(run=self) as tel:
(TaskRunner pid=2255397)          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/telemetry.py", line 42, in __exit__
(TaskRunner pid=2255397)     self._run._telemetry_callback(self._obj)
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 752, in _telemetry_callback
(TaskRunner pid=2255397)     self._telemetry_flush()
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 765, in _telemetry_flush
(TaskRunner pid=2255397)     self._backend.interface._publish_telemetry(self._telemetry_obj)
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/interface/interface_shared.py", line 103, in _publish_telemetry
(TaskRunner pid=2255397)     self._publish(rec)
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/interface/interface_sock.py", line 46, in _publish
(TaskRunner pid=2255397)     self._asyncer.run(lambda: self._client.publish(request))
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run
(TaskRunner pid=2255397)     return future.result()
(TaskRunner pid=2255397)            ^^^^^^^^^^^^^^^
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result
(TaskRunner pid=2255397)     return self.__get_result()
(TaskRunner pid=2255397)            ^^^^^^^^^^^^^^^^^^^
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
(TaskRunner pid=2255397)     raise self._exception
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap
(TaskRunner pid=2255397)     return await fn()
(TaskRunner pid=2255397)            ^^^^^^^^^^
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 38, in publish
(TaskRunner pid=2255397)     await self._send_server_request(request)
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 59, in _send_server_request
(TaskRunner pid=2255397)     self._writer.write(header)
(TaskRunner pid=2255397)   File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 346, in write
(TaskRunner pid=2255397)     self._transport.write(data)
(TaskRunner pid=2255397)   File "uvloop/handles/stream.pyx", line 675, in uvloop.loop.UVStream.write
(TaskRunner pid=2255397)   File "uvloop/handles/handle.pyx", line 159, in uvloop.loop.UVHandle._ensure_alive
(TaskRunner pid=2255397) RuntimeError: unable to perform operation on <UnixTransport closed=True reading=False 0x7c4aaf7460c0>; the handler is closed
main_ppo exit code: 0
[2026-07-03 03:18:43] Finished Qwen/Qwen3-8B chemistry rlsd_tr.
[2026-07-03 03:18:43] Starting Qwen/Qwen3-8B physics grpo.
Experiment: qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8
Dataset:    physics
Method:     grpo
Config:     baseline_grpo
Model:      Qwen/Qwen3-8B
Train:      /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet
Val:        /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet
Ray tmp:    /tmp/ray_q3g_physics_grpo_Qwen3_8B
2026-07-03 03:18:44,953	INFO scripts.py:1364 -- Did not find any active Ray processes.
Experiment: qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8
Config: baseline_grpo
Task: datasets/sciknoweval/physics
Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-GRPO-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_physics_grpo_Qwen3_8B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/physics +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-8B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 trainer.max_actor_ckpt_to_keep=100 actor_rollout_ref.actor.policy_loss.loss_mode=vanilla actor_rollout_ref.actor.kl_loss_coef=0.0
ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_physics_grpo_Qwen3_8B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/physics', 'EXPERIMENT': 'qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}}
2026-07-03 03:18:54,758	INFO worker.py:2007 -- Started a local Ray instance.
/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
  warnings.warn(
(TaskRunner pid=2281424) TaskRunner hostname: mole-workspace-rw, PID: 2281424
(TaskRunner pid=2281424) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig',
(TaskRunner pid=2281424)                                  'calculate_entropy': False,
(TaskRunner pid=2281424)                                  'calculate_sum_pi_squared': False,
(TaskRunner pid=2281424)                                  'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig',
(TaskRunner pid=2281424)                                                 'async_save': False,
(TaskRunner pid=2281424)                                                 'load_contents': ['model',
(TaskRunner pid=2281424)                                                                   'optimizer',
(TaskRunner pid=2281424)                                                                   'extra'],
(TaskRunner pid=2281424)                                                 'save_contents': ['model',
(TaskRunner pid=2281424)                                                                   'optimizer',
(TaskRunner pid=2281424)                                                                   'extra']},
(TaskRunner pid=2281424)                                  'clip_ratio': 0.2,
(TaskRunner pid=2281424)                                  'clip_ratio_c': 3.0,
(TaskRunner pid=2281424)                                  'clip_ratio_high': 0.28,
(TaskRunner pid=2281424)                                  'clip_ratio_low': 0.2,
(TaskRunner pid=2281424)                                  'data_loader_seed': 42,
(TaskRunner pid=2281424)                                  'entropy_checkpointing': False,
(TaskRunner pid=2281424)                                  'entropy_coeff': 0,
(TaskRunner pid=2281424)                                  'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2281424)                                  'freeze_vision_tower': False,
(TaskRunner pid=2281424)                                  'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2281424)                                                  'dtype': 'bfloat16',
(TaskRunner pid=2281424)                                                  'entropy_checkpointing': False,
(TaskRunner pid=2281424)                                                  'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2281424)                                                  'forward_only': False,
(TaskRunner pid=2281424)                                                  'forward_prefetch': False,
(TaskRunner pid=2281424)                                                  'fsdp_size': -1,
(TaskRunner pid=2281424)                                                  'full_determinism': False,
(TaskRunner pid=2281424)                                                  'model_dtype': 'fp32',
(TaskRunner pid=2281424)                                                  'offload_policy': False,
(TaskRunner pid=2281424)                                                  'optimizer_offload': False,
(TaskRunner pid=2281424)                                                  'param_offload': False,
(TaskRunner pid=2281424)                                                  'reshard_after_forward': True,
(TaskRunner pid=2281424)                                                  'seed': 42,
(TaskRunner pid=2281424)                                                  'strategy': 'fsdp',
(TaskRunner pid=2281424)                                                  'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2281424)                                                  'use_orig_params': False,
(TaskRunner pid=2281424)                                                  'use_torch_compile': True,
(TaskRunner pid=2281424)                                                  'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2281424)                                  'grad_clip': 1.0,
(TaskRunner pid=2281424)                                  'kl_loss_coef': 0.0,
(TaskRunner pid=2281424)                                  'kl_loss_type': 'low_var_kl',
(TaskRunner pid=2281424)                                  'loss_agg_mode': 'token-mean',
(TaskRunner pid=2281424)                                  'loss_scale_factor': None,
(TaskRunner pid=2281424)                                  'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig',
(TaskRunner pid=2281424)                                            'betas': [0.9, 0.999],
(TaskRunner pid=2281424)                                            'clip_grad': 1.0,
(TaskRunner pid=2281424)                                            'lr': 1e-06,
(TaskRunner pid=2281424)                                            'lr_scheduler_type': 'constant',
(TaskRunner pid=2281424)                                            'lr_warmup_steps': 10,
(TaskRunner pid=2281424)                                            'lr_warmup_steps_ratio': 0.0,
(TaskRunner pid=2281424)                                            'min_lr_ratio': 0.0,
(TaskRunner pid=2281424)                                            'num_cycles': 0.5,
(TaskRunner pid=2281424)                                            'optimizer': 'AdamW',
(TaskRunner pid=2281424)                                            'optimizer_impl': 'torch.optim',
(TaskRunner pid=2281424)                                            'override_optimizer_config': None,
(TaskRunner pid=2281424)                                            'total_training_steps': -1,
(TaskRunner pid=2281424)                                            'warmup_style': None,
(TaskRunner pid=2281424)                                            'weight_decay': 0.01},
(TaskRunner pid=2281424)                                  'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig',
(TaskRunner pid=2281424)                                                  'clip_cov_lb': 1.0,
(TaskRunner pid=2281424)                                                  'clip_cov_ratio': 0.0002,
(TaskRunner pid=2281424)                                                  'clip_cov_ub': 5.0,
(TaskRunner pid=2281424)                                                  'kl_cov_ratio': 0.0002,
(TaskRunner pid=2281424)                                                  'loss_mode': 'vanilla',
(TaskRunner pid=2281424)                                                  'ppo_kl_coef': 0.1},
(TaskRunner pid=2281424)                                  'ppo_epochs': 1,
(TaskRunner pid=2281424)                                  'ppo_max_token_len_per_gpu': 10240,
(TaskRunner pid=2281424)                                  'ppo_micro_batch_size': None,
(TaskRunner pid=2281424)                                  'ppo_micro_batch_size_per_gpu': 1,
(TaskRunner pid=2281424)                                  'ppo_mini_batch_size': 8,
(TaskRunner pid=2281424)                                  'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2281424)                                               'all_ranks': False,
(TaskRunner pid=2281424)                                               'enable': False,
(TaskRunner pid=2281424)                                               'ranks': [],
(TaskRunner pid=2281424)                                               'save_path': 'outputs/profile',
(TaskRunner pid=2281424)                                               'tool': None,
(TaskRunner pid=2281424)                                               'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2281424)                                                                       'analysis': True,
(TaskRunner pid=2281424)                                                                       'contents': [],
(TaskRunner pid=2281424)                                                                       'discrete': False,
(TaskRunner pid=2281424)                                                                       'level': 'level0'},
(TaskRunner pid=2281424)                                                               'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2281424)                                                                        'discrete': False},
(TaskRunner pid=2281424)                                                               'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2281424)                                                                         'step_end': None,
(TaskRunner pid=2281424)                                                                         'step_start': 0},
(TaskRunner pid=2281424)                                                               'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2281424)                                                                                'stack_depth': 32,
(TaskRunner pid=2281424)                                                                                'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2281424)                                  'rollout_n': 8,
(TaskRunner pid=2281424)                                  'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig',
(TaskRunner pid=2281424)                                                    'mode': 'disabled',
(TaskRunner pid=2281424)                                                    'record_file': None,
(TaskRunner pid=2281424)                                                    'replay_file': None},
(TaskRunner pid=2281424)                                  'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig',
(TaskRunner pid=2281424)                                                        'alpha': 0.5,
(TaskRunner pid=2281424)                                                        'distillation_add_tail': True,
(TaskRunner pid=2281424)                                                        'distillation_topk': 100,
(TaskRunner pid=2281424)                                                        'dont_reprompt_on_self_success': True,
(TaskRunner pid=2281424)                                                        'environment_feedback_only_without_solution': True,
(TaskRunner pid=2281424)                                                        'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig',
(TaskRunner pid=2281424)                                                                             'enable': False,
(TaskRunner pid=2281424)                                                                             'loss_weight': 0.0,
(TaskRunner pid=2281424)                                                                             'mask': 'all'},
(TaskRunner pid=2281424)                                                        'feedback_template': '\n'
(TaskRunner pid=2281424)                                                                             'The '
(TaskRunner pid=2281424)                                                                             'following '
(TaskRunner pid=2281424)                                                                             'is '
(TaskRunner pid=2281424)                                                                             'feedback '
(TaskRunner pid=2281424)                                                                             'from '
(TaskRunner pid=2281424)                                                                             'your '
(TaskRunner pid=2281424)                                                                             'unsuccessful '
(TaskRunner pid=2281424)                                                                             'earlier '
(TaskRunner pid=2281424)                                                                             'attempt:\n'
(TaskRunner pid=2281424)                                                                             '\n'
(TaskRunner pid=2281424)                                                                             '{feedback_raw}',
(TaskRunner pid=2281424)                                                        'full_logit_distillation': True,
(TaskRunner pid=2281424)                                                        'include_environment_feedback': True,
(TaskRunner pid=2281424)                                                        'is_clip': 2,
(TaskRunner pid=2281424)                                                        'max_reprompt_len': 22528,
(TaskRunner pid=2281424)                                                        'remove_thinking_from_demonstration': False,
(TaskRunner pid=2281424)                                                        'reprompt_template': '{prompt}{solution}{feedback}\n'
(TaskRunner pid=2281424)                                                                             '\n'
(TaskRunner pid=2281424)                                                                             'Correctly '
(TaskRunner pid=2281424)                                                                             'solve '
(TaskRunner pid=2281424)                                                                             'the '
(TaskRunner pid=2281424)                                                                             'original '
(TaskRunner pid=2281424)                                                                             'question.',
(TaskRunner pid=2281424)                                                        'reprompt_truncation': 'right',
(TaskRunner pid=2281424)                                                        'solution_template': '\n'
(TaskRunner pid=2281424)                                                                             'Correct '
(TaskRunner pid=2281424)                                                                             'solution:\n'
(TaskRunner pid=2281424)                                                                             '\n'
(TaskRunner pid=2281424)                                                                             '{successful_previous_attempt}',
(TaskRunner pid=2281424)                                                        'srpo_dynamic_weighting': False,
(TaskRunner pid=2281424)                                                        'srpo_dynamic_weighting_temperature': 1.0,
(TaskRunner pid=2281424)                                                        'success_reward_threshold': 0.5,
(TaskRunner pid=2281424)                                                        'teacher_regularization': 'ema',
(TaskRunner pid=2281424)                                                        'teacher_update_rate': 0.0,
(TaskRunner pid=2281424)                                                        'token_reweight_decay_steps': None,
(TaskRunner pid=2281424)                                                        'token_reweight_eps_w': 0.2,
(TaskRunner pid=2281424)                                                        'token_reweight_lambda': 0.5},
(TaskRunner pid=2281424)                                  'shuffle': False,
(TaskRunner pid=2281424)                                  'strategy': 'fsdp',
(TaskRunner pid=2281424)                                  'sum_pi_squared_checkpointing': False,
(TaskRunner pid=2281424)                                  'tau_neg': 1.05,
(TaskRunner pid=2281424)                                  'tau_pos': 1.0,
(TaskRunner pid=2281424)                                  'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2281424)                                  'use_dynamic_bsz': False,
(TaskRunner pid=2281424)                                  'use_fused_kernels': False,
(TaskRunner pid=2281424)                                  'use_kl_loss': False,
(TaskRunner pid=2281424)                                  'use_prefix_grouper': False,
(TaskRunner pid=2281424)                                  'use_remove_padding': True,
(TaskRunner pid=2281424)                                  'use_torch_compile': True},
(TaskRunner pid=2281424)                        'hybrid_engine': True,
(TaskRunner pid=2281424)                        'model': {'_target_': 'verl.workers.config.HFModelConfig',
(TaskRunner pid=2281424)                                  'custom_chat_template': None,
(TaskRunner pid=2281424)                                  'enable_activation_offload': False,
(TaskRunner pid=2281424)                                  'enable_gradient_checkpointing': True,
(TaskRunner pid=2281424)                                  'exclude_modules': None,
(TaskRunner pid=2281424)                                  'external_lib': None,
(TaskRunner pid=2281424)                                  'fused_kernel_options': {'impl_backend': 'torch'},
(TaskRunner pid=2281424)                                  'hf_config_path': None,
(TaskRunner pid=2281424)                                  'lora_adapter_path': None,
(TaskRunner pid=2281424)                                  'lora_alpha': 16,
(TaskRunner pid=2281424)                                  'lora_rank': 0,
(TaskRunner pid=2281424)                                  'override_config': {},
(TaskRunner pid=2281424)                                  'path': 'Qwen/Qwen3-8B',
(TaskRunner pid=2281424)                                  'target_modules': 'all-linear',
(TaskRunner pid=2281424)                                  'tiled_mlp': {'enabled': False,
(TaskRunner pid=2281424)                                                'num_shards': 4},
(TaskRunner pid=2281424)                                  'tokenizer_path': None,
(TaskRunner pid=2281424)                                  'trust_remote_code': True,
(TaskRunner pid=2281424)                                  'use_fused_kernels': False,
(TaskRunner pid=2281424)                                  'use_liger': False,
(TaskRunner pid=2281424)                                  'use_remove_padding': True,
(TaskRunner pid=2281424)                                  'use_shm': False},
(TaskRunner pid=2281424)                        'nccl_timeout': 600,
(TaskRunner pid=2281424)                        'ref': {'_target_': 'verl.workers.config.FSDPActorConfig',
(TaskRunner pid=2281424)                                'entropy_checkpointing': False,
(TaskRunner pid=2281424)                                'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2281424)                                'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2281424)                                                'dtype': 'bfloat16',
(TaskRunner pid=2281424)                                                'entropy_checkpointing': False,
(TaskRunner pid=2281424)                                                'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2281424)                                                'forward_only': True,
(TaskRunner pid=2281424)                                                'forward_prefetch': False,
(TaskRunner pid=2281424)                                                'fsdp_size': -1,
(TaskRunner pid=2281424)                                                'full_determinism': False,
(TaskRunner pid=2281424)                                                'model_dtype': 'fp32',
(TaskRunner pid=2281424)                                                'offload_policy': False,
(TaskRunner pid=2281424)                                                'optimizer_offload': False,
(TaskRunner pid=2281424)                                                'param_offload': False,
(TaskRunner pid=2281424)                                                'reshard_after_forward': True,
(TaskRunner pid=2281424)                                                'seed': 42,
(TaskRunner pid=2281424)                                                'strategy': 'fsdp',
(TaskRunner pid=2281424)                                                'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2281424)                                                'use_orig_params': False,
(TaskRunner pid=2281424)                                                'use_torch_compile': True,
(TaskRunner pid=2281424)                                                'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2281424)                                'log_prob_max_token_len_per_gpu': 10240,
(TaskRunner pid=2281424)                                'log_prob_micro_batch_size': None,
(TaskRunner pid=2281424)                                'log_prob_micro_batch_size_per_gpu': 1,
(TaskRunner pid=2281424)                                'log_prob_use_dynamic_bsz': False,
(TaskRunner pid=2281424)                                'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2281424)                                             'all_ranks': False,
(TaskRunner pid=2281424)                                             'enable': False,
(TaskRunner pid=2281424)                                             'ranks': [],
(TaskRunner pid=2281424)                                             'save_path': 'outputs/profile',
(TaskRunner pid=2281424)                                             'tool': None,
(TaskRunner pid=2281424)                                             'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2281424)                                                                     'analysis': True,
(TaskRunner pid=2281424)                                                                     'contents': [],
(TaskRunner pid=2281424)                                                                     'discrete': False,
(TaskRunner pid=2281424)                                                                     'level': 'level0'},
(TaskRunner pid=2281424)                                                             'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2281424)                                                                      'discrete': False},
(TaskRunner pid=2281424)                                                             'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2281424)                                                                       'step_end': None,
(TaskRunner pid=2281424)                                                                       'step_start': 0},
(TaskRunner pid=2281424)                                                             'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2281424)                                                                              'stack_depth': 32,
(TaskRunner pid=2281424)                                                                              'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2281424)                                'rollout_n': 8,
(TaskRunner pid=2281424)                                'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig',
(TaskRunner pid=2281424)                                                  'mode': 'disabled',
(TaskRunner pid=2281424)                                                  'record_file': None,
(TaskRunner pid=2281424)                                                  'replay_file': None},
(TaskRunner pid=2281424)                                'strategy': 'fsdp',
(TaskRunner pid=2281424)                                'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2281424)                                'use_torch_compile': True},
(TaskRunner pid=2281424)                        'rollout': {'_target_': 'verl.workers.config.RolloutConfig',
(TaskRunner pid=2281424)                                    'agent': {'_target_': 'verl.workers.config.AgentLoopConfig',
(TaskRunner pid=2281424)                                              'agent_loop_config_path': None,
(TaskRunner pid=2281424)                                              'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig',
(TaskRunner pid=2281424)                                                                      'name': None,
(TaskRunner pid=2281424)                                                                      'path': None},
(TaskRunner pid=2281424)                                              'default_agent_loop': 'single_turn_agent',
(TaskRunner pid=2281424)                                              'num_workers': 8},
(TaskRunner pid=2281424)                                    'calculate_log_probs': True,
(TaskRunner pid=2281424)                                    'cudagraph_capture_sizes': None,
(TaskRunner pid=2281424)                                    'data_parallel_size': 1,
(TaskRunner pid=2281424)                                    'disable_log_stats': True,
(TaskRunner pid=2281424)                                    'do_sample': True,
(TaskRunner pid=2281424)                                    'dtype': 'bfloat16',
(TaskRunner pid=2281424)                                    'enable_chunked_prefill': True,
(TaskRunner pid=2281424)                                    'enable_prefix_caching': True,
(TaskRunner pid=2281424)                                    'enable_rollout_routing_replay': False,
(TaskRunner pid=2281424)                                    'enforce_eager': False,
(TaskRunner pid=2281424)                                    'engine_kwargs': {'sglang': {}, 'vllm': {}},
(TaskRunner pid=2281424)                                    'expert_parallel_size': 1,
(TaskRunner pid=2281424)                                    'free_cache_engine': True,
(TaskRunner pid=2281424)                                    'gpu_memory_utilization': 0.8,
(TaskRunner pid=2281424)                                    'ignore_eos': False,
(TaskRunner pid=2281424)                                    'layered_summon': False,
(TaskRunner pid=2281424)                                    'load_format': 'dummy',
(TaskRunner pid=2281424)                                    'log_prob_max_token_len_per_gpu': 10240,
(TaskRunner pid=2281424)                                    'log_prob_micro_batch_size': None,
(TaskRunner pid=2281424)                                    'log_prob_micro_batch_size_per_gpu': 1,
(TaskRunner pid=2281424)                                    'log_prob_use_dynamic_bsz': False,
(TaskRunner pid=2281424)                                    'logprobs_mode': 'processed_logprobs',
(TaskRunner pid=2281424)                                    'max_model_len': 10240,
(TaskRunner pid=2281424)                                    'max_num_batched_tokens': 10240,
(TaskRunner pid=2281424)                                    'max_num_seqs': 1024,
(TaskRunner pid=2281424)                                    'mode': 'async',
(TaskRunner pid=2281424)                                    'multi_stage_wake_up': False,
(TaskRunner pid=2281424)                                    'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig',
(TaskRunner pid=2281424)                                                   'enable': False,
(TaskRunner pid=2281424)                                                   'format': 'hermes',
(TaskRunner pid=2281424)                                                   'interaction_config_path': None,
(TaskRunner pid=2281424)                                                   'max_assistant_turns': None,
(TaskRunner pid=2281424)                                                   'max_parallel_calls': 1,
(TaskRunner pid=2281424)                                                   'max_tool_response_length': 256,
(TaskRunner pid=2281424)                                                   'max_user_turns': None,
(TaskRunner pid=2281424)                                                   'num_repeat_rollouts': None,
(TaskRunner pid=2281424)                                                   'tokenization_sanity_check_mode': 'strict',
(TaskRunner pid=2281424)                                                   'tool_config_path': None,
(TaskRunner pid=2281424)                                                   'tool_response_truncate_side': 'middle',
(TaskRunner pid=2281424)                                                   'use_inference_chat_template': False},
(TaskRunner pid=2281424)                                    'n': 8,
(TaskRunner pid=2281424)                                    'name': 'vllm',
(TaskRunner pid=2281424)                                    'over_sample_rate': 0,
(TaskRunner pid=2281424)                                    'pipeline_model_parallel_size': 1,
(TaskRunner pid=2281424)                                    'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2281424)                                                 'all_ranks': False,
(TaskRunner pid=2281424)                                                 'enable': False,
(TaskRunner pid=2281424)                                                 'ranks': [],
(TaskRunner pid=2281424)                                                 'save_path': 'outputs/profile',
(TaskRunner pid=2281424)                                                 'tool': None,
(TaskRunner pid=2281424)                                                 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2281424)                                                                         'analysis': True,
(TaskRunner pid=2281424)                                                                         'contents': [],
(TaskRunner pid=2281424)                                                                         'discrete': False,
(TaskRunner pid=2281424)                                                                         'level': 'level0'},
(TaskRunner pid=2281424)                                                                 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2281424)                                                                          'discrete': False},
(TaskRunner pid=2281424)                                                                 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2281424)                                                                           'step_end': None,
(TaskRunner pid=2281424)                                                                           'step_start': 0},
(TaskRunner pid=2281424)                                                                 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2281424)                                                                                  'stack_depth': 32,
(TaskRunner pid=2281424)                                                                                  'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2281424)                                    'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig',
(TaskRunner pid=2281424)                                                   'enable': False,
(TaskRunner pid=2281424)                                                   'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml',
(TaskRunner pid=2281424)                                                   'port': 9090,
(TaskRunner pid=2281424)                                                   'served_model_name': 'Qwen/Qwen3-8B'},
(TaskRunner pid=2281424)                                    'prompt_length': 2048,
(TaskRunner pid=2281424)                                    'quantization': None,
(TaskRunner pid=2281424)                                    'quantization_config_file': None,
(TaskRunner pid=2281424)                                    'response_length': 8192,
(TaskRunner pid=2281424)                                    'scheduling_policy': 'fcfs',
(TaskRunner pid=2281424)                                    'skip_dump_dir': '/tmp/rollout_dump',
(TaskRunner pid=2281424)                                    'skip_rollout': False,
(TaskRunner pid=2281424)                                    'skip_tokenizer_init': True,
(TaskRunner pid=2281424)                                    'temperature': 1.0,
(TaskRunner pid=2281424)                                    'tensor_model_parallel_size': 2,
(TaskRunner pid=2281424)                                    'top_k': -1,
(TaskRunner pid=2281424)                                    'top_p': 1.0,
(TaskRunner pid=2281424)                                    'trace': {'_target_': 'verl.workers.config.TraceConfig',
(TaskRunner pid=2281424)                                              'backend': None,
(TaskRunner pid=2281424)                                              'max_samples_per_step_per_worker': None,
(TaskRunner pid=2281424)                                              'token2text': False},
(TaskRunner pid=2281424)                                    'update_weights_bucket_megabytes': 512,
(TaskRunner pid=2281424)                                    'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig',
(TaskRunner pid=2281424)                                                   'do_sample': True,
(TaskRunner pid=2281424)                                                   'n': 16,
(TaskRunner pid=2281424)                                                   'temperature': 0.6,
(TaskRunner pid=2281424)                                                   'top_k': -1,
(TaskRunner pid=2281424)                                                   'top_p': 0.95}}},
(TaskRunner pid=2281424)  'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig',
(TaskRunner pid=2281424)                'adv_estimator': 'grpo',
(TaskRunner pid=2281424)                'gamma': 1.0,
(TaskRunner pid=2281424)                'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig',
(TaskRunner pid=2281424)                            'horizon': 10000,
(TaskRunner pid=2281424)                            'kl_coef': 0.001,
(TaskRunner pid=2281424)                            'target_kl': 0.1,
(TaskRunner pid=2281424)                            'type': 'fixed'},
(TaskRunner pid=2281424)                'kl_penalty': 'kl',
(TaskRunner pid=2281424)                'lam': 1.0,
(TaskRunner pid=2281424)                'norm_adv_by_std_in_grpo': False,
(TaskRunner pid=2281424)                'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0},
(TaskRunner pid=2281424)                'rollout_correction': {'bypass_mode': False,
(TaskRunner pid=2281424)                                       'loss_type': 'ppo_clip',
(TaskRunner pid=2281424)                                       'rollout_is': 'token',
(TaskRunner pid=2281424)                                       'rollout_is_batch_normalize': False,
(TaskRunner pid=2281424)                                       'rollout_is_threshold': 2.0,
(TaskRunner pid=2281424)                                       'rollout_rs': None,
(TaskRunner pid=2281424)                                       'rollout_rs_threshold': None},
(TaskRunner pid=2281424)                'use_kl_in_reward': False,
(TaskRunner pid=2281424)                'use_pf_ppo': False},
(TaskRunner pid=2281424)  'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig',
(TaskRunner pid=2281424)             'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig',
(TaskRunner pid=2281424)                            'async_save': False,
(TaskRunner pid=2281424)                            'load_contents': ['model', 'optimizer', 'extra'],
(TaskRunner pid=2281424)                            'save_contents': ['model', 'optimizer', 'extra']},
(TaskRunner pid=2281424)             'cliprange_value': 0.5,
(TaskRunner pid=2281424)             'data_loader_seed': 42,
(TaskRunner pid=2281424)             'enable': None,
(TaskRunner pid=2281424)             'forward_max_token_len_per_gpu': 32768,
(TaskRunner pid=2281424)             'forward_micro_batch_size': None,
(TaskRunner pid=2281424)             'forward_micro_batch_size_per_gpu': None,
(TaskRunner pid=2281424)             'grad_clip': 1.0,
(TaskRunner pid=2281424)             'loss_agg_mode': 'token-mean',
(TaskRunner pid=2281424)             'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg',
(TaskRunner pid=2281424)                       'enable_activation_offload': False,
(TaskRunner pid=2281424)                       'enable_gradient_checkpointing': True,
(TaskRunner pid=2281424)                       'external_lib': None,
(TaskRunner pid=2281424)                       'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2281424)                                       'dtype': 'bfloat16',
(TaskRunner pid=2281424)                                       'entropy_checkpointing': False,
(TaskRunner pid=2281424)                                       'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2281424)                                       'forward_only': False,
(TaskRunner pid=2281424)                                       'forward_prefetch': False,
(TaskRunner pid=2281424)                                       'fsdp_size': -1,
(TaskRunner pid=2281424)                                       'full_determinism': False,
(TaskRunner pid=2281424)                                       'model_dtype': 'fp32',
(TaskRunner pid=2281424)                                       'offload_policy': False,
(TaskRunner pid=2281424)                                       'optimizer_offload': False,
(TaskRunner pid=2281424)                                       'param_offload': False,
(TaskRunner pid=2281424)                                       'reshard_after_forward': True,
(TaskRunner pid=2281424)                                       'seed': 42,
(TaskRunner pid=2281424)                                       'strategy': 'fsdp',
(TaskRunner pid=2281424)                                       'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2281424)                                       'use_orig_params': False,
(TaskRunner pid=2281424)                                       'use_torch_compile': True,
(TaskRunner pid=2281424)                                       'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2281424)                       'lora_alpha': 16,
(TaskRunner pid=2281424)                       'lora_rank': 0,
(TaskRunner pid=2281424)                       'override_config': {},
(TaskRunner pid=2281424)                       'path': 'Qwen/Qwen3-8B',
(TaskRunner pid=2281424)                       'target_modules': 'all-linear',
(TaskRunner pid=2281424)                       'tiled_mlp': {'enabled': False, 'num_shards': 4},
(TaskRunner pid=2281424)                       'tokenizer_path': 'Qwen/Qwen3-8B',
(TaskRunner pid=2281424)                       'trust_remote_code': True,
(TaskRunner pid=2281424)                       'use_remove_padding': False,
(TaskRunner pid=2281424)                       'use_shm': False},
(TaskRunner pid=2281424)             'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig',
(TaskRunner pid=2281424)                       'betas': [0.9, 0.999],
(TaskRunner pid=2281424)                       'clip_grad': 1.0,
(TaskRunner pid=2281424)                       'lr': 1e-05,
(TaskRunner pid=2281424)                       'lr_scheduler_type': 'constant',
(TaskRunner pid=2281424)                       'lr_warmup_steps': -1,
(TaskRunner pid=2281424)                       'lr_warmup_steps_ratio': 0.0,
(TaskRunner pid=2281424)                       'min_lr_ratio': 0.0,
(TaskRunner pid=2281424)                       'num_cycles': 0.5,
(TaskRunner pid=2281424)                       'optimizer': 'AdamW',
(TaskRunner pid=2281424)                       'optimizer_impl': 'torch.optim',
(TaskRunner pid=2281424)                       'override_optimizer_config': None,
(TaskRunner pid=2281424)                       'total_training_steps': -1,
(TaskRunner pid=2281424)                       'warmup_style': None,
(TaskRunner pid=2281424)                       'weight_decay': 0.01},
(TaskRunner pid=2281424)             'ppo_epochs': 1,
(TaskRunner pid=2281424)             'ppo_max_token_len_per_gpu': 32768,
(TaskRunner pid=2281424)             'ppo_micro_batch_size': None,
(TaskRunner pid=2281424)             'ppo_micro_batch_size_per_gpu': None,
(TaskRunner pid=2281424)             'ppo_mini_batch_size': 8,
(TaskRunner pid=2281424)             'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2281424)                          'all_ranks': False,
(TaskRunner pid=2281424)                          'enable': False,
(TaskRunner pid=2281424)                          'ranks': [],
(TaskRunner pid=2281424)                          'save_path': 'outputs/profile',
(TaskRunner pid=2281424)                          'tool': None,
(TaskRunner pid=2281424)                          'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2281424)                                                  'analysis': True,
(TaskRunner pid=2281424)                                                  'contents': [],
(TaskRunner pid=2281424)                                                  'discrete': False,
(TaskRunner pid=2281424)                                                  'level': 'level0'},
(TaskRunner pid=2281424)                                          'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2281424)                                                   'discrete': False},
(TaskRunner pid=2281424)                                          'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2281424)                                                    'step_end': None,
(TaskRunner pid=2281424)                                                    'step_start': 0},
(TaskRunner pid=2281424)                                          'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2281424)                                                           'stack_depth': 32,
(TaskRunner pid=2281424)                                                           'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2281424)             'rollout_n': 8,
(TaskRunner pid=2281424)             'shuffle': False,
(TaskRunner pid=2281424)             'strategy': 'fsdp',
(TaskRunner pid=2281424)             'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2281424)             'use_dynamic_bsz': False},
(TaskRunner pid=2281424)  'custom_reward_function': {'name': 'compute_score',
(TaskRunner pid=2281424)                             'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'},
(TaskRunner pid=2281424)  'data': {'apply_chat_template_kwargs': {'enable_thinking': False},
(TaskRunner pid=2281424)           'custom_cls': {'name': None, 'path': None},
(TaskRunner pid=2281424)           'datagen': {'name': None, 'path': None},
(TaskRunner pid=2281424)           'dataloader_num_workers': 8,
(TaskRunner pid=2281424)           'filter_overlong_prompts': True,
(TaskRunner pid=2281424)           'filter_overlong_prompts_workers': 1,
(TaskRunner pid=2281424)           'image_key': 'images',
(TaskRunner pid=2281424)           'image_patch_size': 14,
(TaskRunner pid=2281424)           'max_prompt_length': 2048,
(TaskRunner pid=2281424)           'max_response_length': 8192,
(TaskRunner pid=2281424)           'prompt_key': 'prompt',
(TaskRunner pid=2281424)           'return_full_prompt': False,
(TaskRunner pid=2281424)           'return_multi_modal_inputs': True,
(TaskRunner pid=2281424)           'return_raw_chat': True,
(TaskRunner pid=2281424)           'return_raw_input_ids': False,
(TaskRunner pid=2281424)           'reward_fn_key': 'data_source',
(TaskRunner pid=2281424)           'sampler': {'class_name': None, 'class_path': None},
(TaskRunner pid=2281424)           'seed': None,
(TaskRunner pid=2281424)           'shuffle': True,
(TaskRunner pid=2281424)           'tokenizer': None,
(TaskRunner pid=2281424)           'tool_config_path': None,
(TaskRunner pid=2281424)           'train_batch_size': 32,
(TaskRunner pid=2281424)           'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet'],
(TaskRunner pid=2281424)           'train_max_samples': 3200,
(TaskRunner pid=2281424)           'truncation': 'error',
(TaskRunner pid=2281424)           'trust_remote_code': True,
(TaskRunner pid=2281424)           'use_shm': False,
(TaskRunner pid=2281424)           'val_batch_size': None,
(TaskRunner pid=2281424)           'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet'],
(TaskRunner pid=2281424)           'val_max_samples': -1,
(TaskRunner pid=2281424)           'validation_shuffle': False,
(TaskRunner pid=2281424)           'video_key': 'videos'},
(TaskRunner pid=2281424)  'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2281424)                      'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2281424)                                                      'controller_nsight_options': {'cuda-graph-trace': 'graph',
(TaskRunner pid=2281424)                                                                                    'cuda-memory-usage': 'true',
(TaskRunner pid=2281424)                                                                                    'trace': 'cuda,nvtx,cublas,ucx'},
(TaskRunner pid=2281424)                                                      'discrete': False,
(TaskRunner pid=2281424)                                                      'worker_nsight_options': {'capture-range': 'cudaProfilerApi',
(TaskRunner pid=2281424)                                                                                'capture-range-end': None,
(TaskRunner pid=2281424)                                                                                'cuda-graph-trace': 'graph',
(TaskRunner pid=2281424)                                                                                'cuda-memory-usage': 'true',
(TaskRunner pid=2281424)                                                                                'kill': 'none',
(TaskRunner pid=2281424)                                                                                'trace': 'cuda,nvtx,cublas,ucx'}},
(TaskRunner pid=2281424)                                             'torch_memory': {'context': 'all',
(TaskRunner pid=2281424)                                                              'kw_args': {},
(TaskRunner pid=2281424)                                                              'stack_depth': 32,
(TaskRunner pid=2281424)                                                              'stacks': 'all',
(TaskRunner pid=2281424)                                                              'trace_alloc_max_entries': 100000}},
(TaskRunner pid=2281424)                      'profile_continuous_steps': False,
(TaskRunner pid=2281424)                      'save_path': 'outputs/profile',
(TaskRunner pid=2281424)                      'steps': None,
(TaskRunner pid=2281424)                      'tool': None},
(TaskRunner pid=2281424)  'max_model_len': 10240,
(TaskRunner pid=2281424)  'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_physics_grpo_Qwen3_8B',
(TaskRunner pid=2281424)                              'include_dashboard': False,
(TaskRunner pid=2281424)                              'num_cpus': None,
(TaskRunner pid=2281424)                              'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8',
(TaskRunner pid=2281424)                                                           'TASK': 'datasets/sciknoweval/physics',
(TaskRunner pid=2281424)                                                           'USER': 'root'}}},
(TaskRunner pid=2281424)                 'timeline_json_file': None},
(TaskRunner pid=2281424)  'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig',
(TaskRunner pid=2281424)                     'module': {'_target_': 'verl.trainer.config.config.ModuleConfig',
(TaskRunner pid=2281424)                                'name': 'custom_reward_manager',
(TaskRunner pid=2281424)                                'path': None},
(TaskRunner pid=2281424)                     'name': 'naive',
(TaskRunner pid=2281424)                     'source': 'register'},
(TaskRunner pid=2281424)  'reward_model': {'enable': False,
(TaskRunner pid=2281424)                   'enable_resource_pool': False,
(TaskRunner pid=2281424)                   'forward_max_token_len_per_gpu': 32768,
(TaskRunner pid=2281424)                   'launch_reward_fn_async': False,
(TaskRunner pid=2281424)                   'max_length': None,
(TaskRunner pid=2281424)                   'micro_batch_size': None,
(TaskRunner pid=2281424)                   'micro_batch_size_per_gpu': None,
(TaskRunner pid=2281424)                   'model': {'external_lib': None,
(TaskRunner pid=2281424)                             'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2281424)                                             'forward_prefetch': False,
(TaskRunner pid=2281424)                                             'fsdp_size': -1,
(TaskRunner pid=2281424)                                             'param_offload': False,
(TaskRunner pid=2281424)                                             'reshard_after_forward': True,
(TaskRunner pid=2281424)                                             'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2281424)                             'input_tokenizer': 'Qwen/Qwen3-8B',
(TaskRunner pid=2281424)                             'override_config': {},
(TaskRunner pid=2281424)                             'path': '~/models/FsfairX-LLaMA3-RM-v0.1',
(TaskRunner pid=2281424)                             'trust_remote_code': False,
(TaskRunner pid=2281424)                             'use_fused_kernels': False,
(TaskRunner pid=2281424)                             'use_remove_padding': False,
(TaskRunner pid=2281424)                             'use_shm': False},
(TaskRunner pid=2281424)                   'n_gpus_per_node': 8,
(TaskRunner pid=2281424)                   'nnodes': 0,
(TaskRunner pid=2281424)                   'num_workers': 1,
(TaskRunner pid=2281424)                   'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2281424)                                'all_ranks': False,
(TaskRunner pid=2281424)                                'enable': False,
(TaskRunner pid=2281424)                                'ranks': [],
(TaskRunner pid=2281424)                                'save_path': 'outputs/profile',
(TaskRunner pid=2281424)                                'tool': None,
(TaskRunner pid=2281424)                                'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2281424)                                                        'analysis': True,
(TaskRunner pid=2281424)                                                        'contents': [],
(TaskRunner pid=2281424)                                                        'discrete': False,
(TaskRunner pid=2281424)                                                        'level': 'level0'},
(TaskRunner pid=2281424)                                                'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2281424)                                                         'discrete': False},
(TaskRunner pid=2281424)                                                'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2281424)                                                          'step_end': None,
(TaskRunner pid=2281424)                                                          'step_start': 0},
(TaskRunner pid=2281424)                                                'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2281424)                                                                 'stack_depth': 32,
(TaskRunner pid=2281424)                                                                 'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2281424)                   'reward_loop_class_name': None,
(TaskRunner pid=2281424)                   'reward_loop_module_path': None,
(TaskRunner pid=2281424)                   'reward_loop_source': 'register',
(TaskRunner pid=2281424)                   'reward_manager': 'naive',
(TaskRunner pid=2281424)                   'rollout': {'_target_': 'verl.workers.config.RolloutConfig',
(TaskRunner pid=2281424)                               'cudagraph_capture_sizes': None,
(TaskRunner pid=2281424)                               'data_parallel_size': 1,
(TaskRunner pid=2281424)                               'disable_log_stats': True,
(TaskRunner pid=2281424)                               'dtype': 'bfloat16',
(TaskRunner pid=2281424)                               'enable_chunked_prefill': True,
(TaskRunner pid=2281424)                               'enable_prefix_caching': True,
(TaskRunner pid=2281424)                               'enforce_eager': True,
(TaskRunner pid=2281424)                               'engine_kwargs': {},
(TaskRunner pid=2281424)                               'expert_parallel_size': 1,
(TaskRunner pid=2281424)                               'free_cache_engine': True,
(TaskRunner pid=2281424)                               'gpu_memory_utilization': 0.5,
(TaskRunner pid=2281424)                               'limit_images': None,
(TaskRunner pid=2281424)                               'load_format': 'auto',
(TaskRunner pid=2281424)                               'max_model_len': None,
(TaskRunner pid=2281424)                               'max_num_batched_tokens': 8192,
(TaskRunner pid=2281424)                               'max_num_seqs': 1024,
(TaskRunner pid=2281424)                               'name': '???',
(TaskRunner pid=2281424)                               'prompt_length': 2048,
(TaskRunner pid=2281424)                               'response_length': 2048,
(TaskRunner pid=2281424)                               'skip_tokenizer_init': False,
(TaskRunner pid=2281424)                               'tensor_model_parallel_size': 2},
(TaskRunner pid=2281424)                   'sandbox_fusion': {'max_concurrent': 64,
(TaskRunner pid=2281424)                                      'memory_limit_mb': 1024,
(TaskRunner pid=2281424)                                      'url': None},
(TaskRunner pid=2281424)                   'strategy': 'fsdp',
(TaskRunner pid=2281424)                   'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2281424)                   'use_dynamic_bsz': False,
(TaskRunner pid=2281424)                   'use_reward_loop': False},
(TaskRunner pid=2281424)  'trainer': {'balance_batch': True,
(TaskRunner pid=2281424)              'critic_warmup': 0,
(TaskRunner pid=2281424)              'default_hdfs_dir': None,
(TaskRunner pid=2281424)              'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8',
(TaskRunner pid=2281424)              'del_local_ckpt_after_load': False,
(TaskRunner pid=2281424)              'device': 'cuda',
(TaskRunner pid=2281424)              'esi_redundant_time': 0,
(TaskRunner pid=2281424)              'experiment_name': 'qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8',
(TaskRunner pid=2281424)              'group_name': 'QWEN3-GRPO-generalization',
(TaskRunner pid=2281424)              'log_val_generations': 0,
(TaskRunner pid=2281424)              'logger': ['console', 'wandb'],
(TaskRunner pid=2281424)              'max_actor_ckpt_to_keep': 100,
(TaskRunner pid=2281424)              'max_critic_ckpt_to_keep': None,
(TaskRunner pid=2281424)              'n_gpus_per_node': 8,
(TaskRunner pid=2281424)              'nnodes': 1,
(TaskRunner pid=2281424)              'project_name': 'SDPO-root',
(TaskRunner pid=2281424)              'ray_wait_register_center_timeout': 300,
(TaskRunner pid=2281424)              'resume_from_path': None,
(TaskRunner pid=2281424)              'resume_mode': 'auto',
(TaskRunner pid=2281424)              'rollout_data_dir': None,
(TaskRunner pid=2281424)              'save_freq': 10,
(TaskRunner pid=2281424)              'test_freq': 10,
(TaskRunner pid=2281424)              'total_epochs': 30,
(TaskRunner pid=2281424)              'total_training_steps': 100,
(TaskRunner pid=2281424)              'use_legacy_worker_impl': 'auto',
(TaskRunner pid=2281424)              'val_before_train': False,
(TaskRunner pid=2281424)              'val_only': False,
(TaskRunner pid=2281424)              'validation_data_dir': None},
(TaskRunner pid=2281424)  'transfer_queue': {'enable': False},
(TaskRunner pid=2281424)  'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/physics',
(TaskRunner pid=2281424)           'dir': '/users/root/SDPO',
(TaskRunner pid=2281424)           'log_dir': '/users/root/output',
(TaskRunner pid=2281424)           'task': 'datasets/sciknoweval/physics'}}
(TaskRunner pid=2281424) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True
(TaskRunner pid=2281424)   use_critic=need_critic(config),
(TaskRunner pid=2281424) [validate_config] All configuration checks passed successfully!
(TaskRunner pid=2281424) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now.
(TaskRunner pid=2281424)   warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1)
(TaskRunner pid=2281424) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(TaskRunner pid=2281424)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(TaskRunner pid=2281424) Using dataset class: RLHFDataset
(TaskRunner pid=2281424) dataset len: 720
(TaskRunner pid=2281424) Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2281424) WARNING:2026-07-03 03:19:03,686:Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2281424) 
Filtering prompts longer than 2048 tokens (num_proc=1):   0%|          | 0/720 [00:00<?, ? examples/s]
(TaskRunner pid=2281424) 
Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 720/720 [00:00<00:00, 854.26 examples/s]
(TaskRunner pid=2281424) filter dataset len: 720
(TaskRunner pid=2281424) Using dataset class: RLHFDataset
(TaskRunner pid=2281424) 
Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 720/720 [00:00<00:00, 764.92 examples/s]
(TaskRunner pid=2281424) dataset len: 80
(TaskRunner pid=2281424) Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2281424) WARNING:2026-07-03 03:19:05,039:Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2281424) 
Filtering prompts longer than 2048 tokens (num_proc=1):   0%|          | 0/80 [00:00<?, ? examples/s]
(TaskRunner pid=2281424) 
Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 80/80 [00:00<00:00, 145.81 examples/s]
(TaskRunner pid=2281424) filter dataset len: 80
(TaskRunner pid=2281424) Size of train dataloader: 22, Size of val dataloader: 1
(TaskRunner pid=2281424) Total training steps: 100
(TaskRunner pid=2281424) colocated worker base class <class 'verl.single_controller.base.worker.Worker'>
(TaskRunner pid=2281424) bind role actor_rollout method chat_completion to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2281424) bind role actor_rollout method generate to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2281424) bind role actor_rollout method get_zeromq_address to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2281424) bind role actor_rollout method sleep to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2281424) bind role actor_rollout method wake_up to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2281424) 
Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 80/80 [00:00<00:00, 122.84 examples/s]
(TaskRunner pid=2281424) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True
(TaskRunner pid=2281424)   self.use_critic = need_critic(self.config)
(WorkerDict pid=2281814) [W703 03:19:12.382603555 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:53237 (errno: 97 - Address family not supported by protocol).
(WorkerDict pid=2281810) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7
(WorkerDict pid=2281810) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(WorkerDict pid=2281810)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(WorkerDict pid=2281817) [W703 03:19:13.439209755 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:53237 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.)
(WorkerDict pid=2281810) Model config after override: Qwen3Config {
(WorkerDict pid=2281810)   "architectures": [
(WorkerDict pid=2281810)     "Qwen3ForCausalLM"
(WorkerDict pid=2281810)   ],
(WorkerDict pid=2281810)   "attention_bias": false,
(WorkerDict pid=2281810)   "attention_dropout": 0.0,
(WorkerDict pid=2281810)   "dtype": "bfloat16",
(WorkerDict pid=2281810)   "eos_token_id": 151645,
(WorkerDict pid=2281810)   "head_dim": 128,
(WorkerDict pid=2281810)   "hidden_act": "silu",
(WorkerDict pid=2281810)   "hidden_size": 4096,
(WorkerDict pid=2281810)   "initializer_range": 0.02,
(WorkerDict pid=2281810)   "intermediate_size": 12288,
(WorkerDict pid=2281810)   "layer_types": [
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention",
(WorkerDict pid=2281810)     "full_attention"
(WorkerDict pid=2281810)   ],
(WorkerDict pid=2281810)   "max_position_embeddings": 40960,
(WorkerDict pid=2281810)   "max_window_layers": 36,
(WorkerDict pid=2281810)   "model_type": "qwen3",
(WorkerDict pid=2281810)   "num_attention_heads": 32,
(WorkerDict pid=2281810)   "num_hidden_layers": 36,
(WorkerDict pid=2281810)   "num_key_value_heads": 8,
(WorkerDict pid=2281810)   "pad_token_id": 151643,
(WorkerDict pid=2281810)   "rms_norm_eps": 1e-06,
(WorkerDict pid=2281810)   "rope_scaling": null,
(WorkerDict pid=2281810)   "rope_theta": 1000000,
(WorkerDict pid=2281810)   "sliding_window": null,
(WorkerDict pid=2281810)   "tie_word_embeddings": false,
(WorkerDict pid=2281810)   "transformers_version": "4.57.1",
(WorkerDict pid=2281810)   "use_cache": true,
(WorkerDict pid=2281810)   "use_sliding_window": false,
(WorkerDict pid=2281810)   "vocab_size": 151936
(WorkerDict pid=2281810) }
(WorkerDict pid=2281810) 
(WorkerDict pid=2281810) `torch_dtype` is deprecated! Use `dtype` instead!
(WorkerDict pid=2281810) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)`
(WorkerDict pid=2281812) 
Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s]
(WorkerDict pid=2281810) 
Loading checkpoint shards:  20%|██        | 1/5 [00:02<00:11,  2.98s/it]
(WorkerDict pid=2281817) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster]
(WorkerDict pid=2281817)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster]
(WorkerDict pid=2281817) `torch_dtype` is deprecated! Use `dtype` instead! [repeated 7x across cluster]
(WorkerDict pid=2281817) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` [repeated 15x across cluster]
(WorkerDict pid=2281817) 
Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s] [repeated 7x across cluster]
(WorkerDict pid=2281810) 
Loading checkpoint shards:  60%|██████    | 3/5 [00:08<00:05,  2.96s/it] [repeated 16x across cluster]
(WorkerDict pid=2281810) 
Loading checkpoint shards: 100%|██████████| 5/5 [00:12<00:00,  2.07s/it]
Loading checkpoint shards: 100%|██████████| 5/5 [00:12<00:00,  2.43s/it]
(WorkerDict pid=2281810) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention
(WorkerDict pid=2281810) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch
(WorkerDict pid=2281817) [Gloo] Rank 7 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 [repeated 7x across cluster]
(WorkerDict pid=2281810) Qwen3ForCausalLM contains 8.19B parameters
(WorkerDict pid=2281810) wrap_policy: functools.partial(<function _or_policy at 0x7b9735e84680>, policies=[functools.partial(<function transformer_auto_wrap_policy at 0x7b9735e84540>, transformer_layer_cls={<class 'transformers.models.qwen3.modeling_qwen3.Qwen3DecoderLayer'>})])
(WorkerDict pid=2281810) NCCL version 2.27.5+cuda12.9
(WorkerDict pid=2281811) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now.
(WorkerDict pid=2281811)   warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1)
(WorkerDict pid=2281817) 
Loading checkpoint shards:  80%|████████  | 4/5 [00:11<00:02,  2.89s/it] [repeated 15x across cluster]
(WorkerDict pid=2281817) 
Loading checkpoint shards: 100%|██████████| 5/5 [00:12<00:00,  2.20s/it]
Loading checkpoint shards: 100%|██████████| 5/5 [00:12<00:00,  2.58s/it] [repeated 7x across cluster]
(WorkerDict pid=2281810) Total steps: 100, num_warmup_steps: 10
(WorkerDict pid=2281810) Actor use_remove_padding=True
(WorkerDict pid=2281810) Actor use_fused_kernels=False
(WorkerDict pid=2281810) Actor use_prefix_grouper=False
(WorkerDict pid=2281817) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster]
(WorkerDict pid=2281817) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster]
(WorkerDict pid=2281814) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(WorkerDict pid=2281814)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(WorkerDict pid=2281811) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3
(WorkerDict pid=2281810) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster]
(WorkerDict pid=2281810)   warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster]
(WorkerDict pid=2281816) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html .
(WorkerDict pid=2281816)   warnings.warn(
(WorkerDict pid=2281810) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster]
(WorkerDict pid=2281810)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster]
(TaskRunner pid=2281424) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now.
(TaskRunner pid=2281424)   warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1)
(WorkerDict pid=2281817) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster]
(WorkerDict pid=2281817)   warnings.warn( [repeated 7x across cluster]
(vLLMHttpServer pid=2282697) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(vLLMHttpServer pid=2282697)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(vLLMHttpServer pid=2282700) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(vLLMHttpServer pid=2282700)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(vLLMHttpServer pid=2282697) ['serve',
(vLLMHttpServer pid=2282697)  'Qwen/Qwen3-8B',
(vLLMHttpServer pid=2282697)  '--dtype',
(vLLMHttpServer pid=2282697)  'bfloat16',
(vLLMHttpServer pid=2282697)  '--load_format',
(vLLMHttpServer pid=2282697)  'dummy',
(vLLMHttpServer pid=2282697)  '--trust_remote_code',
(vLLMHttpServer pid=2282697)  '--max_model_len',
(vLLMHttpServer pid=2282697)  '40960',
(vLLMHttpServer pid=2282697)  '--max_num_seqs',
(vLLMHttpServer pid=2282697)  '1024',
(vLLMHttpServer pid=2282697)  '--enable_chunked_prefill',
(vLLMHttpServer pid=2282697)  '--max_num_batched_tokens',
(vLLMHttpServer pid=2282697)  '10240',
(vLLMHttpServer pid=2282697)  '--enable_prefix_caching',
(vLLMHttpServer pid=2282697)  '--enable_sleep_mode',
(vLLMHttpServer pid=2282697)  '--logprobs_mode',
(vLLMHttpServer pid=2282697)  'processed_logprobs',
(vLLMHttpServer pid=2282697)  '--disable_custom_all_reduce',
(vLLMHttpServer pid=2282697)  '--gpu_memory_utilization',
(vLLMHttpServer pid=2282697)  '0.8',
(vLLMHttpServer pid=2282697)  '--disable_log_stats',
(vLLMHttpServer pid=2282697)  '--tensor_parallel_size',
(vLLMHttpServer pid=2282697)  '2',
(vLLMHttpServer pid=2282697)  '--seed',
(vLLMHttpServer pid=2282697)  '0',
(vLLMHttpServer pid=2282697)  '--override_generation_config',
(vLLMHttpServer pid=2282697)  '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, '
(vLLMHttpServer pid=2282697)  '"max_new_tokens": 8192}',
(vLLMHttpServer pid=2282697)  '--hf_overrides',
(vLLMHttpServer pid=2282697)  '{}',
(vLLMHttpServer pid=2282697)  '--scheduling_policy',
(vLLMHttpServer pid=2282697)  'fcfs']
(WorkerDict pid=2281817) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster]
(vLLMHttpServer pid=2282697) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead.
(vLLMHttpServer pid=2282697) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(vLLMHttpServer pid=2282697) WARNING 07-03 03:20:04 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned
(WorkerDict pid=2281811) WARNING 07-03 03:20:12 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'.
(vLLMHttpServer pid=2282699) WARNING 07-03 03:20:05 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster]
(WorkerDict pid=2281812) NCCL version 2.27.5+cuda12.9
(WorkerDict pid=2281810) 2026-07-03 03:20:26,670 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ...
(vLLMHttpServer pid=2282699) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 2x across cluster]
(vLLMHttpServer pid=2282699)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 2x across cluster]
(vLLMHttpServer pid=2282699) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster]
(vLLMHttpServer pid=2282699) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster]
(WorkerDict pid=2281814) 2026-07-03 03:20:26,669 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   0%|          | 0/51 [00:00<?, ?it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   4%|▍         | 2/51 [00:00<00:02, 17.29it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   8%|▊         | 4/51 [00:00<00:02, 18.21it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  12%|█▏        | 6/51 [00:00<00:02, 18.03it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  16%|█▌        | 8/51 [00:00<00:02, 17.87it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  20%|█▉        | 10/51 [00:00<00:02, 17.90it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  24%|██▎       | 12/51 [00:00<00:02, 17.93it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  27%|██▋       | 14/51 [00:00<00:02, 17.97it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  31%|███▏      | 16/51 [00:00<00:02, 17.24it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  35%|███▌      | 18/51 [00:01<00:01, 17.10it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  39%|███▉      | 20/51 [00:01<00:01, 16.58it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  43%|████▎     | 22/51 [00:01<00:01, 16.64it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  47%|████▋     | 24/51 [00:01<00:01, 16.63it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  51%|█████     | 26/51 [00:01<00:01, 16.46it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  55%|█████▍    | 28/51 [00:01<00:01, 15.96it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  59%|█████▉    | 30/51 [00:01<00:01, 15.69it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  63%|██████▎   | 32/51 [00:01<00:01, 15.83it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  67%|██████▋   | 34/51 [00:02<00:01, 15.41it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  71%|███████   | 36/51 [00:02<00:01, 14.99it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  75%|███████▍  | 38/51 [00:02<00:00, 14.23it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  78%|███████▊  | 40/51 [00:02<00:00, 14.08it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  82%|████████▏ | 42/51 [00:02<00:00, 14.44it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  86%|████████▋ | 44/51 [00:02<00:00, 12.33it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  90%|█████████ | 46/51 [00:02<00:00, 12.82it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  94%|█████████▍| 48/51 [00:03<00:00, 13.05it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  98%|█████████▊| 50/51 [00:03<00:00, 13.79it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 51/51 [00:03<00:00, 15.34it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (decode, FULL):   0%|          | 0/51 [00:00<?, ?it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (decode, FULL):   6%|▌         | 3/51 [00:00<00:02, 21.91it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (decode, FULL):  12%|█▏        | 6/51 [00:00<00:02, 19.40it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (decode, FULL):  18%|█▊        | 9/51 [00:00<00:02, 20.78it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (decode, FULL):  24%|██▎       | 12/51 [00:00<00:01, 21.78it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (decode, FULL):  29%|██▉       | 15/51 [00:00<00:01, 22.24it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (decode, FULL):  35%|███▌      | 18/51 [00:00<00:01, 22.56it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (decode, FULL):  41%|████      | 21/51 [00:00<00:01, 22.72it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (decode, FULL):  47%|████▋     | 24/51 [00:01<00:01, 22.98it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (decode, FULL):  53%|█████▎    | 27/51 [00:01<00:01, 23.16it/s]
(WorkerDict pid=2281817) 2026-07-03 03:20:26,984 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... [repeated 7x across cluster]
(WorkerDict pid=2281817) 2026-07-03 03:20:26,999 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends [repeated 7x across cluster]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (decode, FULL):  59%|█████▉    | 30/51 [00:01<00:00, 23.30it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (decode, FULL):  65%|██████▍   | 33/51 [00:01<00:00, 22.65it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (decode, FULL):  71%|███████   | 36/51 [00:01<00:00, 22.72it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (decode, FULL):  76%|███████▋  | 39/51 [00:01<00:00, 22.62it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (decode, FULL):  82%|████████▏ | 42/51 [00:01<00:00, 22.65it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (decode, FULL):  88%|████████▊ | 45/51 [00:02<00:00, 22.56it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (decode, FULL):  94%|█████████▍| 48/51 [00:02<00:00, 22.59it/s]
(WorkerDict pid=2281810) 
Capturing CUDA graphs (decode, FULL): 100%|██████████| 51/51 [00:02<00:00, 22.48it/s]
Capturing CUDA graphs (decode, FULL): 100%|██████████| 51/51 [00:02<00:00, 22.42it/s]
(vLLMHttpServer pid=2282699) WARNING 07-03 03:20:34 [api_server.py:1358] LoRA dynamic loading & unloading is enabled in the API server. This should ONLY be used for local development!
(WorkerDict pid=2281815) WARNING 07-03 03:20:13 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. [repeated 7x across cluster]
(WorkerDict pid=2281816) NCCL version 2.27.5+cuda12.9 [repeated 2x across cluster]
(vLLMHttpServer pid=2282699) WARNING 07-03 03:20:35 [model.py:1576] Default sampling parameters have been overridden by the model's Hugging Face generation config recommended from the model creator. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`.
(TaskRunner pid=2281424) AgentLoopManager: ['198.19.44.212:40921', '198.19.44.212:40541', '198.19.44.212:37869', '198.19.44.212:42381']
(TaskRunner pid=2281424) wandb: Currently logged in as: seongryongjung (seongryongjung-chung-ang-university) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
(TaskRunner pid=2281424) wandb: Tracking run with wandb version 0.23.1
(TaskRunner pid=2281424) wandb: Run data is saved locally in /mnt/mole/SDPO/L2T/wandb/run-20260703_032040-6ig3l55l
(TaskRunner pid=2281424) wandb: Run `wandb offline` to turn off syncing.
(TaskRunner pid=2281424) wandb: Syncing run qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8
(TaskRunner pid=2281424) wandb: ⭐️ View project at https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root
(TaskRunner pid=2281424) wandb: 🚀 View run at https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/6ig3l55l
(TaskRunner pid=2281424) Checkpoint tracker file does not exist: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/latest_checkpointed_iteration.txt
(TaskRunner pid=2281424) Training from scratch
(vLLMHttpServer pid=2282700) WARNING 07-03 03:20:35 [api_server.py:1358] LoRA dynamic loading & unloading is enabled in the API server. This should ONLY be used for local development! [repeated 3x across cluster]
(vLLMHttpServer pid=2282700) WARNING 07-03 03:20:35 [model.py:1576] Default sampling parameters have been overridden by the model's Hugging Face generation config recommended from the model creator. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`. [repeated 3x across cluster]
(TaskRunner pid=2281424) wandb: Detected [openai] in use.
(TaskRunner pid=2281424) wandb: Use W&B Weave for improved LLM call tracing. Install Weave with `pip install weave` then add `import weave` to the top of your script.
(TaskRunner pid=2281424) wandb: For more information, check out the docs at: https://weave-docs.wandb.ai/
(TaskRunner pid=2281424) 
Training Progress:   0%|          | 0/100 [00:00<?, ?it/s]
(AgentLoopWorker pid=2283645) Using dataset class: RLHFDataset
(AgentLoopWorker pid=2283645) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(AgentLoopWorker pid=2283645)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(AgentLoopWorker pid=2283645) You're using a Qwen2TokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.
(AgentLoopWorker pid=2283648) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster]
(AgentLoopWorker pid=2283648)   warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster]
(AgentLoopWorker pid=2283647) You're using a Qwen2TokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding. [repeated 7x across cluster]
(TaskRunner pid=2281424) step:1 - global_seqlen/min:21599 - global_seqlen/max:39795 - global_seqlen/minmax_diff:18196 - global_seqlen/balanced_min:27997 - global_seqlen/balanced_max:28074 - global_seqlen/mean:28049.0 - actor/entropy:0.2933548390865326 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3997599184513092 - training/rollout_probs_diff_mean:0.0031216552015393972 - training/rollout_probs_diff_std:0.008815591223537922 - training/rollout_actor_probs_pearson_corr:0.9993525743484497 - rollout_corr/rollout_is_mean:0.9999884963035583 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.3050060260866303e-05 - rollout_corr/rollout_is_max:1.6403627395629883 - rollout_corr/rollout_is_min:0.4432307779788971 - rollout_corr/rollout_is_std:0.02760624885559082 - rollout_corr/rollout_is_eff_sample_size:0.9992384753736182 - rollout_corr/rollout_is_seq_mean:0.9998464584350586 - rollout_corr/rollout_is_seq_std:0.001634377520531416 - rollout_corr/rollout_is_seq_max:1.0051524639129639 - rollout_corr/rollout_is_seq_min:0.9904565215110779 - rollout_corr/rollout_is_seq_max_deviation:0.00954347848892212 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3432956486940384) - rollout_corr/training_log_ppl:np.float64(0.274512951305951) - rollout_corr/kl:np.float64(0.0005876388563024193) - rollout_corr/k3_kl:np.float64(0.0004341341748954619) - rollout_corr/rollout_ppl:np.float64(1.3425641637295485) - rollout_corr/rollout_log_ppl:np.float64(0.2739253109320998) - rollout_corr/log_ppl_diff:np.float64(0.0005876403738511726) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011765340605052188) - rollout_corr/log_ppl_diff_max:np.float64(0.011144682765007019) - rollout_corr/log_ppl_diff_min:np.float64(-0.003832995891571045) - rollout_corr/ppl_ratio:np.float64(1.0005893006455153) - rollout_corr/chi2_token:np.float64(0.0005533059593290091) - rollout_corr/chi2_seq:np.float64(0.49172084289602935) - actor/pg_loss:np.float64(7.330859079957008e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(0.0) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1784479133784771) - perf/mfu/actor:np.float64(0.1053086686554278) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.935546875) - perf/cpu_memory_used_gb:np.float64(100.71479415893555) - actor/lr:np.float64(0.0) - training/global_step:1 - training/epoch:0 - critic/score/mean:0.6015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004050412215292454 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004050412215292454 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:598.65625 - response_length/max:2273.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:598.65625 - response_length_non_aborted/max:2273.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.875 - prompt_length/max:375.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00023187324404716492 - timing_s/agent_loop/generate_sequences/min:np.float64(10.81203643977642) - timing_s/agent_loop/generate_sequences/max:np.float64(31.67797383107245) - timing_s/agent_loop/generate_sequences/mean:np.float64(20.031546598176647) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(31.67797383107245) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/
(AgentLoopWorker pid=2283648) Using dataset class: RLHFDataset [repeated 7x across cluster]
(TaskRunner pid=2281424) 
Training Progress:   1%|          | 1/100 [01:02<1:43:04, 62.47s/it]
(TaskRunner pid=2281424) step:2 - global_seqlen/min:18488 - global_seqlen/max:35485 - global_seqlen/minmax_diff:16997 - global_seqlen/balanced_min:28661 - global_seqlen/balanced_max:28687 - global_seqlen/mean:28676.875 - actor/entropy:0.2852829098701477 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5235563516616821 - training/rollout_probs_diff_mean:0.0032305719796568155 - training/rollout_probs_diff_std:0.009228591807186604 - training/rollout_actor_probs_pearson_corr:0.9993439316749573 - rollout_corr/rollout_is_mean:0.9998864531517029 - rollout_corr/rollout_is_ratio_fraction_high:6.217088412086014e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.217088412086014e-06 - rollout_corr/rollout_is_max:2.544478178024292 - rollout_corr/rollout_is_min:0.39712417125701904 - rollout_corr/rollout_is_std:0.02848818711936474 - rollout_corr/rollout_is_eff_sample_size:0.9991889622635779 - rollout_corr/rollout_is_seq_mean:0.99990314245224 - rollout_corr/rollout_is_seq_std:0.0016033288557082415 - rollout_corr/rollout_is_seq_max:1.0100219249725342 - rollout_corr/rollout_is_seq_min:0.9953435063362122 - rollout_corr/rollout_is_seq_max_deviation:0.01002192497253418 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3009517984464765) - rollout_corr/training_log_ppl:np.float64(0.25215593120083213) - rollout_corr/kl:np.float64(0.0004823018046948846) - rollout_corr/k3_kl:np.float64(0.0004502152170005047) - rollout_corr/rollout_ppl:np.float64(1.3002900667488575) - rollout_corr/rollout_log_ppl:np.float64(0.2516736279503675) - rollout_corr/log_ppl_diff:np.float64(0.00048230325046461076) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011988449696218595) - rollout_corr/log_ppl_diff_max:np.float64(0.005951404571533203) - rollout_corr/log_ppl_diff_min:np.float64(-0.008813902735710144) - rollout_corr/ppl_ratio:np.float64(1.0004837419837713) - rollout_corr/chi2_token:np.float64(0.0008451999165117741) - rollout_corr/chi2_seq:np.float64(0.24925414193421602) - actor/pg_loss:np.float64(-0.0001076720654964447) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001199317946429801) - actor/ppo_kl:np.float64(-5.8633315513034745e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.14331618882715702) - perf/mfu/actor:np.float64(0.11175082545657457) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.935546875) - perf/cpu_memory_used_gb:np.float64(100.56762313842773) - actor/lr:np.float64(1e-07) - training/global_step:2 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002406013198196888 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002406013198196888 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:628.30859375 - response_length/max:2049.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:628.30859375 - response_length_non_aborted/max:2049.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.84375 - prompt_length/max:342.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001306328922510147 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0788820944726467) - timing_s/agent_loop/generate_sequences/max:np.float64(16.030704772099853) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.596538595229504) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.030704772099853) - timi
(TaskRunner pid=2281424) 
Training Progress:   2%|▏         | 2/100 [01:42<1:20:39, 49.39s/it]
(TaskRunner pid=2281424) step:3 - global_seqlen/min:17827 - global_seqlen/max:47471 - global_seqlen/minmax_diff:29644 - global_seqlen/balanced_min:29373 - global_seqlen/balanced_max:29478 - global_seqlen/mean:29437.25 - actor/entropy:0.3266855776309967 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2997521162033081 - training/rollout_probs_diff_mean:0.003248069901019335 - training/rollout_probs_diff_std:0.008916190825402737 - training/rollout_actor_probs_pearson_corr:0.999448299407959 - rollout_corr/rollout_is_mean:0.9999076724052429 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.8091252059093677e-05 - rollout_corr/rollout_is_max:1.636006236076355 - rollout_corr/rollout_is_min:0.3921360969543457 - rollout_corr/rollout_is_std:0.0288705974817276 - rollout_corr/rollout_is_eff_sample_size:0.9991670637992444 - rollout_corr/rollout_is_seq_mean:0.9999990463256836 - rollout_corr/rollout_is_seq_std:0.0017653178656473756 - rollout_corr/rollout_is_seq_max:1.0086678266525269 - rollout_corr/rollout_is_seq_min:0.992326557636261 - rollout_corr/rollout_is_seq_max_deviation:0.008667826652526855 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.305179648566991) - rollout_corr/training_log_ppl:np.float64(0.25337072674301453) - rollout_corr/kl:np.float64(0.00037616747434299214) - rollout_corr/k3_kl:np.float64(0.0004263165984070838) - rollout_corr/rollout_ppl:np.float64(1.30465651396662) - rollout_corr/rollout_log_ppl:np.float64(0.2529945558635518) - rollout_corr/log_ppl_diff:np.float64(0.00037617087946273386) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011907302250619978) - rollout_corr/log_ppl_diff_max:np.float64(0.006459161639213562) - rollout_corr/log_ppl_diff_min:np.float64(-0.006830498576164246) - rollout_corr/ppl_ratio:np.float64(1.0003775698132813) - rollout_corr/chi2_token:np.float64(0.0009592922870069742) - rollout_corr/chi2_seq:np.float64(0.461652651662007) - actor/pg_loss:np.float64(6.696488708257675e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017301455625329254) - actor/ppo_kl:np.float64(-5.806259433693839e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.19614363089203835) - perf/mfu/actor:np.float64(0.11421010958811989) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.935546875) - perf/cpu_memory_used_gb:np.float64(99.99443435668945) - actor/lr:np.float64(2e-07) - training/global_step:3 - training/epoch:0 - critic/score/mean:0.52734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.52734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004827348981052637 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004827348981052637 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:647.7578125 - response_length/max:3010.0 - response_length/min:73.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:647.7578125 - response_length_non_aborted/max:3010.0 - response_length_non_aborted/min:73.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.15625 - prompt_length/max:365.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011022016406059265 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9544193036854267) - timing_s/agent_loop/generate_sequences/max:np.float64(22.102071948349476) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.615853465111286) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.102071948349476) - timing_s/agent_loop/slowest/t
(TaskRunner pid=2281424) 
Training Progress:   3%|▎         | 3/100 [02:29<1:18:08, 48.33s/it]
(TaskRunner pid=2281424) step:4 - global_seqlen/min:19941 - global_seqlen/max:38325 - global_seqlen/minmax_diff:18384 - global_seqlen/balanced_min:28308 - global_seqlen/balanced_max:28336 - global_seqlen/mean:28325.875 - actor/entropy:0.2808186411857605 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2771974205970764 - training/rollout_probs_diff_mean:0.0030747849959880114 - training/rollout_probs_diff_std:0.008857703767716885 - training/rollout_actor_probs_pearson_corr:0.9993070363998413 - rollout_corr/rollout_is_mean:1.0000015497207642 - rollout_corr/rollout_is_ratio_fraction_high:1.2669855095737148e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.334927547868574e-06 - rollout_corr/rollout_is_max:2.03224778175354 - rollout_corr/rollout_is_min:0.4781637489795685 - rollout_corr/rollout_is_std:0.02775697410106659 - rollout_corr/rollout_is_eff_sample_size:0.9992299054457299 - rollout_corr/rollout_is_seq_mean:0.9999985694885254 - rollout_corr/rollout_is_seq_std:0.0015701246447861195 - rollout_corr/rollout_is_seq_max:1.0103716850280762 - rollout_corr/rollout_is_seq_min:0.9926313161849976 - rollout_corr/rollout_is_seq_max_deviation:0.010371685028076172 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2888889200985432) - rollout_corr/training_log_ppl:np.float64(0.24067001044750214) - rollout_corr/kl:np.float64(0.00041824670213985726) - rollout_corr/k3_kl:np.float64(0.0004014854371803267) - rollout_corr/rollout_ppl:np.float64(1.2883517127484083) - rollout_corr/rollout_log_ppl:np.float64(0.24025176231953083) - rollout_corr/log_ppl_diff:np.float64(0.00041824812797131017) - rollout_corr/log_ppl_abs_diff:np.float64(0.001111818324716296) - rollout_corr/log_ppl_diff_max:np.float64(0.010206438601016998) - rollout_corr/log_ppl_diff_min:np.float64(-0.007612407207489014) - rollout_corr/ppl_ratio:np.float64(1.0004195608198643) - rollout_corr/chi2_token:np.float64(0.0007662170100957155) - rollout_corr/chi2_seq:np.float64(0.5071867927908897) - actor/pg_loss:np.float64(-1.1301599442958832e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016651269106660038) - actor/ppo_kl:np.float64(1.1325929784034017e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.19906450808048248) - perf/mfu/actor:np.float64(0.11065199852544017) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.935546875) - perf/cpu_memory_used_gb:np.float64(99.7738823890686) - actor/lr:np.float64(3e-07) - training/global_step:4 - training/epoch:0 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0001662918512010947 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0001662918512010947 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:616.62109375 - response_length/max:2113.0 - response_length/min:89.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:616.62109375 - response_length_non_aborted/max:2113.0 - response_length_non_aborted/min:89.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.5625 - prompt_length/max:361.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014886818826198578 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1396879702806473) - timing_s/agent_loop/generate_sequences/max:np.float64(16.492680540308356) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.509170034310955) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.492680540308356) - t
(TaskRunner pid=2281424) 
Training Progress:   4%|▍         | 4/100 [03:10<1:12:33, 45.35s/it]
(TaskRunner pid=2281424) step:5 - global_seqlen/min:23939 - global_seqlen/max:37277 - global_seqlen/minmax_diff:13338 - global_seqlen/balanced_min:28100 - global_seqlen/balanced_max:29575 - global_seqlen/mean:28450.375 - actor/entropy:0.23747068643569946 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7566088438034058 - training/rollout_probs_diff_mean:0.0028660795651376247 - training/rollout_probs_diff_std:0.009149988181889057 - training/rollout_actor_probs_pearson_corr:0.9992654919624329 - rollout_corr/rollout_is_mean:1.0000168085098267 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.0813411285635084e-05 - rollout_corr/rollout_is_max:1.8653022050857544 - rollout_corr/rollout_is_min:0.1207408457994461 - rollout_corr/rollout_is_std:0.026950737461447716 - rollout_corr/rollout_is_eff_sample_size:0.9992743040251587 - rollout_corr/rollout_is_seq_mean:1.0000309944152832 - rollout_corr/rollout_is_seq_std:0.0014656533021479845 - rollout_corr/rollout_is_seq_max:1.007324457168579 - rollout_corr/rollout_is_seq_min:0.9938392043113708 - rollout_corr/rollout_is_seq_max_deviation:0.0073244571685791016 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2311981176026165) - rollout_corr/training_log_ppl:np.float64(0.20067966838541906) - rollout_corr/kl:np.float64(0.00032477841484368497) - rollout_corr/k3_kl:np.float64(0.0004131826744924183) - rollout_corr/rollout_ppl:np.float64(1.230793579481542) - rollout_corr/rollout_log_ppl:np.float64(0.2003548895881977) - rollout_corr/log_ppl_diff:np.float64(0.00032477879722137004) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011101132404292002) - rollout_corr/log_ppl_diff_max:np.float64(0.005720242857933044) - rollout_corr/log_ppl_diff_min:np.float64(-0.003920510411262512) - rollout_corr/ppl_ratio:np.float64(1.0003259072545916) - rollout_corr/chi2_token:np.float64(0.0010012227576225996) - rollout_corr/chi2_seq:np.float64(0.8521349627990276) - actor/pg_loss:np.float64(8.134916424751282e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019610086025068085) - actor/ppo_kl:np.float64(-6.217524514084971e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17139575630426407) - perf/mfu/actor:np.float64(0.10970825383409716) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(99.72949981689453) - actor/lr:np.float64(4e-07) - training/global_step:5 - training/epoch:0 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0029827384278178215 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0029827384278178215 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:633.85546875 - response_length/max:4229.0 - response_length/min:75.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:633.85546875 - response_length_non_aborted/max:4229.0 - response_length_non_aborted/min:75.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.21875 - prompt_length/max:350.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011628493666648865 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9417712222784758) - timing_s/agent_loop/generate_sequences/max:np.float64(27.8466968331486) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.5835448407015065) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.8466968331486) - timing_s/agent_loo
(TaskRunner pid=2281424) 
Training Progress:   5%|▌         | 5/100 [04:02<1:15:36, 47.76s/it]
(TaskRunner pid=2281424) step:6 - global_seqlen/min:22147 - global_seqlen/max:43094 - global_seqlen/minmax_diff:20947 - global_seqlen/balanced_min:31606 - global_seqlen/balanced_max:32026 - global_seqlen/mean:31664.375 - actor/entropy:0.27507346868515015 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5490837097167969 - training/rollout_probs_diff_mean:0.002936719683930278 - training/rollout_probs_diff_std:0.008653098717331886 - training/rollout_actor_probs_pearson_corr:0.9993214011192322 - rollout_corr/rollout_is_mean:1.0000293254852295 - rollout_corr/rollout_is_ratio_fraction_high:1.1111419553344604e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.2222839106689207e-05 - rollout_corr/rollout_is_max:2.245166063308716 - rollout_corr/rollout_is_min:0.017338119447231293 - rollout_corr/rollout_is_std:0.027201691642403603 - rollout_corr/rollout_is_eff_sample_size:0.9992606150343992 - rollout_corr/rollout_is_seq_mean:1.000042200088501 - rollout_corr/rollout_is_seq_std:0.0012306083226576447 - rollout_corr/rollout_is_seq_max:1.0037089586257935 - rollout_corr/rollout_is_seq_min:0.9957495927810669 - rollout_corr/rollout_is_seq_max_deviation:0.0042504072189331055 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.313299536705017) - rollout_corr/training_log_ppl:np.float64(0.26095605584851) - rollout_corr/kl:np.float64(0.00043005530495854316) - rollout_corr/k3_kl:np.float64(0.00047133550992839446) - rollout_corr/rollout_ppl:np.float64(1.3127399231307209) - rollout_corr/rollout_log_ppl:np.float64(0.2605260002455907) - rollout_corr/log_ppl_diff:np.float64(0.00043005560291931033) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010077479528263211) - rollout_corr/log_ppl_diff_max:np.float64(0.00828494131565094) - rollout_corr/log_ppl_diff_min:np.float64(-0.004989057779312134) - rollout_corr/ppl_ratio:np.float64(1.0004311946686357) - rollout_corr/chi2_token:np.float64(0.000960806617513299) - rollout_corr/chi2_seq:np.float64(0.3135292446240783) - actor/pg_loss:np.float64(8.26478935778141e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020431413167898427) - actor/ppo_kl:np.float64(4.7137278594192034e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15935598872601986) - perf/mfu/actor:np.float64(0.12093475083929373) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(99.71675491333008) - actor/lr:np.float64(5e-07) - training/global_step:6 - training/epoch:0 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008676629513502121 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008676629513502121 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:703.10546875 - response_length/max:2812.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:703.10546875 - response_length_non_aborted/max:2812.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.40625 - prompt_length/max:375.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001495908945798874 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4465590845793486) - timing_s/agent_loop/generate_sequences/max:np.float64(21.05473268404603) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.725667599523149) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.05473268404603) - timing_s/a
(TaskRunner pid=2281424) 
Training Progress:   6%|▌         | 6/100 [04:48<1:13:43, 47.06s/it]
(TaskRunner pid=2281424) step:7 - global_seqlen/min:20814 - global_seqlen/max:34501 - global_seqlen/minmax_diff:13687 - global_seqlen/balanced_min:26181 - global_seqlen/balanced_max:26228 - global_seqlen/mean:26214.75 - actor/entropy:0.2326890081167221 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35570627450942993 - training/rollout_probs_diff_mean:0.0028703256975859404 - training/rollout_probs_diff_std:0.00889651384204626 - training/rollout_actor_probs_pearson_corr:0.9992586970329285 - rollout_corr/rollout_is_mean:1.0000824928283691 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.117336281226017e-06 - rollout_corr/rollout_is_max:1.9689844846725464 - rollout_corr/rollout_is_min:0.4276629388332367 - rollout_corr/rollout_is_std:0.026294749230146408 - rollout_corr/rollout_is_eff_sample_size:0.9993091828880825 - rollout_corr/rollout_is_seq_mean:1.0001779794692993 - rollout_corr/rollout_is_seq_std:0.0015202864306047559 - rollout_corr/rollout_is_seq_max:1.0066863298416138 - rollout_corr/rollout_is_seq_min:0.992588460445404 - rollout_corr/rollout_is_seq_max_deviation:0.007411539554595947 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.243249746505171) - rollout_corr/training_log_ppl:np.float64(0.21141109146992676) - rollout_corr/kl:np.float64(0.000269961312439837) - rollout_corr/k3_kl:np.float64(0.0004097315033675386) - rollout_corr/rollout_ppl:np.float64(1.2429048488847911) - rollout_corr/rollout_log_ppl:np.float64(0.21114112783106975) - rollout_corr/log_ppl_diff:np.float64(0.000269963638857007) - rollout_corr/log_ppl_abs_diff:np.float64(0.001071124483132735) - rollout_corr/log_ppl_diff_max:np.float64(0.005365625023841858) - rollout_corr/log_ppl_diff_min:np.float64(-0.0069854408502578735) - rollout_corr/ppl_ratio:np.float64(1.0002711091656238) - rollout_corr/chi2_token:np.float64(0.0011220404412597418) - rollout_corr/chi2_seq:np.float64(0.38850099057890475) - actor/pg_loss:np.float64(2.9145623557269573e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.624894073567702e-05) - actor/ppo_kl:np.float64(5.5716070178846167e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1705261506140232) - perf/mfu/actor:np.float64(0.10573418722291973) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(99.6202507019043) - actor/lr:np.float64(6e-07) - training/global_step:7 - training/epoch:0 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005430527497082949 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005430527497082949 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:548.8359375 - response_length/max:1959.0 - response_length/min:89.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:548.8359375 - response_length_non_aborted/max:1959.0 - response_length_non_aborted/min:89.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.375 - prompt_length/max:363.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011132284998893738 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1634128130972385) - timing_s/agent_loop/generate_sequences/max:np.float64(14.78265242651105) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.892496629217931) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.78265242651105) - timing_s/agent_loop/slowest/tool_
(TaskRunner pid=2281424) 
Training Progress:   7%|▋         | 7/100 [05:26<1:08:30, 44.19s/it]
(TaskRunner pid=2281424) step:8 - global_seqlen/min:24414 - global_seqlen/max:43215 - global_seqlen/minmax_diff:18801 - global_seqlen/balanced_min:30100 - global_seqlen/balanced_max:30612 - global_seqlen/mean:30198.5 - actor/entropy:0.26404014229774475 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1850457787513733 - training/rollout_probs_diff_mean:0.002853685524314642 - training/rollout_probs_diff_std:0.0084674758836627 - training/rollout_actor_probs_pearson_corr:0.9993511438369751 - rollout_corr/rollout_is_mean:1.0000009536743164 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1623852515185717e-05 - rollout_corr/rollout_is_max:1.9149731397628784 - rollout_corr/rollout_is_min:0.3131442368030548 - rollout_corr/rollout_is_std:0.026572102680802345 - rollout_corr/rollout_is_eff_sample_size:0.9992943025293132 - rollout_corr/rollout_is_seq_mean:0.99997878074646 - rollout_corr/rollout_is_seq_std:0.0011704006465151906 - rollout_corr/rollout_is_seq_max:1.0035500526428223 - rollout_corr/rollout_is_seq_min:0.996343731880188 - rollout_corr/rollout_is_seq_max_deviation:0.0036562681198120117 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3104631286114454) - rollout_corr/training_log_ppl:np.float64(0.2542687382810982) - rollout_corr/kl:np.float64(0.0003127987160311818) - rollout_corr/k3_kl:np.float64(0.0003740117848280988) - rollout_corr/rollout_ppl:np.float64(1.310010947752744) - rollout_corr/rollout_log_ppl:np.float64(0.25395593749999534) - rollout_corr/log_ppl_diff:np.float64(0.00031280078110285103) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009792603377718478) - rollout_corr/log_ppl_diff_max:np.float64(0.004227757453918457) - rollout_corr/log_ppl_diff_min:np.float64(-0.003130972385406494) - rollout_corr/ppl_ratio:np.float64(1.0003135872539133) - rollout_corr/chi2_token:np.float64(0.0008723102509975433) - rollout_corr/chi2_seq:np.float64(1.1971292272210121) - actor/pg_loss:np.float64(-1.5925150364637375e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001960772210622963) - actor/ppo_kl:np.float64(-8.012754377872966e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.200693067163229) - perf/mfu/actor:np.float64(0.11682536401655431) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(99.71234512329102) - actor/lr:np.float64(7e-07) - training/global_step:8 - training/epoch:0 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006695338990539312 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006695338990539312 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:672.109375 - response_length/max:3280.0 - response_length/min:183.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:672.109375 - response_length_non_aborted/max:3280.0 - response_length_non_aborted/min:183.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.59375 - prompt_length/max:375.0 - prompt_length/min:205.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.800944149494171e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1623081266880035) - timing_s/agent_loop/generate_sequences/max:np.float64(23.256134187802672) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.173095271158672) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.256134187802672) - timing_s/agent_loop/slowest/too
(TaskRunner pid=2281424) 
Training Progress:   8%|▊         | 8/100 [06:13<1:09:17, 45.19s/it]
(TaskRunner pid=2281424) step:9 - global_seqlen/min:19363 - global_seqlen/max:38902 - global_seqlen/minmax_diff:19539 - global_seqlen/balanced_min:27273 - global_seqlen/balanced_max:27330 - global_seqlen/mean:27308.125 - actor/entropy:0.278557687997818 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2980630397796631 - training/rollout_probs_diff_mean:0.0030410082545131445 - training/rollout_probs_diff_std:0.008712326176464558 - training/rollout_actor_probs_pearson_corr:0.9993500709533691 - rollout_corr/rollout_is_mean:1.0001354217529297 - rollout_corr/rollout_is_ratio_fraction_high:6.632663371419767e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.3265326742839534e-05 - rollout_corr/rollout_is_max:2.081381320953369 - rollout_corr/rollout_is_min:0.375669002532959 - rollout_corr/rollout_is_std:0.02771829441189766 - rollout_corr/rollout_is_eff_sample_size:0.9992325240192108 - rollout_corr/rollout_is_seq_mean:1.000201940536499 - rollout_corr/rollout_is_seq_std:0.001359231653623283 - rollout_corr/rollout_is_seq_max:1.0056883096694946 - rollout_corr/rollout_is_seq_min:0.9940919280052185 - rollout_corr/rollout_is_seq_max_deviation:0.005908071994781494 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3391040116548538) - rollout_corr/training_log_ppl:np.float64(0.27259969667647965) - rollout_corr/kl:np.float64(0.00025990279606591393) - rollout_corr/k3_kl:np.float64(0.0004612994821684424) - rollout_corr/rollout_ppl:np.float64(1.3387267710641026) - rollout_corr/rollout_log_ppl:np.float64(0.2723397923691664) - rollout_corr/log_ppl_diff:np.float64(0.0002599043073132634) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011203358299098909) - rollout_corr/log_ppl_diff_max:np.float64(0.0065671950578689575) - rollout_corr/log_ppl_diff_min:np.float64(-0.004946023225784302) - rollout_corr/ppl_ratio:np.float64(1.000261099776253) - rollout_corr/chi2_token:np.float64(0.0013498091138899326) - rollout_corr/chi2_seq:np.float64(0.847581728361547) - actor/pg_loss:np.float64(-4.3141539208590984e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003405330644454807) - actor/ppo_kl:np.float64(4.417660383637667e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.26127634942531586) - perf/mfu/actor:np.float64(0.10783332125209669) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(99.68988418579102) - actor/lr:np.float64(8e-07) - training/global_step:9 - training/epoch:0 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010788856074213982 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010788856074213982 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:588.94140625 - response_length/max:2145.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:588.94140625 - response_length_non_aborted/max:2145.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.4375 - prompt_length/max:362.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011837668716907501 - timing_s/agent_loop/generate_sequences/min:np.float64(0.915661258623004) - timing_s/agent_loop/generate_sequences/max:np.float64(15.536512579768896) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.094332097287406) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.536512579768896) - timing_s/
(TaskRunner pid=2281424) 
Training Progress:   9%|▉         | 9/100 [06:53<1:05:43, 43.33s/it]
(TaskRunner pid=2281424) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 10}
(TaskRunner pid=2281424) validation generation end
(TaskRunner pid=2281424) [prompt] system
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2281424) <reasoning>
(TaskRunner pid=2281424) ...
(TaskRunner pid=2281424) </reasoning>
(TaskRunner pid=2281424) <answer>
(TaskRunner pid=2281424) ...
(TaskRunner pid=2281424) </answer>
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2281424) <answer>
(TaskRunner pid=2281424) A
(TaskRunner pid=2281424) </answer>
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) user
(TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$.
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) A: 50 pC
(TaskRunner pid=2281424) B: 56 pC
(TaskRunner pid=2281424) C: 60 pC
(TaskRunner pid=2281424) D: 64 pC
(TaskRunner pid=2281424) Please reason step by step.
(TaskRunner pid=2281424) assistant
(TaskRunner pid=2281424) <think>
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) </think>
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) [response] <reasoning>
(TaskRunner pid=2281424) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is:
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) E = k \frac{q}{r^2}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) We need to solve for $ q $, the charge. Rearranging the formula:
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q = \frac{E r^2}{k}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) Substitute the known values:
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q = \frac{(2.0 \, \mathrm{N/C}) \times (0.50 \, \mathrm{m})^2}{8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2}}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q = \frac{2.0 \times 0.25}{8.99 \times 10^9}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q = \frac{0.5}{8.99 \times 10^9}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $):
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q \approx 55.6 \, \mathrm{pC}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) The closest option is **B: 56 pC**.
(TaskRunner pid=2281424) </reasoning>
(TaskRunner pid=2281424) <answer>
(TaskRunner pid=2281424) B
(TaskRunner pid=2281424) </answer>
(TaskRunner pid=2281424) [ground_truth] B
(TaskRunner pid=2281424) [score] 1.0
(TaskRunner pid=2281424) [acc] 1.0
(TaskRunner pid=2281424) [pred] B
(TaskRunner pid=2281424) [incorrect_format] 1
(TaskRunner pid=2281424) [feedback] 
(TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10
(TaskRunner pid=2281424) step:10 - global_seqlen/min:18370 - global_seqlen/max:31673 - global_seqlen/minmax_diff:13303 - global_seqlen/balanced_min:26421 - global_seqlen/balanced_max:26444 - global_seqlen/mean:26434.75 - actor/entropy:0.32802483439445496 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45646193623542786 - training/rollout_probs_diff_mean:0.0033578479196876287 - training/rollout_probs_diff_std:0.00903936568647623 - training/rollout_actor_probs_pearson_corr:0.9993472099304199 - rollout_corr/rollout_is_mean:0.9999197721481323 - rollout_corr/rollout_is_ratio_fraction_high:7.013114554865751e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.4026229109731503e-05 - rollout_corr/rollout_is_max:3.0499253273010254 - rollout_corr/rollout_is_min:0.36710941791534424 - rollout_corr/rollout_is_std:0.029133660718798637 - rollout_corr/rollout_is_eff_sample_size:0.999151830651037 - rollout_corr/rollout_is_seq_mean:0.9999980330467224 - rollout_corr/rollout_is_seq_std:0.0017985748127102852 - rollout_corr/rollout_is_seq_max:1.008771300315857 - rollout_corr/rollout_is_seq_min:0.9932235479354858 - rollout_corr/rollout_is_seq_max_deviation:0.008771300315856934 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.377936966251582) - rollout_corr/training_log_ppl:np.float64(0.3002843804570148) - rollout_corr/kl:np.float64(0.0005245888745832872) - rollout_corr/k3_kl:np.float64(0.000498560940002335) - rollout_corr/rollout_ppl:np.float64(1.377146857790649) - rollout_corr/rollout_log_ppl:np.float64(0.29975979137816466) - rollout_corr/log_ppl_diff:np.float64(0.0005245890788501129) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013371909590205178) - rollout_corr/log_ppl_diff_max:np.float64(0.0069696009159088135) - rollout_corr/log_ppl_diff_min:np.float64(-0.004811465740203857) - rollout_corr/ppl_ratio:np.float64(1.0005262338090688) - rollout_corr/chi2_token:np.float64(0.0009716309141367674) - rollout_corr/chi2_seq:np.float64(1.0234597465023398) - actor/pg_loss:np.float64(2.4606240913271904e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024767978288764425) - actor/ppo_kl:np.float64(4.621054888964693e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18785973638296127) - perf/mfu/actor:np.float64(0.10560325767310388) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(99.76485443115234) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.58359375) - val-aux/sciknoweval/reward/std@16:np.float64(0.21906251159087287) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.67415) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.19020325207669364) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.4956125) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.17791978642388112) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5849125000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2188371622304282) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7478) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.14805144079456356) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.425825) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12986404819208913) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6028874999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.17818578143360303) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8051999999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.10766830979919102) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.3753875) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08819187467072624) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6146375) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13509611267304375) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8500624999999999) - val-aux/sciknoweval/rewa
(TaskRunner pid=2281424) 
Training Progress:  10%|█         | 10/100 [09:39<2:02:00, 81.34s/it]
(TaskRunner pid=2281424) step:11 - global_seqlen/min:20995 - global_seqlen/max:35423 - global_seqlen/minmax_diff:14428 - global_seqlen/balanced_min:27571 - global_seqlen/balanced_max:27590 - global_seqlen/mean:27580.5 - actor/entropy:0.24689078330993652 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6190903186798096 - training/rollout_probs_diff_mean:0.0029323738999664783 - training/rollout_probs_diff_std:0.008981931023299694 - training/rollout_actor_probs_pearson_corr:0.999229907989502 - rollout_corr/rollout_is_mean:0.9999557137489319 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.325087163830176e-05 - rollout_corr/rollout_is_max:1.9537171125411987 - rollout_corr/rollout_is_min:0.08846962451934814 - rollout_corr/rollout_is_std:0.0269706342369318 - rollout_corr/rollout_is_eff_sample_size:0.9992731136632474 - rollout_corr/rollout_is_seq_mean:1.0000543594360352 - rollout_corr/rollout_is_seq_std:0.0017303781351074576 - rollout_corr/rollout_is_seq_max:1.0074354410171509 - rollout_corr/rollout_is_seq_min:0.9947044253349304 - rollout_corr/rollout_is_seq_max_deviation:0.007435441017150879 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2710583903826773) - rollout_corr/training_log_ppl:np.float64(0.22995059203822166) - rollout_corr/kl:np.float64(0.0004979196115613149) - rollout_corr/k3_kl:np.float64(0.00043510435352800414) - rollout_corr/rollout_ppl:np.float64(1.270424968097359) - rollout_corr/rollout_log_ppl:np.float64(0.22945267148315907) - rollout_corr/log_ppl_diff:np.float64(0.000497920555062592) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013768301578238606) - rollout_corr/log_ppl_diff_max:np.float64(0.006137266755104065) - rollout_corr/log_ppl_diff_min:np.float64(-0.007355779409408569) - rollout_corr/ppl_ratio:np.float64(1.000499751418829) - rollout_corr/chi2_token:np.float64(0.0007434843573719263) - rollout_corr/chi2_seq:np.float64(0.47075980668887496) - actor/pg_loss:np.float64(4.087574779987335e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001813313299408037) - actor/ppo_kl:np.float64(0.00013543530537063475) - actor/pg_clipfrac_lower:np.float64(4.278477717889473e-06) - actor/grad_norm:np.float64(0.17897598445415497) - perf/mfu/actor:np.float64(0.10733546751479987) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(104.49744415283203) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004445641301572323 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.004445641301572323 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:587.390625 - response_length/max:1781.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:587.390625 - response_length_non_aborted/max:1781.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.5 - prompt_length/max:437.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.45193862915039e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9449637047946453) - timing_s/agent_loop/generate_sequences/max:np.float64(13.539711721241474) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.947522593545727) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.539711721241474) - timing_s/agent_lo
(TaskRunner pid=2281424) 
Training Progress:  11%|█         | 11/100 [10:17<1:40:48, 67.96s/it]
(TaskRunner pid=2281424) step:12 - global_seqlen/min:18284 - global_seqlen/max:36197 - global_seqlen/minmax_diff:17913 - global_seqlen/balanced_min:27117 - global_seqlen/balanced_max:27742 - global_seqlen/mean:27219.25 - actor/entropy:0.2918388545513153 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18463563919067383 - training/rollout_probs_diff_mean:0.00314357434399426 - training/rollout_probs_diff_std:0.008670532144606113 - training/rollout_actor_probs_pearson_corr:0.9993590712547302 - rollout_corr/rollout_is_mean:0.9999263882637024 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.328426969848806e-05 - rollout_corr/rollout_is_max:1.6952860355377197 - rollout_corr/rollout_is_min:0.2599896490573883 - rollout_corr/rollout_is_std:0.02788124419748783 - rollout_corr/rollout_is_eff_sample_size:0.9992230019996114 - rollout_corr/rollout_is_seq_mean:0.9999516010284424 - rollout_corr/rollout_is_seq_std:0.0013419243041425943 - rollout_corr/rollout_is_seq_max:1.0067940950393677 - rollout_corr/rollout_is_seq_min:0.9959049224853516 - rollout_corr/rollout_is_seq_max_deviation:0.006794095039367676 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.284353747498244) - rollout_corr/training_log_ppl:np.float64(0.24118711793562397) - rollout_corr/kl:np.float64(0.0004640274163274505) - rollout_corr/k3_kl:np.float64(0.000438987553025072) - rollout_corr/rollout_ppl:np.float64(1.2837417596019804) - rollout_corr/rollout_log_ppl:np.float64(0.2407230889366474) - rollout_corr/log_ppl_diff:np.float64(0.00046402899897657335) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012675680045504123) - rollout_corr/log_ppl_diff_max:np.float64(0.00536249577999115) - rollout_corr/log_ppl_diff_min:np.float64(-0.005736485123634338) - rollout_corr/ppl_ratio:np.float64(1.0004653770010918) - rollout_corr/chi2_token:np.float64(0.0008103374857455492) - rollout_corr/chi2_seq:np.float64(0.7635552103165537) - actor/pg_loss:np.float64(6.2944600358605385e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022421280800699606) - actor/ppo_kl:np.float64(8.907836768656807e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1681164726614952) - perf/mfu/actor:np.float64(0.10657234495140916) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.30431365966797) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.5234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02310134656727314 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02310134656727314 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:588.1015625 - response_length/max:3570.0 - response_length/min:116.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:588.1015625 - response_length_non_aborted/max:3570.0 - response_length_non_aborted/min:116.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.5 - prompt_length/max:364.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.758848369121552e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9172126762568951) - timing_s/agent_loop/generate_sequences/max:np.float64(22.92699559032917) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.20948198466067) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.92699559032917) - timing_s/agent_loop/slowest/tool_ca
(TaskRunner pid=2281424) 
Training Progress:  12%|█▏        | 12/100 [11:03<1:30:10, 61.48s/it]
(TaskRunner pid=2281424) step:13 - global_seqlen/min:24964 - global_seqlen/max:33451 - global_seqlen/minmax_diff:8487 - global_seqlen/balanced_min:28949 - global_seqlen/balanced_max:28972 - global_seqlen/mean:28959.375 - actor/entropy:0.31463027000427246 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24114426970481873 - training/rollout_probs_diff_mean:0.00320067978464067 - training/rollout_probs_diff_std:0.008584943599998951 - training/rollout_actor_probs_pearson_corr:0.9994125962257385 - rollout_corr/rollout_is_mean:1.0001263618469238 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.238653440959752e-06 - rollout_corr/rollout_is_max:1.859130859375 - rollout_corr/rollout_is_min:0.4844902455806732 - rollout_corr/rollout_is_std:0.02811329998075962 - rollout_corr/rollout_is_eff_sample_size:0.9992105046242893 - rollout_corr/rollout_is_seq_mean:1.0000969171524048 - rollout_corr/rollout_is_seq_std:0.0011731568956747651 - rollout_corr/rollout_is_seq_max:1.0035706758499146 - rollout_corr/rollout_is_seq_min:0.9957525134086609 - rollout_corr/rollout_is_seq_max_deviation:0.004247486591339111 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3780980156734586) - rollout_corr/training_log_ppl:np.float64(0.2991240085102618) - rollout_corr/kl:np.float64(0.0003781683996528784) - rollout_corr/k3_kl:np.float64(0.0004593968702977236) - rollout_corr/rollout_ppl:np.float64(1.3775346302427351) - rollout_corr/rollout_log_ppl:np.float64(0.2987458378338488) - rollout_corr/log_ppl_diff:np.float64(0.0003781706764129922) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010912520374404266) - rollout_corr/log_ppl_diff_max:np.float64(0.004893004894256592) - rollout_corr/log_ppl_diff_min:np.float64(-0.003899678587913513) - rollout_corr/ppl_ratio:np.float64(1.0003791959024966) - rollout_corr/chi2_token:np.float64(0.001073798630386591) - rollout_corr/chi2_seq:np.float64(0.506177173461765) - actor/pg_loss:np.float64(3.237277269363403e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004119513098430616) - actor/ppo_kl:np.float64(7.283486395426308e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21577078476548195) - perf/mfu/actor:np.float64(0.11467366616050297) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.31635284423828) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.5703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0002386284904787317 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0002386284904787317 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:626.13671875 - response_length/max:1861.0 - response_length/min:160.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:626.13671875 - response_length_non_aborted/max:1861.0 - response_length_non_aborted/min:160.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.84375 - prompt_length/max:363.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013312511146068573 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9897179938852787) - timing_s/agent_loop/generate_sequences/max:np.float64(15.492640441283584) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.851898552646162) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.492640441283584) - timing_s/agent_loop/slowest/
(TaskRunner pid=2281424) 
Training Progress:  13%|█▎        | 13/100 [11:43<1:19:27, 54.80s/it]
(TaskRunner pid=2281424) step:14 - global_seqlen/min:18482 - global_seqlen/max:39950 - global_seqlen/minmax_diff:21468 - global_seqlen/balanced_min:27989 - global_seqlen/balanced_max:28043 - global_seqlen/mean:28025.875 - actor/entropy:0.27694693207740784 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9996616840362549 - training/rollout_probs_diff_mean:0.0032118367962539196 - training/rollout_probs_diff_std:0.009403628297150135 - training/rollout_actor_probs_pearson_corr:0.9992220997810364 - rollout_corr/rollout_is_mean:0.9999675154685974 - rollout_corr/rollout_is_ratio_fraction_high:2.5921677661244757e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.2402098440798e-05 - rollout_corr/rollout_is_max:2.0761139392852783 - rollout_corr/rollout_is_min:0.000335350981913507 - rollout_corr/rollout_is_std:0.028911858797073364 - rollout_corr/rollout_is_eff_sample_size:0.9991645645790151 - rollout_corr/rollout_is_seq_mean:0.9999456405639648 - rollout_corr/rollout_is_seq_std:0.001373259350657463 - rollout_corr/rollout_is_seq_max:1.0052764415740967 - rollout_corr/rollout_is_seq_min:0.995773434638977 - rollout_corr/rollout_is_seq_max_deviation:0.00527644157409668 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3075349098071456) - rollout_corr/training_log_ppl:np.float64(0.2546500996249961) - rollout_corr/kl:np.float64(0.0006805974277988014) - rollout_corr/k3_kl:np.float64(0.0006370248192979489) - rollout_corr/rollout_ppl:np.float64(1.3065974819473922) - rollout_corr/rollout_log_ppl:np.float64(0.2539695030573057) - rollout_corr/log_ppl_diff:np.float64(0.0006805965676903725) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013074282323941588) - rollout_corr/log_ppl_diff_max:np.float64(0.01114201545715332) - rollout_corr/log_ppl_diff_min:np.float64(-0.004195064306259155) - rollout_corr/ppl_ratio:np.float64(1.000682536046952) - rollout_corr/chi2_token:np.float64(0.0010629112366586924) - rollout_corr/chi2_seq:np.float64(0.7319330512546003) - actor/pg_loss:np.float64(-7.280893623828888e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004904413003714581) - actor/ppo_kl:np.float64(0.00013513596405800854) - actor/pg_clipfrac_lower:np.float64(5.243288796918932e-06) - actor/grad_norm:np.float64(0.22375838086009026) - perf/mfu/actor:np.float64(0.11024404131767285) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.50100326538086) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004044591914862394 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004044591914862394 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:602.77734375 - response_length/max:2091.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:602.77734375 - response_length_non_aborted/max:2091.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.03125 - prompt_length/max:375.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.801425039768219e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1079022623598576) - timing_s/agent_loop/generate_sequences/max:np.float64(15.903715757653117) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.53672656758863) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.903715
(TaskRunner pid=2281424) 
Training Progress:  14%|█▍        | 14/100 [12:23<1:12:00, 50.24s/it]
(TaskRunner pid=2281424) step:15 - global_seqlen/min:18409 - global_seqlen/max:34972 - global_seqlen/minmax_diff:16563 - global_seqlen/balanced_min:27584 - global_seqlen/balanced_max:27657 - global_seqlen/mean:27632.0 - actor/entropy:0.23792505264282227 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2081579566001892 - training/rollout_probs_diff_mean:0.0028373654931783676 - training/rollout_probs_diff_std:0.008734497241675854 - training/rollout_actor_probs_pearson_corr:0.9992759227752686 - rollout_corr/rollout_is_mean:1.000104308128357 - rollout_corr/rollout_is_ratio_fraction_high:6.5414201344538014e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0567126274108887 - rollout_corr/rollout_is_min:0.6020901799201965 - rollout_corr/rollout_is_std:0.0269706342369318 - rollout_corr/rollout_is_eff_sample_size:0.9992733517354029 - rollout_corr/rollout_is_seq_mean:1.0001739263534546 - rollout_corr/rollout_is_seq_std:0.0016109951538965106 - rollout_corr/rollout_is_seq_max:1.0077435970306396 - rollout_corr/rollout_is_seq_min:0.9901288747787476 - rollout_corr/rollout_is_seq_max_deviation:0.009871125221252441 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2825340307317674) - rollout_corr/training_log_ppl:np.float64(0.2307681181409862) - rollout_corr/kl:np.float64(0.00024018173945883348) - rollout_corr/k3_kl:np.float64(0.00046571865181022076) - rollout_corr/rollout_ppl:np.float64(1.2822559108026326) - rollout_corr/rollout_log_ppl:np.float64(0.23052793518581893) - rollout_corr/log_ppl_diff:np.float64(0.0002401829551672563) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012830473860958591) - rollout_corr/log_ppl_diff_max:np.float64(0.011005207896232605) - rollout_corr/log_ppl_diff_min:np.float64(-0.004439152777194977) - rollout_corr/ppl_ratio:np.float64(1.0002417913638055) - rollout_corr/chi2_token:np.float64(0.0013833614066243172) - rollout_corr/chi2_seq:np.float64(0.8579549191053957) - actor/pg_loss:np.float64(0.00013076758477836847) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022705169044456852) - actor/ppo_kl:np.float64(1.2435915886399584e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1974821835756302) - perf/mfu/actor:np.float64(0.10945352184092855) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.5802116394043) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011041100136935711 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011041100136935711 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:597.15625 - response_length/max:2544.0 - response_length/min:91.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:597.15625 - response_length_non_aborted/max:2544.0 - response_length_non_aborted/min:91.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:460.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001034904271364212 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0850455239415169) - timing_s/agent_loop/generate_sequences/max:np.float64(18.34066752716899) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.139947620045859) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.34066752716899) - timing_s/agent_loop/slowest/to
(TaskRunner pid=2281424) 
Training Progress:  15%|█▌        | 15/100 [13:04<1:07:32, 47.68s/it]
(TaskRunner pid=2281424) step:16 - global_seqlen/min:19774 - global_seqlen/max:37148 - global_seqlen/minmax_diff:17374 - global_seqlen/balanced_min:26832 - global_seqlen/balanced_max:26869 - global_seqlen/mean:26857.125 - actor/entropy:0.269243985414505 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29323524236679077 - training/rollout_probs_diff_mean:0.0030834192875772715 - training/rollout_probs_diff_std:0.009029646404087543 - training/rollout_actor_probs_pearson_corr:0.9992733001708984 - rollout_corr/rollout_is_mean:1.000123381614685 - rollout_corr/rollout_is_ratio_fraction_high:1.3476997992256656e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.39180064201355 - rollout_corr/rollout_is_min:0.5188149213790894 - rollout_corr/rollout_is_std:0.028928346931934357 - rollout_corr/rollout_is_eff_sample_size:0.9991640885384842 - rollout_corr/rollout_is_seq_mean:1.0000724792480469 - rollout_corr/rollout_is_seq_std:0.0016198219964280725 - rollout_corr/rollout_is_seq_max:1.0117334127426147 - rollout_corr/rollout_is_seq_min:0.9945037961006165 - rollout_corr/rollout_is_seq_max_deviation:0.011733412742614746 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2698871567845345) - rollout_corr/training_log_ppl:np.float64(0.22661357806646265) - rollout_corr/kl:np.float64(0.000278438638499523) - rollout_corr/k3_kl:np.float64(0.0004574081480726022) - rollout_corr/rollout_ppl:np.float64(1.2695180620066822) - rollout_corr/rollout_log_ppl:np.float64(0.22633513917389791) - rollout_corr/log_ppl_diff:np.float64(0.0002784388925647363) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010494471789570525) - rollout_corr/log_ppl_diff_max:np.float64(0.00465761125087738) - rollout_corr/log_ppl_diff_min:np.float64(-0.009550660848617554) - rollout_corr/ppl_ratio:np.float64(1.000279544852674) - rollout_corr/chi2_token:np.float64(0.0013024122454226017) - rollout_corr/chi2_seq:np.float64(0.9144449539016932) - actor/pg_loss:np.float64(-1.0943855158984661e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023535769855698163) - actor/ppo_kl:np.float64(-7.823348023361376e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21825583279132843) - perf/mfu/actor:np.float64(0.10663394115254357) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.46512222290039) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005873444024473429 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005873444024473429 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:579.69140625 - response_length/max:1869.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:579.69140625 - response_length_non_aborted/max:1869.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.59375 - prompt_length/max:344.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001010652631521225 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3836285527795553) - timing_s/agent_loop/generate_sequences/max:np.float64(14.655153315514326) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.240751736462698) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.655153315514326) - timing_s/agent_loop/slow
(TaskRunner pid=2281424) 
Training Progress:  16%|█▌        | 16/100 [13:42<1:02:41, 44.77s/it]
(TaskRunner pid=2281424) step:17 - global_seqlen/min:21078 - global_seqlen/max:38139 - global_seqlen/minmax_diff:17061 - global_seqlen/balanced_min:27815 - global_seqlen/balanced_max:28195 - global_seqlen/mean:27876.125 - actor/entropy:0.3084404766559601 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105865716934204 - training/rollout_probs_diff_mean:0.003280392149463296 - training/rollout_probs_diff_std:0.008956593461334705 - training/rollout_actor_probs_pearson_corr:0.9993877410888672 - rollout_corr/rollout_is_mean:1.0001431703567505 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.458905318140751e-06 - rollout_corr/rollout_is_max:1.8297889232635498 - rollout_corr/rollout_is_min:0.49442058801651 - rollout_corr/rollout_is_std:0.02931314706802368 - rollout_corr/rollout_is_eff_sample_size:0.9991418341499507 - rollout_corr/rollout_is_seq_mean:1.0001916885375977 - rollout_corr/rollout_is_seq_std:0.0019043951760977507 - rollout_corr/rollout_is_seq_max:1.009162425994873 - rollout_corr/rollout_is_seq_min:0.9893993139266968 - rollout_corr/rollout_is_seq_max_deviation:0.010600686073303223 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3411333807744086) - rollout_corr/training_log_ppl:np.float64(0.2804811333771795) - rollout_corr/kl:np.float64(0.00039379035426634346) - rollout_corr/k3_kl:np.float64(0.0005168410488067821) - rollout_corr/rollout_ppl:np.float64(1.3405519826337695) - rollout_corr/rollout_log_ppl:np.float64(0.28008734311151784) - rollout_corr/log_ppl_diff:np.float64(0.0003937902656616643) - rollout_corr/log_ppl_abs_diff:np.float64(0.001501921404269524) - rollout_corr/log_ppl_diff_max:np.float64(0.011711418628692627) - rollout_corr/log_ppl_diff_min:np.float64(-0.007628694176673889) - rollout_corr/ppl_ratio:np.float64(1.0003962186165154) - rollout_corr/chi2_token:np.float64(0.001294472487643361) - rollout_corr/chi2_seq:np.float64(1.0754117609467357) - actor/pg_loss:np.float64(-0.00010975566692650318) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020725219656014815) - actor/ppo_kl:np.float64(0.00011342834537941826) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2514360435307026) - perf/mfu/actor:np.float64(0.10841431290124035) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.30791854858398) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012801550328731537 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012801550328731537 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:604.78515625 - response_length/max:2740.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:604.78515625 - response_length_non_aborted/max:2740.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.358139216899872e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0903181321918964) - timing_s/agent_loop/generate_sequences/max:np.float64(20.050666142255068) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.366174238610256) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.050666142255068) - timing_s/agent_loop/sl
(TaskRunner pid=2281424) 
Training Progress:  17%|█▋        | 17/100 [14:27<1:01:48, 44.68s/it]
(TaskRunner pid=2281424) step:18 - global_seqlen/min:18777 - global_seqlen/max:44614 - global_seqlen/minmax_diff:25837 - global_seqlen/balanced_min:32860 - global_seqlen/balanced_max:32946 - global_seqlen/mean:32927.0 - actor/entropy:0.2818203866481781 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2900807857513428 - training/rollout_probs_diff_mean:0.002938726684078574 - training/rollout_probs_diff_std:0.00846165232360363 - training/rollout_actor_probs_pearson_corr:0.9994096159934998 - rollout_corr/rollout_is_mean:1.0000146627426147 - rollout_corr/rollout_is_ratio_fraction_high:5.105583568365546e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.552791738708038e-05 - rollout_corr/rollout_is_max:2.0839157104492188 - rollout_corr/rollout_is_min:0.4158320426940918 - rollout_corr/rollout_is_std:0.02699493244290352 - rollout_corr/rollout_is_eff_sample_size:0.9992719233041721 - rollout_corr/rollout_is_seq_mean:1.0000157356262207 - rollout_corr/rollout_is_seq_std:0.0014222179306671023 - rollout_corr/rollout_is_seq_max:1.0088167190551758 - rollout_corr/rollout_is_seq_min:0.995079755783081 - rollout_corr/rollout_is_seq_max_deviation:0.008816719055175781 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.289564796257764) - rollout_corr/training_log_ppl:np.float64(0.2442400810541585) - rollout_corr/kl:np.float64(0.0005266999699861663) - rollout_corr/k3_kl:np.float64(0.0004649700730112727) - rollout_corr/rollout_ppl:np.float64(1.2888782764784992) - rollout_corr/rollout_log_ppl:np.float64(0.24371338007040322) - rollout_corr/log_ppl_diff:np.float64(0.0005267009837552905) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011411418090574443) - rollout_corr/log_ppl_diff_max:np.float64(0.010496377944946289) - rollout_corr/log_ppl_diff_min:np.float64(-0.007471442222595215) - rollout_corr/ppl_ratio:np.float64(1.0005280999466777) - rollout_corr/chi2_token:np.float64(0.0008352911099791527) - rollout_corr/chi2_seq:np.float64(0.4274897330906242) - actor/pg_loss:np.float64(9.232200682163239e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002937083892220471) - actor/ppo_kl:np.float64(0.00015827277887936475) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1933615617454052) - perf/mfu/actor:np.float64(0.12426924001280201) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.3476791381836) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007050810847431421 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007050810847431421 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:765.09375 - response_length/max:3008.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:765.09375 - response_length_non_aborted/max:3008.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.875 - prompt_length/max:349.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.749613046646118e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9996219798922539) - timing_s/agent_loop/generate_sequences/max:np.float64(21.299407305195928) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.81618346637697) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.299407305195928) - timing_s/agent_loop/s
(TaskRunner pid=2281424) 
Training Progress:  18%|█▊        | 18/100 [15:13<1:01:41, 45.14s/it]
(TaskRunner pid=2281424) step:19 - global_seqlen/min:21194 - global_seqlen/max:44487 - global_seqlen/minmax_diff:23293 - global_seqlen/balanced_min:30623 - global_seqlen/balanced_max:30828 - global_seqlen/mean:30657.0 - actor/entropy:0.2834802567958832 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35189008712768555 - training/rollout_probs_diff_mean:0.002977204043418169 - training/rollout_probs_diff_std:0.00879612471908331 - training/rollout_actor_probs_pearson_corr:0.9994228482246399 - rollout_corr/rollout_is_mean:1.0000439882278442 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.2517451725434512e-05 - rollout_corr/rollout_is_max:1.6279520988464355 - rollout_corr/rollout_is_min:0.2959691882133484 - rollout_corr/rollout_is_std:0.027716144919395447 - rollout_corr/rollout_is_eff_sample_size:0.9992325240192108 - rollout_corr/rollout_is_seq_mean:1.000134825706482 - rollout_corr/rollout_is_seq_std:0.0016275998204946518 - rollout_corr/rollout_is_seq_max:1.009211540222168 - rollout_corr/rollout_is_seq_min:0.9948177337646484 - rollout_corr/rollout_is_seq_max_deviation:0.009211540222167969 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2855765600688756) - rollout_corr/training_log_ppl:np.float64(0.23977286802255549) - rollout_corr/kl:np.float64(0.0005027599546281181) - rollout_corr/k3_kl:np.float64(0.0004755220272727456) - rollout_corr/rollout_ppl:np.float64(1.2849308247677982) - rollout_corr/rollout_log_ppl:np.float64(0.2392701047792798) - rollout_corr/log_ppl_diff:np.float64(0.0005027632432756945) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011823297390947118) - rollout_corr/log_ppl_diff_max:np.float64(0.009016215801239014) - rollout_corr/log_ppl_diff_min:np.float64(-0.008047908544540405) - rollout_corr/ppl_ratio:np.float64(1.0005042462144047) - rollout_corr/chi2_token:np.float64(0.0009039849974215031) - rollout_corr/chi2_seq:np.float64(0.45300188730470836) - actor/pg_loss:np.float64(8.254800923168659e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003364918597981159) - actor/ppo_kl:np.float64(0.0002041434732245051) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.19276084750890732) - perf/mfu/actor:np.float64(0.11752787272188393) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.36764144897461) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0269034281373024 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0269034281373024 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:693.90625 - response_length/max:3216.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:693.90625 - response_length_non_aborted/max:3216.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.125 - prompt_length/max:339.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013180077075958252 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2591454926878214) - timing_s/agent_loop/generate_sequences/max:np.float64(22.810867067426443) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.210481837464613) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.810867067426443) - timing_s/agent_loop/slowest/tool_calls
(TaskRunner pid=2281424) 
Training Progress:  19%|█▉        | 19/100 [16:01<1:01:55, 45.87s/it]
(TaskRunner pid=2281424) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20}
(TaskRunner pid=2281424) validation generation end
(TaskRunner pid=2281424) [prompt] system
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2281424) <reasoning>
(TaskRunner pid=2281424) ...
(TaskRunner pid=2281424) </reasoning>
(TaskRunner pid=2281424) <answer>
(TaskRunner pid=2281424) ...
(TaskRunner pid=2281424) </answer>
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2281424) <answer>
(TaskRunner pid=2281424) A
(TaskRunner pid=2281424) </answer>
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) user
(TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$.
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) A: 50 pC
(TaskRunner pid=2281424) B: 56 pC
(TaskRunner pid=2281424) C: 60 pC
(TaskRunner pid=2281424) D: 64 pC
(TaskRunner pid=2281424) Please reason step by step.
(TaskRunner pid=2281424) assistant
(TaskRunner pid=2281424) <think>
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) </think>
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) [response] <reasoning>
(TaskRunner pid=2281424) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The electric field due to a point charge is given by:
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) E = k \frac{q}{r^2}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) We need to solve for the charge $ q $:
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q = \frac{E \cdot r^2}{k}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) Substituting the given values:
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q = \frac{0.50}{8.99 \times 10^9}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) Convert to picocoulombs ($ 1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C} $):
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q \approx 55.6 \, \mathrm{pC}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) The closest option is:
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) <answer>
(TaskRunner pid=2281424) B
(TaskRunner pid=2281424) </answer>
(TaskRunner pid=2281424) [ground_truth] B
(TaskRunner pid=2281424) [score] 1.0
(TaskRunner pid=2281424) [acc] 1.0
(TaskRunner pid=2281424) [pred] B
(TaskRunner pid=2281424) [incorrect_format] 1
(TaskRunner pid=2281424) [feedback] 
(TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20
(TaskRunner pid=2281424) step:20 - global_seqlen/min:19499 - global_seqlen/max:36025 - global_seqlen/minmax_diff:16526 - global_seqlen/balanced_min:27704 - global_seqlen/balanced_max:28173 - global_seqlen/mean:27765.0 - actor/entropy:0.280457079410553 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135375022888184 - training/rollout_probs_diff_mean:0.003101746318861842 - training/rollout_probs_diff_std:0.008900674059987068 - training/rollout_actor_probs_pearson_corr:0.999406099319458 - rollout_corr/rollout_is_mean:0.9998956918716431 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9279214029666036e-05 - rollout_corr/rollout_is_max:1.8279300928115845 - rollout_corr/rollout_is_min:0.40598663687705994 - rollout_corr/rollout_is_std:0.028816867619752884 - rollout_corr/rollout_is_eff_sample_size:0.9991700390777238 - rollout_corr/rollout_is_seq_mean:0.999803900718689 - rollout_corr/rollout_is_seq_std:0.001518259639851749 - rollout_corr/rollout_is_seq_max:1.0038286447525024 - rollout_corr/rollout_is_seq_min:0.9914582967758179 - rollout_corr/rollout_is_seq_max_deviation:0.008541703224182129 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2934490512125194) - rollout_corr/training_log_ppl:np.float64(0.24451396349468268) - rollout_corr/kl:np.float64(0.0004959776629758217) - rollout_corr/k3_kl:np.float64(0.0004771725971863816) - rollout_corr/rollout_ppl:np.float64(1.2928053527139127) - rollout_corr/rollout_log_ppl:np.float64(0.2440179859258933) - rollout_corr/log_ppl_diff:np.float64(0.0004959775687893853) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011907425214303657) - rollout_corr/log_ppl_diff_max:np.float64(0.009751290082931519) - rollout_corr/log_ppl_diff_min:np.float64(-0.004419475793838501) - rollout_corr/ppl_ratio:np.float64(1.0004974079784006) - rollout_corr/chi2_token:np.float64(0.0009594245348125696) - rollout_corr/chi2_seq:np.float64(0.41463141702115536) - actor/pg_loss:np.float64(1.230929046869278e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021071931905680685) - actor/ppo_kl:np.float64(-0.0001457839747551759) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1746744140982628) - perf/mfu/actor:np.float64(0.10426063797607768) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.34674835205078) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.59921875) - val-aux/sciknoweval/reward/std@16:np.float64(0.20532643626967628) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.684225) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.18261389709388515) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.514475) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1643688800388951) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.60015) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2055792779718459) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7575125) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.14221272724332604) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4511375) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11570816324542939) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6215249999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.17291949688631747) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8132375) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09937488352535821) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4064375) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07134373339436899) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6354124999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1400502136179348) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8518374999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.
(TaskRunner pid=2281424) 
Training Progress:  20%|██        | 20/100 [19:23<2:03:58, 92.98s/it]
(TaskRunner pid=2281424) step:21 - global_seqlen/min:25301 - global_seqlen/max:34398 - global_seqlen/minmax_diff:9097 - global_seqlen/balanced_min:29031 - global_seqlen/balanced_max:29324 - global_seqlen/mean:29106.375 - actor/entropy:0.24065154790878296 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37683379650115967 - training/rollout_probs_diff_mean:0.002658553421497345 - training/rollout_probs_diff_std:0.008330690674483776 - training/rollout_actor_probs_pearson_corr:0.9994138479232788 - rollout_corr/rollout_is_mean:1.0000982284545898 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.2177079042885453e-05 - rollout_corr/rollout_is_max:1.7682775259017944 - rollout_corr/rollout_is_min:0.3936323821544647 - rollout_corr/rollout_is_std:0.02563127689063549 - rollout_corr/rollout_is_eff_sample_size:0.9993435879743411 - rollout_corr/rollout_is_seq_mean:1.0001851320266724 - rollout_corr/rollout_is_seq_std:0.0012425798922777176 - rollout_corr/rollout_is_seq_max:1.0056188106536865 - rollout_corr/rollout_is_seq_min:0.9965409636497498 - rollout_corr/rollout_is_seq_max_deviation:0.0056188106536865234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2498526307754219) - rollout_corr/training_log_ppl:np.float64(0.21081234834855422) - rollout_corr/kl:np.float64(0.00032340780622153176) - rollout_corr/k3_kl:np.float64(0.0003890872453951033) - rollout_corr/rollout_ppl:np.float64(1.2494138525798917) - rollout_corr/rollout_log_ppl:np.float64(0.21048893986153416) - rollout_corr/log_ppl_diff:np.float64(0.0003234084870200604) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010021366470027715) - rollout_corr/log_ppl_diff_max:np.float64(0.004034250974655151) - rollout_corr/log_ppl_diff_min:np.float64(-0.004479348659515381) - rollout_corr/ppl_ratio:np.float64(1.0003242590464652) - rollout_corr/chi2_token:np.float64(0.0009132004342973232) - rollout_corr/chi2_seq:np.float64(0.7536862585693598) - actor/pg_loss:np.float64(3.263808321207762e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000137664294015849) - actor/ppo_kl:np.float64(0.00017732822421123728) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.13653844129294157) - perf/mfu/actor:np.float64(0.11284239253639325) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(104.92810821533203) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0021431660279631615 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0021431660279631615 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:641.57421875 - response_length/max:3017.0 - response_length/min:180.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:641.57421875 - response_length_non_aborted/max:3017.0 - response_length_non_aborted/min:180.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.0 - prompt_length/max:364.0 - prompt_length/min:195.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.9451907873153687e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2093537878245115) - timing_s/agent_loop/generate_sequences/max:np.float64(21.849115535616875) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.7584272636668175) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.849115535616875) - timing_s/agent_lo
(TaskRunner pid=2281424) 
Training Progress:  21%|██        | 21/100 [20:09<1:43:38, 78.71s/it]
(TaskRunner pid=2281424) step:22 - global_seqlen/min:20123 - global_seqlen/max:45886 - global_seqlen/minmax_diff:25763 - global_seqlen/balanced_min:32892 - global_seqlen/balanced_max:32946 - global_seqlen/mean:32919.625 - actor/entropy:0.2954756021499634 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5972118377685547 - training/rollout_probs_diff_mean:0.0029836043249815702 - training/rollout_probs_diff_std:0.008759740740060806 - training/rollout_actor_probs_pearson_corr:0.9994177222251892 - rollout_corr/rollout_is_mean:1.000027060508728 - rollout_corr/rollout_is_ratio_fraction_high:1.0150790330953896e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.5226184586936142e-05 - rollout_corr/rollout_is_max:5.339309215545654 - rollout_corr/rollout_is_min:0.3289409577846527 - rollout_corr/rollout_is_std:0.027539236471056938 - rollout_corr/rollout_is_eff_sample_size:0.999242284277643 - rollout_corr/rollout_is_seq_mean:1.000131607055664 - rollout_corr/rollout_is_seq_std:0.0019331261282786727 - rollout_corr/rollout_is_seq_max:1.0212081670761108 - rollout_corr/rollout_is_seq_min:0.990929365158081 - rollout_corr/rollout_is_seq_max_deviation:0.02120816707611084 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2931092074140906) - rollout_corr/training_log_ppl:np.float64(0.24355409591225907) - rollout_corr/kl:np.float64(0.0005864815489502462) - rollout_corr/k3_kl:np.float64(0.0005580453379678829) - rollout_corr/rollout_ppl:np.float64(1.292321430053562) - rollout_corr/rollout_log_ppl:np.float64(0.24296761279401835) - rollout_corr/log_ppl_diff:np.float64(0.0005864831182407215) - rollout_corr/log_ppl_abs_diff:np.float64(0.001289499065023847) - rollout_corr/log_ppl_diff_max:np.float64(0.019712358713150024) - rollout_corr/log_ppl_diff_min:np.float64(-0.007148668169975281) - rollout_corr/ppl_ratio:np.float64(1.0005892012268305) - rollout_corr/chi2_token:np.float64(0.0012211541179567575) - rollout_corr/chi2_seq:np.float64(0.6782732140272856) - actor/pg_loss:np.float64(0.00016800372395664454) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002511092553731942) - actor/ppo_kl:np.float64(0.00027240615360524245) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29514269530773163) - perf/mfu/actor:np.float64(0.12364022737507745) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(104.99796676635742) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005714894738048315 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005714894738048315 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:769.64453125 - response_length/max:2870.0 - response_length/min:138.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:769.64453125 - response_length_non_aborted/max:2870.0 - response_length_non_aborted/min:138.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.09375 - prompt_length/max:375.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.717978537082672e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7690473347902298) - timing_s/agent_loop/generate_sequences/max:np.float64(21.679362626746297) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.051158084097551) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.679362626746297) - timin
(TaskRunner pid=2281424) 
Training Progress:  22%|██▏       | 22/100 [20:55<1:29:44, 69.04s/it]
(TaskRunner pid=2281424) step:23 - global_seqlen/min:22934 - global_seqlen/max:53809 - global_seqlen/minmax_diff:30875 - global_seqlen/balanced_min:39243 - global_seqlen/balanced_max:39551 - global_seqlen/mean:39306.5 - actor/entropy:0.33397042751312256 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4463748633861542 - training/rollout_probs_diff_mean:0.003143494250252843 - training/rollout_probs_diff_std:0.008556975051760674 - training/rollout_actor_probs_pearson_corr:0.9994155764579773 - rollout_corr/rollout_is_mean:0.9999798536300659 - rollout_corr/rollout_is_ratio_fraction_high:8.167132364178542e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.6334264728357084e-05 - rollout_corr/rollout_is_max:2.7373318672180176 - rollout_corr/rollout_is_min:0.23669613897800446 - rollout_corr/rollout_is_std:0.028452597558498383 - rollout_corr/rollout_is_eff_sample_size:0.9991909855397003 - rollout_corr/rollout_is_seq_mean:0.9999604821205139 - rollout_corr/rollout_is_seq_std:0.001166414120234549 - rollout_corr/rollout_is_seq_max:1.0046206712722778 - rollout_corr/rollout_is_seq_min:0.9961167573928833 - rollout_corr/rollout_is_seq_max_deviation:0.004620671272277832 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.385243427939713) - rollout_corr/training_log_ppl:np.float64(0.3101144570391625) - rollout_corr/kl:np.float64(0.0005338342346767888) - rollout_corr/k3_kl:np.float64(0.000465180856878078) - rollout_corr/rollout_ppl:np.float64(1.3844622108153999) - rollout_corr/rollout_log_ppl:np.float64(0.30958062058198266) - rollout_corr/log_ppl_diff:np.float64(0.0005338364571798593) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009809565672185272) - rollout_corr/log_ppl_diff_max:np.float64(0.006667211651802063) - rollout_corr/log_ppl_diff_min:np.float64(-0.0032280683517456055) - rollout_corr/ppl_ratio:np.float64(1.0005347987171263) - rollout_corr/chi2_token:np.float64(0.000802481546998024) - rollout_corr/chi2_seq:np.float64(0.3838173961266875) - actor/pg_loss:np.float64(6.97027426213026e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030751674410112173) - actor/ppo_kl:np.float64(6.243631206404032e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20673737302422523) - perf/mfu/actor:np.float64(0.1370845535950862) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(105.6169319152832) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:1 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0067174662835896015 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0067174662835896015 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:956.578125 - response_length/max:4024.0 - response_length/min:152.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:956.578125 - response_length_non_aborted/max:4024.0 - response_length_non_aborted/min:152.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.75 - prompt_length/max:375.0 - prompt_length/min:203.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00028231181204319 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8497151639312506) - timing_s/agent_loop/generate_sequences/max:np.float64(29.862245429307222) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.941006362365442) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(29.862245429307222) - timing_s/agent_loop/slowes
(TaskRunner pid=2281424) 
Training Progress:  23%|██▎       | 23/100 [21:52<1:23:49, 65.31s/it]
(TaskRunner pid=2281424) step:24 - global_seqlen/min:21247 - global_seqlen/max:40900 - global_seqlen/minmax_diff:19653 - global_seqlen/balanced_min:30630 - global_seqlen/balanced_max:32071 - global_seqlen/mean:30857.125 - actor/entropy:0.31415796279907227 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3094596862792969 - training/rollout_probs_diff_mean:0.003211991162970662 - training/rollout_probs_diff_std:0.008898185566067696 - training/rollout_actor_probs_pearson_corr:0.9993066191673279 - rollout_corr/rollout_is_mean:0.9999200105667114 - rollout_corr/rollout_is_ratio_fraction_high:5.59005866307416e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.59005866307416e-06 - rollout_corr/rollout_is_max:2.404808521270752 - rollout_corr/rollout_is_min:0.19617219269275665 - rollout_corr/rollout_is_std:0.02825077250599861 - rollout_corr/rollout_is_eff_sample_size:0.9992022922335758 - rollout_corr/rollout_is_seq_mean:0.9998365640640259 - rollout_corr/rollout_is_seq_std:0.001564831123687327 - rollout_corr/rollout_is_seq_max:1.0052552223205566 - rollout_corr/rollout_is_seq_min:0.992292582988739 - rollout_corr/rollout_is_seq_max_deviation:0.007707417011260986 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.303541206754744) - rollout_corr/training_log_ppl:np.float64(0.25006278717773966) - rollout_corr/kl:np.float64(0.0005476736968059726) - rollout_corr/k3_kl:np.float64(0.00043206283888252983) - rollout_corr/rollout_ppl:np.float64(1.3028357764706016) - rollout_corr/rollout_log_ppl:np.float64(0.24951511321705766) - rollout_corr/log_ppl_diff:np.float64(0.0005476739606820047) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011344742961227894) - rollout_corr/log_ppl_diff_max:np.float64(0.008412361145019531) - rollout_corr/log_ppl_diff_min:np.float64(-0.00302673876285553) - rollout_corr/ppl_ratio:np.float64(1.0005489422474056) - rollout_corr/chi2_token:np.float64(0.0006379920523613691) - rollout_corr/chi2_seq:np.float64(0.4201601871754974) - actor/pg_loss:np.float64(-3.0018971301615238e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.42312039076387e-05) - actor/ppo_kl:np.float64(-2.9987399832176465e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1152929738163948) - perf/mfu/actor:np.float64(0.11669484239312708) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.80078125) - perf/cpu_memory_used_gb:np.float64(105.66312026977539) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:1 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013332289643585682 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013332289643585682 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:698.78515625 - response_length/max:4217.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:698.78515625 - response_length_non_aborted/max:4217.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.5 - prompt_length/max:383.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001115109771490097 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1027332954108715) - timing_s/agent_loop/generate_sequences/max:np.float64(28.570936573669314) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.08747353394574) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(28.570936573669314) - timing_s/age
(TaskRunner pid=2281424) 
Training Progress:  24%|██▍       | 24/100 [22:45<1:17:57, 61.54s/it]
(TaskRunner pid=2281424) step:25 - global_seqlen/min:23246 - global_seqlen/max:52171 - global_seqlen/minmax_diff:28925 - global_seqlen/balanced_min:34912 - global_seqlen/balanced_max:36421 - global_seqlen/mean:35164.75 - actor/entropy:0.35677048563957214 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5146784782409668 - training/rollout_probs_diff_mean:0.0033477256074547768 - training/rollout_probs_diff_std:0.00895012728869915 - training/rollout_actor_probs_pearson_corr:0.9994227886199951 - rollout_corr/rollout_is_mean:0.9999942779541016 - rollout_corr/rollout_is_ratio_fraction_high:1.4294563698058482e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.4294563698058482e-05 - rollout_corr/rollout_is_max:2.550842046737671 - rollout_corr/rollout_is_min:0.2724951505661011 - rollout_corr/rollout_is_std:0.02962060645222664 - rollout_corr/rollout_is_eff_sample_size:0.9991233887504097 - rollout_corr/rollout_is_seq_mean:1.0000044107437134 - rollout_corr/rollout_is_seq_std:0.001464587403461337 - rollout_corr/rollout_is_seq_max:1.0044803619384766 - rollout_corr/rollout_is_seq_min:0.9913833737373352 - rollout_corr/rollout_is_seq_max_deviation:0.008616626262664795 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3856255286373198) - rollout_corr/training_log_ppl:np.float64(0.3002330707240617) - rollout_corr/kl:np.float64(0.00044336768771691126) - rollout_corr/k3_kl:np.float64(0.0005020772942430085) - rollout_corr/rollout_ppl:np.float64(1.3849827772937715) - rollout_corr/rollout_log_ppl:np.float64(0.2997897020104574) - rollout_corr/log_ppl_diff:np.float64(0.0004433687136042863) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010328854841645807) - rollout_corr/log_ppl_diff_max:np.float64(0.010214567184448242) - rollout_corr/log_ppl_diff_min:np.float64(-0.005682602524757385) - rollout_corr/ppl_ratio:np.float64(1.000444594072178) - rollout_corr/chi2_token:np.float64(0.0011366589460521936) - rollout_corr/chi2_seq:np.float64(0.17277534841559827) - actor/pg_loss:np.float64(-0.00010206212755292654) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003034779521726705) - actor/ppo_kl:np.float64(-3.511990449567293e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1835123896598816) - perf/mfu/actor:np.float64(0.1283748813468745) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(139.953125) - perf/cpu_memory_used_gb:np.float64(105.58767604827881) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:1 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.025912469252943993 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.025912469252943993 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:819.8046875 - response_length/max:5123.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:819.8046875 - response_length_non_aborted/max:5123.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.09375 - prompt_length/max:365.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001536533236503601 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2272972408682108) - timing_s/agent_loop/generate_sequences/max:np.float64(33.647482462227345) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.377409354601696) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(33.647482462227345) - timing_s/agen
(TaskRunner pid=2281424) 
Training Progress:  25%|██▌       | 25/100 [23:43<1:15:51, 60.69s/it]
(TaskRunner pid=2281424) step:26 - global_seqlen/min:24712 - global_seqlen/max:39123 - global_seqlen/minmax_diff:14411 - global_seqlen/balanced_min:30297 - global_seqlen/balanced_max:30365 - global_seqlen/mean:30344.375 - actor/entropy:0.2876966595649719 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.20822957158088684 - training/rollout_probs_diff_mean:0.0029764524661004543 - training/rollout_probs_diff_std:0.008548058569431305 - training/rollout_actor_probs_pearson_corr:0.9995021820068359 - rollout_corr/rollout_is_mean:1.0000873804092407 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9248120784759521 - rollout_corr/rollout_is_min:0.5073025822639465 - rollout_corr/rollout_is_std:0.02774408645927906 - rollout_corr/rollout_is_eff_sample_size:0.9992309766786769 - rollout_corr/rollout_is_seq_mean:1.0000227689743042 - rollout_corr/rollout_is_seq_std:0.0015661150682717562 - rollout_corr/rollout_is_seq_max:1.0046665668487549 - rollout_corr/rollout_is_seq_min:0.9934303760528564 - rollout_corr/rollout_is_seq_max_deviation:0.006569623947143555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3186388644389808) - rollout_corr/training_log_ppl:np.float64(0.2604680637450656) - rollout_corr/kl:np.float64(0.0005275833006166408) - rollout_corr/k3_kl:np.float64(0.0005207556534685409) - rollout_corr/rollout_ppl:np.float64(1.3179062088020146) - rollout_corr/rollout_log_ppl:np.float64(0.259940478586941) - rollout_corr/log_ppl_diff:np.float64(0.0005275851581245661) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012775544892065227) - rollout_corr/log_ppl_diff_max:np.float64(0.009042024612426758) - rollout_corr/log_ppl_diff_min:np.float64(-0.0037787258625030518) - rollout_corr/ppl_ratio:np.float64(1.0005293833091855) - rollout_corr/chi2_token:np.float64(0.0010106251575052738) - rollout_corr/chi2_seq:np.float64(1.5079653931315988) - actor/pg_loss:np.float64(0.00019484246149659157) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004149369976857997) - actor/ppo_kl:np.float64(0.00010909117361990184) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17218420654535294) - perf/mfu/actor:np.float64(0.11675351697015622) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(139.953125) - perf/cpu_memory_used_gb:np.float64(105.56155776977539) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:1 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007410742342472076 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007410742342472076 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:688.73046875 - response_length/max:2773.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:688.73046875 - response_length_non_aborted/max:2773.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.53125 - prompt_length/max:363.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011996924877166748 - timing_s/agent_loop/generate_sequences/min:np.float64(1.479305861517787) - timing_s/agent_loop/generate_sequences/max:np.float64(20.65613953024149) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.7965023621873115) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.65613953024149) - timing_s/agent_loop/slowest/tool_calls:np.
(TaskRunner pid=2281424) 
Training Progress:  26%|██▌       | 26/100 [24:28<1:08:50, 55.82s/it]
(TaskRunner pid=2281424) step:27 - global_seqlen/min:25977 - global_seqlen/max:48183 - global_seqlen/minmax_diff:22206 - global_seqlen/balanced_min:36093 - global_seqlen/balanced_max:36153 - global_seqlen/mean:36121.125 - actor/entropy:0.30439427495002747 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2310372292995453 - training/rollout_probs_diff_mean:0.002941888989880681 - training/rollout_probs_diff_std:0.008349236100912094 - training/rollout_actor_probs_pearson_corr:0.99952632188797 - rollout_corr/rollout_is_mean:0.9999545216560364 - rollout_corr/rollout_is_ratio_fraction_high:4.5820902414561715e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.3746271179115865e-05 - rollout_corr/rollout_is_max:2.7575740814208984 - rollout_corr/rollout_is_min:0.4973885416984558 - rollout_corr/rollout_is_std:0.027530577033758163 - rollout_corr/rollout_is_eff_sample_size:0.9992425223351087 - rollout_corr/rollout_is_seq_mean:0.9999428987503052 - rollout_corr/rollout_is_seq_std:0.0012769426684826612 - rollout_corr/rollout_is_seq_max:1.004395604133606 - rollout_corr/rollout_is_seq_min:0.9921810030937195 - rollout_corr/rollout_is_seq_max_deviation:0.007818996906280518 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3105780305340886) - rollout_corr/training_log_ppl:np.float64(0.25283126058639027) - rollout_corr/kl:np.float64(0.00047030424940075477) - rollout_corr/k3_kl:np.float64(0.0004522940477897919) - rollout_corr/rollout_ppl:np.float64(1.3099122606217861) - rollout_corr/rollout_log_ppl:np.float64(0.2523609542840859) - rollout_corr/log_ppl_diff:np.float64(0.00047030630230437964) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010534748871577904) - rollout_corr/log_ppl_diff_max:np.float64(0.009107545018196106) - rollout_corr/log_ppl_diff_min:np.float64(-0.00359538197517395) - rollout_corr/ppl_ratio:np.float64(1.0004714394453913) - rollout_corr/chi2_token:np.float64(0.0008890700992196798) - rollout_corr/chi2_seq:np.float64(1.7405338953249156) - actor/pg_loss:np.float64(-9.15073323994875e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002051412569699096) - actor/ppo_kl:np.float64(1.4041290314459332e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15001486241817474) - perf/mfu/actor:np.float64(0.13349702138291186) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(139.953125) - perf/cpu_memory_used_gb:np.float64(105.56867599487305) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:1 - critic/score/mean:0.796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006906355731189251 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006906355731189251 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:852.50390625 - response_length/max:3307.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:852.50390625 - response_length_non_aborted/max:3307.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.28125 - prompt_length/max:375.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015197880566120148 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5302176997065544) - timing_s/agent_loop/generate_sequences/max:np.float64(24.74932035431266) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.707113990400103) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(24.74932035431266) - timing_s
(TaskRunner pid=2281424) 
Training Progress:  27%|██▋       | 27/100 [25:18<1:05:42, 54.01s/it]
(TaskRunner pid=2281424) step:28 - global_seqlen/min:21975 - global_seqlen/max:58966 - global_seqlen/minmax_diff:36991 - global_seqlen/balanced_min:38635 - global_seqlen/balanced_max:38827 - global_seqlen/mean:38769.375 - actor/entropy:0.3461316227912903 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34711864590644836 - training/rollout_probs_diff_mean:0.003255465067923069 - training/rollout_probs_diff_std:0.008680691011250019 - training/rollout_actor_probs_pearson_corr:0.9994996190071106 - rollout_corr/rollout_is_mean:1.00006103515625 - rollout_corr/rollout_is_ratio_fraction_high:8.337119652424008e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.084279913106002e-05 - rollout_corr/rollout_is_max:2.25616192817688 - rollout_corr/rollout_is_min:0.17730121314525604 - rollout_corr/rollout_is_std:0.028798246756196022 - rollout_corr/rollout_is_eff_sample_size:0.9991714672176878 - rollout_corr/rollout_is_seq_mean:1.0000207424163818 - rollout_corr/rollout_is_seq_std:0.0014608351048082113 - rollout_corr/rollout_is_seq_max:1.006380558013916 - rollout_corr/rollout_is_seq_min:0.9955966472625732 - rollout_corr/rollout_is_seq_max_deviation:0.006380558013916016 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3715200861915946) - rollout_corr/training_log_ppl:np.float64(0.2959491069341311) - rollout_corr/kl:np.float64(0.0004920310259990401) - rollout_corr/k3_kl:np.float64(0.000492703045580356) - rollout_corr/rollout_ppl:np.float64(1.3707749480381608) - rollout_corr/rollout_log_ppl:np.float64(0.2954570743313525) - rollout_corr/log_ppl_diff:np.float64(0.0004920326027786359) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011141302093164995) - rollout_corr/log_ppl_diff_max:np.float64(0.00506967306137085) - rollout_corr/log_ppl_diff_min:np.float64(-0.004320159554481506) - rollout_corr/ppl_ratio:np.float64(1.0004931495059282) - rollout_corr/chi2_token:np.float64(0.0010096586775034666) - rollout_corr/chi2_seq:np.float64(1.0986658702604473) - actor/pg_loss:np.float64(-3.765069413930178e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001984241041554924) - actor/ppo_kl:np.float64(4.411382598945579e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1757737323641777) - perf/mfu/actor:np.float64(0.13821070890339499) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(139.953125) - perf/cpu_memory_used_gb:np.float64(105.6130256652832) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01265887450426817 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01265887450426817 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:937.07421875 - response_length/max:4442.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:937.07421875 - response_length_non_aborted/max:4442.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.46875 - prompt_length/max:363.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013406015932559967 - timing_s/agent_loop/generate_sequences/min:np.float64(1.040765456855297) - timing_s/agent_loop/generate_sequences/max:np.float64(31.025565603747964) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.124727715730842) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(31.025565603747964) - timing_s/agent_l
(TaskRunner pid=2281424) 
Training Progress:  28%|██▊       | 28/100 [26:14<1:05:49, 54.86s/it]
(TaskRunner pid=2281424) step:29 - global_seqlen/min:26468 - global_seqlen/max:47121 - global_seqlen/minmax_diff:20653 - global_seqlen/balanced_min:34309 - global_seqlen/balanced_max:34448 - global_seqlen/mean:34354.375 - actor/entropy:0.2958447337150574 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135512113571167 - training/rollout_probs_diff_mean:0.003082486568018794 - training/rollout_probs_diff_std:0.00864423532038927 - training/rollout_actor_probs_pearson_corr:0.9994667768478394 - rollout_corr/rollout_is_mean:0.9999562501907349 - rollout_corr/rollout_is_ratio_fraction_high:4.822275059268577e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.446682472305838e-05 - rollout_corr/rollout_is_max:2.0740530490875244 - rollout_corr/rollout_is_min:0.37482571601867676 - rollout_corr/rollout_is_std:0.027990059927105904 - rollout_corr/rollout_is_eff_sample_size:0.9992171698522248 - rollout_corr/rollout_is_seq_mean:0.9999083280563354 - rollout_corr/rollout_is_seq_std:0.0016167733119800687 - rollout_corr/rollout_is_seq_max:1.008034586906433 - rollout_corr/rollout_is_seq_min:0.993806004524231 - rollout_corr/rollout_is_seq_max_deviation:0.008034586906433105 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3131270464509726) - rollout_corr/training_log_ppl:np.float64(0.26043001232028473) - rollout_corr/kl:np.float64(0.0006687698600753578) - rollout_corr/k3_kl:np.float64(0.0005304543523152461) - rollout_corr/rollout_ppl:np.float64(1.3122303052805364) - rollout_corr/rollout_log_ppl:np.float64(0.2597612422541715) - rollout_corr/log_ppl_diff:np.float64(0.0006687700661132112) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013376595597947016) - rollout_corr/log_ppl_diff_max:np.float64(0.009146973490715027) - rollout_corr/log_ppl_diff_min:np.float64(-0.004970654845237732) - rollout_corr/ppl_ratio:np.float64(1.0006705992855132) - rollout_corr/chi2_token:np.float64(0.0007638768292963505) - rollout_corr/chi2_seq:np.float64(0.7327017430216074) - actor/pg_loss:np.float64(4.302803426980972e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002477214987948173) - actor/ppo_kl:np.float64(0.00011317849087255638) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1297485176473856) - perf/mfu/actor:np.float64(0.1281224223428748) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(139.953125) - perf/cpu_memory_used_gb:np.float64(105.53421020507812) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:1 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.04238719493150711 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.04238719493150711 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:810.04296875 - response_length/max:3813.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:810.04296875 - response_length_non_aborted/max:3813.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.53125 - prompt_length/max:375.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014430470764636993 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3452266938984394) - timing_s/agent_loop/generate_sequences/max:np.float64(27.088402163237333) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.822816854313714) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.088402163237333) - timing
(TaskRunner pid=2281424) 
Training Progress:  29%|██▉       | 29/100 [27:06<1:03:53, 53.99s/it]
(TaskRunner pid=2281424) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30}
(TaskRunner pid=2281424) validation generation end
(TaskRunner pid=2281424) [prompt] system
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2281424) <reasoning>
(TaskRunner pid=2281424) ...
(TaskRunner pid=2281424) </reasoning>
(TaskRunner pid=2281424) <answer>
(TaskRunner pid=2281424) ...
(TaskRunner pid=2281424) </answer>
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2281424) <answer>
(TaskRunner pid=2281424) A
(TaskRunner pid=2281424) </answer>
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) user
(TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$.
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) A: 50 pC
(TaskRunner pid=2281424) B: 56 pC
(TaskRunner pid=2281424) C: 60 pC
(TaskRunner pid=2281424) D: 64 pC
(TaskRunner pid=2281424) Please reason step by step.
(TaskRunner pid=2281424) assistant
(TaskRunner pid=2281424) <think>
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) </think>
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) [response] <reasoning>
(TaskRunner pid=2281424) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is:
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) E = k \frac{q}{r^2}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) We can solve for $ q $:
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q = \frac{E r^2}{k}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) Substituting the values:
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q = \frac{(2.0 \, \mathrm{N/C}) \cdot (0.50 \, \mathrm{m})^2}{8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2}}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q = \frac{(2.0) \cdot (0.25)}{8.99 \times 10^9}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q = \frac{0.50}{8.99 \times 10^9}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) Now, convert coulombs to picocoulombs (1 pC = $1 \times 10^{-12} \, \mathrm{C}$):
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) The closest option is:
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) <answer>
(TaskRunner pid=2281424) B
(TaskRunner pid=2281424) </answer>
(TaskRunner pid=2281424) [ground_truth] B
(TaskRunner pid=2281424) [score] 1.0
(TaskRunner pid=2281424) [acc] 1.0
(TaskRunner pid=2281424) [pred] B
(TaskRunner pid=2281424) [incorrect_format] 1
(TaskRunner pid=2281424) [feedback] 
(TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30
(TaskRunner pid=2281424) step:30 - global_seqlen/min:21152 - global_seqlen/max:43378 - global_seqlen/minmax_diff:22226 - global_seqlen/balanced_min:32690 - global_seqlen/balanced_max:36508 - global_seqlen/mean:33253.625 - actor/entropy:0.3238883912563324 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3535161018371582 - training/rollout_probs_diff_mean:0.0032308220397681 - training/rollout_probs_diff_std:0.00875203125178814 - training/rollout_actor_probs_pearson_corr:0.9993578195571899 - rollout_corr/rollout_is_mean:1.0000488758087158 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9363911151885986 - rollout_corr/rollout_is_min:0.5356025099754333 - rollout_corr/rollout_is_std:0.028555061668157578 - rollout_corr/rollout_is_eff_sample_size:0.9991853917962915 - rollout_corr/rollout_is_seq_mean:1.0000839233398438 - rollout_corr/rollout_is_seq_std:0.0014182842569425702 - rollout_corr/rollout_is_seq_max:1.00834321975708 - rollout_corr/rollout_is_seq_min:0.9960029721260071 - rollout_corr/rollout_is_seq_max_deviation:0.008343219757080078 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3260084195062518) - rollout_corr/training_log_ppl:np.float64(0.26798260471878166) - rollout_corr/kl:np.float64(0.00034142893949073994) - rollout_corr/k3_kl:np.float64(0.00046080480333898777) - rollout_corr/rollout_ppl:np.float64(1.3255186867900193) - rollout_corr/rollout_log_ppl:np.float64(0.2676411745214864) - rollout_corr/log_ppl_diff:np.float64(0.00034143019729526713) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010471087371115573) - rollout_corr/log_ppl_diff_max:np.float64(0.0044901371002197266) - rollout_corr/log_ppl_diff_min:np.float64(-0.006970256567001343) - rollout_corr/ppl_ratio:np.float64(1.0003424619790167) - rollout_corr/chi2_token:np.float64(0.0011780557688325644) - rollout_corr/chi2_seq:np.float64(9.059563745046034) - actor/pg_loss:np.float64(-7.204711437225342e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002318803169032435) - actor/ppo_kl:np.float64(-2.2715349961899278e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1330014280974865) - perf/mfu/actor:np.float64(0.11964730091498024) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.54463195800781) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6046875) - val-aux/sciknoweval/reward/std@16:np.float64(0.19331331768802099) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6854) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1699013098775457) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5269125000000001) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1573208332576977) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6056250000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.19436455511085113) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.75095) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.13208485427187178) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4644124999999999) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11165691249499834) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6343625) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.16176855495856937) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8039375) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0971781630054231) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.42205000000000004) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07114364007214655) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6537749999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.12550218875311617) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.84275) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.0
(TaskRunner pid=2281424) 
Training Progress:  30%|███       | 30/100 [31:17<2:11:52, 113.03s/it]
(TaskRunner pid=2281424) step:31 - global_seqlen/min:26976 - global_seqlen/max:51365 - global_seqlen/minmax_diff:24389 - global_seqlen/balanced_min:37711 - global_seqlen/balanced_max:37821 - global_seqlen/mean:37765.0 - actor/entropy:0.33241793513298035 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.301638126373291 - training/rollout_probs_diff_mean:0.003111757803708315 - training/rollout_probs_diff_std:0.008627519011497498 - training/rollout_actor_probs_pearson_corr:0.9994876980781555 - rollout_corr/rollout_is_mean:0.9999532699584961 - rollout_corr/rollout_is_ratio_fraction_high:8.658907972858287e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.2988362868782133e-05 - rollout_corr/rollout_is_max:2.167019844055176 - rollout_corr/rollout_is_min:0.3715561032295227 - rollout_corr/rollout_is_std:0.028119659051299095 - rollout_corr/rollout_is_eff_sample_size:0.9992096714770495 - rollout_corr/rollout_is_seq_mean:0.9999525547027588 - rollout_corr/rollout_is_seq_std:0.0012215663446113467 - rollout_corr/rollout_is_seq_max:1.004585862159729 - rollout_corr/rollout_is_seq_min:0.9942855834960938 - rollout_corr/rollout_is_seq_max_deviation:0.00571441650390625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3133017774671316) - rollout_corr/training_log_ppl:np.float64(0.25711654864426237) - rollout_corr/kl:np.float64(0.0004556128830559558) - rollout_corr/k3_kl:np.float64(0.00039641217307462284) - rollout_corr/rollout_ppl:np.float64(1.3126854803413153) - rollout_corr/rollout_log_ppl:np.float64(0.25666093576001003) - rollout_corr/log_ppl_diff:np.float64(0.00045561288425233215) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010043695365311578) - rollout_corr/log_ppl_diff_max:np.float64(0.006338976323604584) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035679787397384644) - rollout_corr/ppl_ratio:np.float64(1.0004564973060042) - rollout_corr/chi2_token:np.float64(0.0006840897258371115) - rollout_corr/chi2_seq:np.float64(0.9153507652226835) - actor/pg_loss:np.float64(1.1453521437942982e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001992799257095612) - actor/ppo_kl:np.float64(2.699381930071354e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.14760523661971092) - perf/mfu/actor:np.float64(0.13769359300581646) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.38329315185547) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:1 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009976144880056381 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009976144880056381 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:902.25 - response_length/max:3456.0 - response_length/min:174.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:902.25 - response_length_non_aborted/max:3456.0 - response_length_non_aborted/min:174.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.90625 - prompt_length/max:364.0 - prompt_length/min:188.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00019257515668869019 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1455713659524918) - timing_s/agent_loop/generate_sequences/max:np.float64(25.956687213853) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.312753760583291) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(25.956687213853) - timing_s/agent_l
(TaskRunner pid=2281424) 
Training Progress:  31%|███       | 31/100 [32:08<1:48:34, 94.42s/it] 
(TaskRunner pid=2281424) step:32 - global_seqlen/min:17749 - global_seqlen/max:40522 - global_seqlen/minmax_diff:22773 - global_seqlen/balanced_min:29749 - global_seqlen/balanced_max:30114 - global_seqlen/mean:29860.25 - actor/entropy:0.3001790940761566 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2931319773197174 - training/rollout_probs_diff_mean:0.0032425629906356335 - training/rollout_probs_diff_std:0.009085338562726974 - training/rollout_actor_probs_pearson_corr:0.9993113279342651 - rollout_corr/rollout_is_mean:1.0000972747802734 - rollout_corr/rollout_is_ratio_fraction_high:5.759240593761206e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.3036962375044823e-05 - rollout_corr/rollout_is_max:2.2561657428741455 - rollout_corr/rollout_is_min:0.11440415680408478 - rollout_corr/rollout_is_std:0.028905672952532768 - rollout_corr/rollout_is_eff_sample_size:0.999165278640662 - rollout_corr/rollout_is_seq_mean:1.0000715255737305 - rollout_corr/rollout_is_seq_std:0.001591670559719205 - rollout_corr/rollout_is_seq_max:1.0057041645050049 - rollout_corr/rollout_is_seq_min:0.9924244284629822 - rollout_corr/rollout_is_seq_max_deviation:0.007575571537017822 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3085072333924472) - rollout_corr/training_log_ppl:np.float64(0.25189579716243315) - rollout_corr/kl:np.float64(0.0004256223451228891) - rollout_corr/k3_kl:np.float64(0.0005207587888378384) - rollout_corr/rollout_ppl:np.float64(1.307915877085179) - rollout_corr/rollout_log_ppl:np.float64(0.25147017439303454) - rollout_corr/log_ppl_diff:np.float64(0.00042562276939861476) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012707264686468989) - rollout_corr/log_ppl_diff_max:np.float64(0.008575737476348877) - rollout_corr/log_ppl_diff_min:np.float64(-0.005954384803771973) - rollout_corr/ppl_ratio:np.float64(1.0004271555226296) - rollout_corr/chi2_token:np.float64(0.0012784942518919706) - rollout_corr/chi2_seq:np.float64(1.3941792568657547) - actor/pg_loss:np.float64(-4.464993253350258e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011340672926962725) - actor/ppo_kl:np.float64(3.421145366999667e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.10089067462831736) - perf/mfu/actor:np.float64(0.11507569937509035) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.87258911132812) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:1 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02760116197168827 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02760116197168827 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:678.2578125 - response_length/max:3951.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:678.2578125 - response_length_non_aborted/max:3951.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.875 - prompt_length/max:375.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010407157242298126 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1292062569409609) - timing_s/agent_loop/generate_sequences/max:np.float64(27.004830926656723) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.73606214641768) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.004830926656723) - timin
(TaskRunner pid=2281424) 
Training Progress:  32%|███▏      | 32/100 [33:00<1:32:25, 81.55s/it]
(TaskRunner pid=2281424) step:33 - global_seqlen/min:27211 - global_seqlen/max:50719 - global_seqlen/minmax_diff:23508 - global_seqlen/balanced_min:36245 - global_seqlen/balanced_max:36302 - global_seqlen/mean:36278.125 - actor/entropy:0.30930498242378235 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35351723432540894 - training/rollout_probs_diff_mean:0.0030934372916817665 - training/rollout_probs_diff_std:0.00848071463406086 - training/rollout_actor_probs_pearson_corr:0.9993887543678284 - rollout_corr/rollout_is_mean:0.9999690055847168 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.3896545169700403e-05 - rollout_corr/rollout_is_max:1.8591357469558716 - rollout_corr/rollout_is_min:0.3342134654521942 - rollout_corr/rollout_is_std:0.027681713923811913 - rollout_corr/rollout_is_eff_sample_size:0.999234190391299 - rollout_corr/rollout_is_seq_mean:0.9999369382858276 - rollout_corr/rollout_is_seq_std:0.0011744012590497732 - rollout_corr/rollout_is_seq_max:1.003006100654602 - rollout_corr/rollout_is_seq_min:0.9960953593254089 - rollout_corr/rollout_is_seq_max_deviation:0.0039046406745910645 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.354571030009538) - rollout_corr/training_log_ppl:np.float64(0.2829173822829034) - rollout_corr/kl:np.float64(0.0005067846795761) - rollout_corr/k3_kl:np.float64(0.00047523474137278754) - rollout_corr/rollout_ppl:np.float64(1.353815943468362) - rollout_corr/rollout_log_ppl:np.float64(0.28241059606079943) - rollout_corr/log_ppl_diff:np.float64(0.0005067862221039832) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010225145379081368) - rollout_corr/log_ppl_diff_max:np.float64(0.005544401705265045) - rollout_corr/log_ppl_diff_min:np.float64(-0.003694787621498108) - rollout_corr/ppl_ratio:np.float64(1.000507768476382) - rollout_corr/chi2_token:np.float64(0.0008780690841376781) - rollout_corr/chi2_seq:np.float64(0.740443660877645) - actor/pg_loss:np.float64(-8.333136793226004e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002747284138422401) - actor/ppo_kl:np.float64(2.7713305339016614e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16628609038889408) - perf/mfu/actor:np.float64(0.1349622504868255) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.00208282470703) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:1 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004228602629154921 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004228602629154921 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:843.28515625 - response_length/max:3164.0 - response_length/min:142.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:843.28515625 - response_length_non_aborted/max:3164.0 - response_length_non_aborted/min:142.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.40625 - prompt_length/max:437.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013750791549682617 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8266812413930893) - timing_s/agent_loop/generate_sequences/max:np.float64(24.079156171530485) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.841072585782968) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(24.079156171530485) - timing_s/agent_loop/
(TaskRunner pid=2281424) 
Training Progress:  33%|███▎      | 33/100 [33:48<1:20:06, 71.74s/it]
(TaskRunner pid=2281424) step:34 - global_seqlen/min:25666 - global_seqlen/max:73494 - global_seqlen/minmax_diff:47828 - global_seqlen/balanced_min:35473 - global_seqlen/balanced_max:36595 - global_seqlen/mean:35820.75 - actor/entropy:0.35148361325263977 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45545026659965515 - training/rollout_probs_diff_mean:0.0031844459008425474 - training/rollout_probs_diff_std:0.00856610108166933 - training/rollout_actor_probs_pearson_corr:0.9994761347770691 - rollout_corr/rollout_is_mean:0.9999700784683228 - rollout_corr/rollout_is_ratio_fraction_high:9.077787581190933e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.8155575162381865e-05 - rollout_corr/rollout_is_max:2.0403316020965576 - rollout_corr/rollout_is_min:0.33807873725891113 - rollout_corr/rollout_is_std:0.02805599570274353 - rollout_corr/rollout_is_eff_sample_size:0.9992133611396591 - rollout_corr/rollout_is_seq_mean:0.9998204112052917 - rollout_corr/rollout_is_seq_std:0.0015267465496435761 - rollout_corr/rollout_is_seq_max:1.003962516784668 - rollout_corr/rollout_is_seq_min:0.9919791221618652 - rollout_corr/rollout_is_seq_max_deviation:0.008020877838134766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.309415325988084) - rollout_corr/training_log_ppl:np.float64(0.25629871332785115) - rollout_corr/kl:np.float64(0.0007180621188593861) - rollout_corr/k3_kl:np.float64(0.000561066810490729) - rollout_corr/rollout_ppl:np.float64(1.3084843065589666) - rollout_corr/rollout_log_ppl:np.float64(0.2555806517775636) - rollout_corr/log_ppl_diff:np.float64(0.0007180615502875298) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012585615331772715) - rollout_corr/log_ppl_diff_max:np.float64(0.009126394987106323) - rollout_corr/log_ppl_diff_min:np.float64(-0.0031400546431541443) - rollout_corr/ppl_ratio:np.float64(1.0007198799867183) - rollout_corr/chi2_token:np.float64(0.0008239294402301311) - rollout_corr/chi2_seq:np.float64(1.3956339918076992) - actor/pg_loss:np.float64(6.344064604490995e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036903620963357753) - actor/ppo_kl:np.float64(0.00010281039390358515) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1495772898197174) - perf/mfu/actor:np.float64(0.12881182116660295) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.90487289428711) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:1 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0018949881196022034 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0018949881196022034 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:860.6171875 - response_length/max:6530.0 - response_length/min:121.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:860.6171875 - response_length_non_aborted/max:6530.0 - response_length_non_aborted/min:121.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.78125 - prompt_length/max:382.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010554678738117218 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5495978090912104) - timing_s/agent_loop/generate_sequences/max:np.float64(42.11380794644356) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.242849451620714) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(42.11380794644356) - timing
(TaskRunner pid=2281424) 
Training Progress:  34%|███▍      | 34/100 [34:56<1:17:37, 70.56s/it]
(TaskRunner pid=2281424) step:35 - global_seqlen/min:22711 - global_seqlen/max:52151 - global_seqlen/minmax_diff:29440 - global_seqlen/balanced_min:33570 - global_seqlen/balanced_max:34828 - global_seqlen/mean:33923.125 - actor/entropy:0.356091171503067 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23818081617355347 - training/rollout_probs_diff_mean:0.003347822232171893 - training/rollout_probs_diff_std:0.008754725567996502 - training/rollout_actor_probs_pearson_corr:0.9993891716003418 - rollout_corr/rollout_is_mean:0.999956488609314 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.74996237346204e-06 - rollout_corr/rollout_is_max:1.6463863849639893 - rollout_corr/rollout_is_min:0.14987757802009583 - rollout_corr/rollout_is_std:0.02889123558998108 - rollout_corr/rollout_is_eff_sample_size:0.9991658736928141 - rollout_corr/rollout_is_seq_mean:0.9999445676803589 - rollout_corr/rollout_is_seq_std:0.0016311032231897116 - rollout_corr/rollout_is_seq_max:1.0061169862747192 - rollout_corr/rollout_is_seq_min:0.994307279586792 - rollout_corr/rollout_is_seq_max_deviation:0.006116986274719238 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3848898322321475) - rollout_corr/training_log_ppl:np.float64(0.3058403903269209) - rollout_corr/kl:np.float64(0.0004393863629239547) - rollout_corr/k3_kl:np.float64(0.0005089315998816346) - rollout_corr/rollout_ppl:np.float64(1.3842959650792181) - rollout_corr/rollout_log_ppl:np.float64(0.3054010035120882) - rollout_corr/log_ppl_diff:np.float64(0.0004393868148326874) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012084643822163343) - rollout_corr/log_ppl_diff_max:np.float64(0.006951093673706055) - rollout_corr/log_ppl_diff_min:np.float64(-0.005402863025665283) - rollout_corr/ppl_ratio:np.float64(1.0004409453831613) - rollout_corr/chi2_token:np.float64(0.001150362892076373) - rollout_corr/chi2_seq:np.float64(1.6086886166594923) - actor/pg_loss:np.float64(2.0964303985238075e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002854640780469708) - actor/ppo_kl:np.float64(-8.599662149322285e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18192359805107117) - perf/mfu/actor:np.float64(0.12649445809002435) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.95371627807617) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:1 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00438565481454134 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00438565481454134 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:801.28515625 - response_length/max:5012.0 - response_length/min:130.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:801.28515625 - response_length_non_aborted/max:5012.0 - response_length_non_aborted/min:130.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.8125 - prompt_length/max:365.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.070756793022156e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6557252500206232) - timing_s/agent_loop/generate_sequences/max:np.float64(33.535721046850085) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.11258227860526) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(33.535721046850085) - timing_s/agent_loop/slowe
(TaskRunner pid=2281424) 
Training Progress:  35%|███▌      | 35/100 [35:55<1:12:28, 66.89s/it]
(TaskRunner pid=2281424) step:36 - global_seqlen/min:33717 - global_seqlen/max:49536 - global_seqlen/minmax_diff:15819 - global_seqlen/balanced_min:43612 - global_seqlen/balanced_max:43672 - global_seqlen/mean:43658.375 - actor/entropy:0.362861692905426 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8421130180358887 - training/rollout_probs_diff_mean:0.003083376446738839 - training/rollout_probs_diff_std:0.00820560660213232 - training/rollout_actor_probs_pearson_corr:0.9995745420455933 - rollout_corr/rollout_is_mean:0.9999458193778992 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.136867225199239e-06 - rollout_corr/rollout_is_max:1.5991979837417603 - rollout_corr/rollout_is_min:0.010655050165951252 - rollout_corr/rollout_is_std:0.027175385504961014 - rollout_corr/rollout_is_eff_sample_size:0.9992619243999026 - rollout_corr/rollout_is_seq_mean:0.999880313873291 - rollout_corr/rollout_is_seq_std:0.0011923309648409486 - rollout_corr/rollout_is_seq_max:1.0057610273361206 - rollout_corr/rollout_is_seq_min:0.9923911094665527 - rollout_corr/rollout_is_seq_max_deviation:0.007608890533447266 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3939749002456665) - rollout_corr/training_log_ppl:np.float64(0.3143607703241287) - rollout_corr/kl:np.float64(0.0004875935232391271) - rollout_corr/k3_kl:np.float64(0.00043940715849544176) - rollout_corr/rollout_ppl:np.float64(1.3933153660036623) - rollout_corr/rollout_log_ppl:np.float64(0.3138731736544287) - rollout_corr/log_ppl_diff:np.float64(0.0004875966696999967) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008989509078674018) - rollout_corr/log_ppl_diff_max:np.float64(0.008858025074005127) - rollout_corr/log_ppl_diff_min:np.float64(-0.003800034523010254) - rollout_corr/ppl_ratio:np.float64(1.0004885424859822) - rollout_corr/chi2_token:np.float64(0.0007559196092188358) - rollout_corr/chi2_seq:np.float64(0.28486403776332736) - actor/pg_loss:np.float64(4.073767922818661e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001381784221621274) - actor/ppo_kl:np.float64(-3.878047037275678e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11497985571622849) - perf/mfu/actor:np.float64(0.15230643812180578) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.80973815917969) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:1 - critic/score/mean:0.578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006595357321202755 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006595357321202755 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1094.66796875 - response_length/max:3514.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1094.66796875 - response_length_non_aborted/max:3514.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.65625 - prompt_length/max:375.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7200207114219666e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3092289455235004) - timing_s/agent_loop/generate_sequences/max:np.float64(27.297075459733605) - timing_s/agent_loop/generate_sequences/mean:np.float64(11.346446073104744) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.297075459733605) - timing_s/agent_lo
(TaskRunner pid=2281424) 
Training Progress:  36%|███▌      | 36/100 [36:49<1:07:18, 63.10s/it]
(TaskRunner pid=2281424) step:37 - global_seqlen/min:34704 - global_seqlen/max:44098 - global_seqlen/minmax_diff:9394 - global_seqlen/balanced_min:39539 - global_seqlen/balanced_max:39624 - global_seqlen/mean:39597.875 - actor/entropy:0.35228487849235535 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2413027286529541 - training/rollout_probs_diff_mean:0.003145897062495351 - training/rollout_probs_diff_std:0.0083078658208251 - training/rollout_actor_probs_pearson_corr:0.9995119571685791 - rollout_corr/rollout_is_mean:0.9999536275863647 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.998736247012857e-06 - rollout_corr/rollout_is_max:1.6415951251983643 - rollout_corr/rollout_is_min:0.39738908410072327 - rollout_corr/rollout_is_std:0.027896204963326454 - rollout_corr/rollout_is_eff_sample_size:0.9992224068794121 - rollout_corr/rollout_is_seq_mean:0.9999120235443115 - rollout_corr/rollout_is_seq_std:0.0014193912502378225 - rollout_corr/rollout_is_seq_max:1.0059196949005127 - rollout_corr/rollout_is_seq_min:0.9947477579116821 - rollout_corr/rollout_is_seq_max_deviation:0.005919694900512695 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3658450348302722) - rollout_corr/training_log_ppl:np.float64(0.29802232861402445) - rollout_corr/kl:np.float64(0.0005081376050801722) - rollout_corr/k3_kl:np.float64(0.00047723877818839355) - rollout_corr/rollout_ppl:np.float64(1.365188850555569) - rollout_corr/rollout_log_ppl:np.float64(0.2975141915230779) - rollout_corr/log_ppl_diff:np.float64(0.0005081370909465477) - rollout_corr/log_ppl_abs_diff:np.float64(0.001064729891368188) - rollout_corr/log_ppl_diff_max:np.float64(0.005850955843925476) - rollout_corr/log_ppl_diff_min:np.float64(-0.00405728816986084) - rollout_corr/ppl_ratio:np.float64(1.000509355450049) - rollout_corr/chi2_token:np.float64(0.0009017251431941986) - rollout_corr/chi2_seq:np.float64(1.061360603896901) - actor/pg_loss:np.float64(6.26088585704565e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021849221991487866) - actor/ppo_kl:np.float64(-2.4668701797736503e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1671565007418394) - perf/mfu/actor:np.float64(0.14026788981271016) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.84344863891602) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011376905255019665 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011376905255019665 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:976.87109375 - response_length/max:3632.0 - response_length/min:123.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:976.87109375 - response_length_non_aborted/max:3632.0 - response_length_non_aborted/min:123.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:260.5625 - prompt_length/max:350.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.6365742087364197e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.545783119276166) - timing_s/agent_loop/generate_sequences/max:np.float64(27.446757026016712) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.701898429557332) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.446757026016712) - timing_s/agent_loop/slowe
(TaskRunner pid=2281424) 
Training Progress:  37%|███▋      | 37/100 [37:42<1:03:03, 60.05s/it]
(TaskRunner pid=2281424) step:38 - global_seqlen/min:25876 - global_seqlen/max:56353 - global_seqlen/minmax_diff:30477 - global_seqlen/balanced_min:36571 - global_seqlen/balanced_max:36714 - global_seqlen/mean:36659.625 - actor/entropy:0.32807788252830505 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9003579616546631 - training/rollout_probs_diff_mean:0.002977442229166627 - training/rollout_probs_diff_std:0.008599979802966118 - training/rollout_actor_probs_pearson_corr:0.9994491338729858 - rollout_corr/rollout_is_mean:0.9999948143959045 - rollout_corr/rollout_is_ratio_fraction_high:4.4448197513702326e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.3334459254110698e-05 - rollout_corr/rollout_is_max:3.384216547012329 - rollout_corr/rollout_is_min:0.005166077055037022 - rollout_corr/rollout_is_std:0.027199501171708107 - rollout_corr/rollout_is_eff_sample_size:0.999260734067485 - rollout_corr/rollout_is_seq_mean:0.9999963641166687 - rollout_corr/rollout_is_seq_std:0.0012283420655876398 - rollout_corr/rollout_is_seq_max:1.00580632686615 - rollout_corr/rollout_is_seq_min:0.9936442375183105 - rollout_corr/rollout_is_seq_max_deviation:0.006355762481689453 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3002572190016508) - rollout_corr/training_log_ppl:np.float64(0.24828266966505907) - rollout_corr/kl:np.float64(0.0003652476746376365) - rollout_corr/k3_kl:np.float64(0.00043928875018650615) - rollout_corr/rollout_ppl:np.float64(1.2997558075003326) - rollout_corr/rollout_log_ppl:np.float64(0.2479174185282318) - rollout_corr/log_ppl_diff:np.float64(0.0003652511368272826) - rollout_corr/log_ppl_abs_diff:np.float64(0.000950878209550865) - rollout_corr/log_ppl_diff_max:np.float64(0.008495032787322998) - rollout_corr/log_ppl_diff_min:np.float64(-0.004545837640762329) - rollout_corr/ppl_ratio:np.float64(1.0003662216477096) - rollout_corr/chi2_token:np.float64(0.0010224031284451485) - rollout_corr/chi2_seq:np.float64(0.804062022594735) - actor/pg_loss:np.float64(-4.1241059079766273e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001981328330202814) - actor/ppo_kl:np.float64(-5.4830987617826565e-05) - actor/pg_clipfrac_lower:np.float64(3.98596921513672e-06) - actor/grad_norm:np.float64(0.12819637544453144) - perf/mfu/actor:np.float64(0.13466560382170992) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.90673160552979) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:1 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004230912774801254 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004230912774801254 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:878.83203125 - response_length/max:3854.0 - response_length/min:123.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:878.83203125 - response_length_non_aborted/max:3854.0 - response_length_non_aborted/min:123.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.78125 - prompt_length/max:328.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014106743037700653 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5907990671694279) - timing_s/agent_loop/generate_sequences/max:np.float64(27.61575059592724) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.814689949009335) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2
(TaskRunner pid=2281424) 
Training Progress:  38%|███▊      | 38/100 [38:35<1:00:02, 58.10s/it]
(TaskRunner pid=2281424) step:39 - global_seqlen/min:19213 - global_seqlen/max:40355 - global_seqlen/minmax_diff:21142 - global_seqlen/balanced_min:28562 - global_seqlen/balanced_max:29340 - global_seqlen/mean:28757.375 - actor/entropy:0.3156418800354004 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22961103916168213 - training/rollout_probs_diff_mean:0.003309106919914484 - training/rollout_probs_diff_std:0.00906699150800705 - training/rollout_actor_probs_pearson_corr:0.9993242025375366 - rollout_corr/rollout_is_mean:1.0000478029251099 - rollout_corr/rollout_is_ratio_fraction_high:1.219846944877645e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.099234724388225e-06 - rollout_corr/rollout_is_max:3.2739274501800537 - rollout_corr/rollout_is_min:0.49741330742836 - rollout_corr/rollout_is_std:0.029006537050008774 - rollout_corr/rollout_is_eff_sample_size:0.9991593281581235 - rollout_corr/rollout_is_seq_mean:1.0000423192977905 - rollout_corr/rollout_is_seq_std:0.0014898530207574368 - rollout_corr/rollout_is_seq_max:1.010595679283142 - rollout_corr/rollout_is_seq_min:0.9943899512290955 - rollout_corr/rollout_is_seq_max_deviation:0.01059567928314209 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3278274578042328) - rollout_corr/training_log_ppl:np.float64(0.26448297980823554) - rollout_corr/kl:np.float64(0.00044818446750127805) - rollout_corr/k3_kl:np.float64(0.00045055161507434605) - rollout_corr/rollout_ppl:np.float64(1.3272340912371874) - rollout_corr/rollout_log_ppl:np.float64(0.264034794352483) - rollout_corr/log_ppl_diff:np.float64(0.00044818545575253665) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011384486278984696) - rollout_corr/log_ppl_diff_max:np.float64(0.006218463182449341) - rollout_corr/log_ppl_diff_min:np.float64(-0.00863182544708252) - rollout_corr/ppl_ratio:np.float64(1.0004495589528233) - rollout_corr/chi2_token:np.float64(0.0009194693993777037) - rollout_corr/chi2_seq:np.float64(0.4609958524815738) - actor/pg_loss:np.float64(1.4589284546673298e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.937741061894485e-05) - actor/ppo_kl:np.float64(4.479127906620306e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.10549908224493265) - perf/mfu/actor:np.float64(0.11212622068994285) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.87326049804688) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:1 - critic/score/mean:0.87890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.87890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009453050792217255 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009453050792217255 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:640.44921875 - response_length/max:3306.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:640.44921875 - response_length_non_aborted/max:3306.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.21875 - prompt_length/max:326.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001562461256980896 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4932099115103483) - timing_s/agent_loop/generate_sequences/max:np.float64(23.016741577535868) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.735508493344241) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.016741577535868) - timi
(TaskRunner pid=2281424) 
Training Progress:  39%|███▉      | 39/100 [39:23<55:49, 54.90s/it]  
(TaskRunner pid=2281424) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40}
(TaskRunner pid=2281424) validation generation end
(TaskRunner pid=2281424) [prompt] system
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2281424) <reasoning>
(TaskRunner pid=2281424) ...
(TaskRunner pid=2281424) </reasoning>
(TaskRunner pid=2281424) <answer>
(TaskRunner pid=2281424) ...
(TaskRunner pid=2281424) </answer>
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2281424) <answer>
(TaskRunner pid=2281424) A
(TaskRunner pid=2281424) </answer>
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) user
(TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$.
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) A: 50 pC
(TaskRunner pid=2281424) B: 56 pC
(TaskRunner pid=2281424) C: 60 pC
(TaskRunner pid=2281424) D: 64 pC
(TaskRunner pid=2281424) Please reason step by step.
(TaskRunner pid=2281424) assistant
(TaskRunner pid=2281424) <think>
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) </think>
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) [response] To find the magnitude of the particle's charge, we can use the formula for the electric field due to a point charge:
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) E = k \frac{q}{r^2}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) Where:
(TaskRunner pid=2281424) - $ E = 2.0 \, \mathrm{N/C} $
(TaskRunner pid=2281424) - $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $
(TaskRunner pid=2281424) - $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) We need to solve for $ q $:
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q = \frac{E \cdot r^2}{k}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) Now, substitute the known values:
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q = \frac{0.5}{8.99 \times 10^9}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) Now, convert coulombs to picocoulombs ($1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C}$):
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) So, the closest answer is:
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) <answer>
(TaskRunner pid=2281424) B
(TaskRunner pid=2281424) </answer>
(TaskRunner pid=2281424) [ground_truth] B
(TaskRunner pid=2281424) [score] 1.0
(TaskRunner pid=2281424) [acc] 1.0
(TaskRunner pid=2281424) [pred] B
(TaskRunner pid=2281424) [incorrect_format] 1
(TaskRunner pid=2281424) [feedback] 
(TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40
(TaskRunner pid=2281424) step:40 - global_seqlen/min:25284 - global_seqlen/max:51779 - global_seqlen/minmax_diff:26495 - global_seqlen/balanced_min:36707 - global_seqlen/balanced_max:38750 - global_seqlen/mean:36970.5 - actor/entropy:0.2809111177921295 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35805636644363403 - training/rollout_probs_diff_mean:0.0029524413403123617 - training/rollout_probs_diff_std:0.008548099547624588 - training/rollout_actor_probs_pearson_corr:0.9994715452194214 - rollout_corr/rollout_is_mean:0.9999876022338867 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.368109330243897e-06 - rollout_corr/rollout_is_max:1.9496772289276123 - rollout_corr/rollout_is_min:0.451233834028244 - rollout_corr/rollout_is_std:0.026926398277282715 - rollout_corr/rollout_is_eff_sample_size:0.999275494389906 - rollout_corr/rollout_is_seq_mean:0.9999891519546509 - rollout_corr/rollout_is_seq_std:0.0013239040272310376 - rollout_corr/rollout_is_seq_max:1.0059114694595337 - rollout_corr/rollout_is_seq_min:0.9929170608520508 - rollout_corr/rollout_is_seq_max_deviation:0.007082939147949219 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2970258207060397) - rollout_corr/training_log_ppl:np.float64(0.24729529570322484) - rollout_corr/kl:np.float64(0.00048714316951858905) - rollout_corr/k3_kl:np.float64(0.00042656138907659624) - rollout_corr/rollout_ppl:np.float64(1.2963653528131545) - rollout_corr/rollout_log_ppl:np.float64(0.24680815133615397) - rollout_corr/log_ppl_diff:np.float64(0.0004871443670708686) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010458430915605277) - rollout_corr/log_ppl_diff_max:np.float64(0.007907375693321228) - rollout_corr/log_ppl_diff_min:np.float64(-0.00467248260974884) - rollout_corr/ppl_ratio:np.float64(1.0004882379435003) - rollout_corr/chi2_token:np.float64(0.0007523114327341318) - rollout_corr/chi2_seq:np.float64(0.4919047087896615) - actor/pg_loss:np.float64(0.00013426528312265873) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.462851380703796e-05) - actor/ppo_kl:np.float64(9.811125503489393e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.10963228065520525) - perf/mfu/actor:np.float64(0.13216087132668086) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.88793182373047) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.62109375) - val-aux/sciknoweval/reward/std@16:np.float64(0.20978195620494514) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7098125) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.19512127996631243) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5332000000000001) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.16365501611769506) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.62185) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.209297903919098) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7927750000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.15226235422617695) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.47158749999999994) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10047297589092027) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6512499999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1840588452797897) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8536250000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09931525131049548) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.43674999999999997) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.05502894877052068) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6747375) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.15323121047661067) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8911) - val-aux/sciknoweval/r
(TaskRunner pid=2281424) 
Training Progress:  40%|████      | 40/100 [43:20<1:49:42, 109.70s/it]
(TaskRunner pid=2281424) step:41 - global_seqlen/min:29658 - global_seqlen/max:51620 - global_seqlen/minmax_diff:21962 - global_seqlen/balanced_min:38538 - global_seqlen/balanced_max:38782 - global_seqlen/mean:38637.75 - actor/entropy:0.33072540163993835 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35299545526504517 - training/rollout_probs_diff_mean:0.0031244975980371237 - training/rollout_probs_diff_std:0.008380025625228882 - training/rollout_actor_probs_pearson_corr:0.9994889497756958 - rollout_corr/rollout_is_mean:1.000058650970459 - rollout_corr/rollout_is_ratio_fraction_high:1.260218232346233e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:4.29278564453125 - rollout_corr/rollout_is_min:0.55013507604599 - rollout_corr/rollout_is_std:0.027836313471198082 - rollout_corr/rollout_is_eff_sample_size:0.999225739561659 - rollout_corr/rollout_is_seq_mean:0.9999604225158691 - rollout_corr/rollout_is_seq_std:0.0012300461530685425 - rollout_corr/rollout_is_seq_max:1.006003737449646 - rollout_corr/rollout_is_seq_min:0.9945253133773804 - rollout_corr/rollout_is_seq_max_deviation:0.006003737449645996 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3499207119457424) - rollout_corr/training_log_ppl:np.float64(0.2855829408799764) - rollout_corr/kl:np.float64(0.0005269990794536383) - rollout_corr/k3_kl:np.float64(0.0004426697283577141) - rollout_corr/rollout_ppl:np.float64(1.349194892682135) - rollout_corr/rollout_log_ppl:np.float64(0.28505594066518825) - rollout_corr/log_ppl_diff:np.float64(0.0005270002147881314) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011361698125256225) - rollout_corr/log_ppl_diff_max:np.float64(0.005944006145000458) - rollout_corr/log_ppl_diff_min:np.float64(-0.005369067192077637) - rollout_corr/ppl_ratio:np.float64(1.0005282380152494) - rollout_corr/chi2_token:np.float64(0.0007350665982812643) - rollout_corr/chi2_seq:np.float64(4.942351899808273) - actor/pg_loss:np.float64(4.62154857814312e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001543072065146589) - actor/ppo_kl:np.float64(0.0001058366084052409) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12146356515586376) - perf/mfu/actor:np.float64(0.13938522742671805) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.29471588134766) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:1 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004173422697931528 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004173422697931528 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:929.8984375 - response_length/max:4125.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:929.8984375 - response_length_non_aborted/max:4125.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:361.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015160441398620605 - timing_s/agent_loop/generate_sequences/min:np.float64(1.333501910790801) - timing_s/agent_loop/generate_sequences/max:np.float64(29.424998305737972) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.613472268974874) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(29.424998305737972) - timing_s/agent_loop/slowest/
(TaskRunner pid=2281424) 
Training Progress:  41%|████      | 41/100 [44:15<1:31:40, 93.23s/it] 
(TaskRunner pid=2281424) step:42 - global_seqlen/min:34807 - global_seqlen/max:57617 - global_seqlen/minmax_diff:22810 - global_seqlen/balanced_min:46153 - global_seqlen/balanced_max:46569 - global_seqlen/mean:46354.0 - actor/entropy:0.4125242531299591 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27730005979537964 - training/rollout_probs_diff_mean:0.0032492319587618113 - training/rollout_probs_diff_std:0.00808805413544178 - training/rollout_actor_probs_pearson_corr:0.9996025562286377 - rollout_corr/rollout_is_mean:0.9999741911888123 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.913554094964638e-06 - rollout_corr/rollout_is_max:1.8443630933761597 - rollout_corr/rollout_is_min:0.3683568835258484 - rollout_corr/rollout_is_std:0.028246553614735603 - rollout_corr/rollout_is_eff_sample_size:0.9992026492912346 - rollout_corr/rollout_is_seq_mean:1.0000536441802979 - rollout_corr/rollout_is_seq_std:0.0012193896109238267 - rollout_corr/rollout_is_seq_max:1.009535312652588 - rollout_corr/rollout_is_seq_min:0.9968177080154419 - rollout_corr/rollout_is_seq_max_deviation:0.00953531265258789 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4070447161793709) - rollout_corr/training_log_ppl:np.float64(0.3194918569060974) - rollout_corr/kl:np.float64(0.00035904505688222343) - rollout_corr/k3_kl:np.float64(0.00043011571953854855) - rollout_corr/rollout_ppl:np.float64(1.4064954705536366) - rollout_corr/rollout_log_ppl:np.float64(0.3191328103421256) - rollout_corr/log_ppl_diff:np.float64(0.0003590465639717877) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009108688100241125) - rollout_corr/log_ppl_diff_max:np.float64(0.0039010122418403625) - rollout_corr/log_ppl_diff_min:np.float64(-0.003787517547607422) - rollout_corr/ppl_ratio:np.float64(1.0003597426693887) - rollout_corr/chi2_token:np.float64(0.0010020083282142878) - rollout_corr/chi2_seq:np.float64(0.939779143081978) - actor/pg_loss:np.float64(0.00010238576214760542) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001587477142379612) - actor/ppo_kl:np.float64(2.258576988367622e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11312472820281982) - perf/mfu/actor:np.float64(0.15362174541859897) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.15521240234375) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01894480176270008 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01894480176270008 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1182.09375 - response_length/max:6248.0 - response_length/min:160.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1182.09375 - response_length_non_aborted/max:6248.0 - response_length_non_aborted/min:160.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:355.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.07406210899353e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8849900215864182) - timing_s/agent_loop/generate_sequences/max:np.float64(42.688118137419224) - timing_s/agent_loop/generate_sequences/mean:np.float64(11.67329497004539) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(42.688118137419224) - timing_s/agent_loop/slowest/
(TaskRunner pid=2281424) 
Training Progress:  42%|████▏     | 42/100 [45:26<1:23:31, 86.41s/it]
(TaskRunner pid=2281424) step:43 - global_seqlen/min:27945 - global_seqlen/max:57053 - global_seqlen/minmax_diff:29108 - global_seqlen/balanced_min:46139 - global_seqlen/balanced_max:46442 - global_seqlen/mean:46394.125 - actor/entropy:0.3777778148651123 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24276472628116608 - training/rollout_probs_diff_mean:0.0031969803385436535 - training/rollout_probs_diff_std:0.008178768679499626 - training/rollout_actor_probs_pearson_corr:0.9996250867843628 - rollout_corr/rollout_is_mean:0.9999624490737915 - rollout_corr/rollout_is_ratio_fraction_high:3.310370402687113e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.620740805374226e-06 - rollout_corr/rollout_is_max:3.3981943130493164 - rollout_corr/rollout_is_min:0.40390288829803467 - rollout_corr/rollout_is_std:0.027761269360780716 - rollout_corr/rollout_is_eff_sample_size:0.9992297864199887 - rollout_corr/rollout_is_seq_mean:0.9999164938926697 - rollout_corr/rollout_is_seq_std:0.0013294965028762817 - rollout_corr/rollout_is_seq_max:1.0057275295257568 - rollout_corr/rollout_is_seq_min:0.9940543174743652 - rollout_corr/rollout_is_seq_max_deviation:0.005945682525634766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.368367766495794) - rollout_corr/training_log_ppl:np.float64(0.297560689446982) - rollout_corr/kl:np.float64(0.0004662561269233034) - rollout_corr/k3_kl:np.float64(0.00041893447971119713) - rollout_corr/rollout_ppl:np.float64(1.3677249136380851) - rollout_corr/rollout_log_ppl:np.float64(0.29709443140018266) - rollout_corr/log_ppl_diff:np.float64(0.00046625804679933935) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009343025594716892) - rollout_corr/log_ppl_diff_max:np.float64(0.006767094135284424) - rollout_corr/log_ppl_diff_min:np.float64(-0.0033786147832870483) - rollout_corr/ppl_ratio:np.float64(1.000467125326395) - rollout_corr/chi2_token:np.float64(0.0007502012886106968) - rollout_corr/chi2_seq:np.float64(0.8961137712467462) - actor/pg_loss:np.float64(7.715250831097364e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018895000766860903) - actor/ppo_kl:np.float64(-1.2429667751945317e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12724709697067738) - perf/mfu/actor:np.float64(0.15634425835350255) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.0877799987793) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:1 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015876950696110725 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015876950696110725 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:1180.00390625 - response_length/max:5656.0 - response_length/min:122.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1180.00390625 - response_length_non_aborted/max:5656.0 - response_length_non_aborted/min:122.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.8125 - prompt_length/max:365.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011191703379154205 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1672064159065485) - timing_s/agent_loop/generate_sequences/max:np.float64(40.370598992332816) - timing_s/agent_loop/generate_sequences/mean:np.float64(10.840922129631508) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(40.3705989923328
(TaskRunner pid=2281424) 
Training Progress:  43%|████▎     | 43/100 [46:33<1:16:37, 80.66s/it]
(TaskRunner pid=2281424) step:44 - global_seqlen/min:31039 - global_seqlen/max:81483 - global_seqlen/minmax_diff:50444 - global_seqlen/balanced_min:49966 - global_seqlen/balanced_max:50221 - global_seqlen/mean:50039.375 - actor/entropy:0.42505836486816406 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40381795167922974 - training/rollout_probs_diff_mean:0.0033156690187752247 - training/rollout_probs_diff_std:0.00811709649860859 - training/rollout_actor_probs_pearson_corr:0.9995504021644592 - rollout_corr/rollout_is_mean:0.9999823570251465 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.108491212828085e-06 - rollout_corr/rollout_is_max:1.868242859840393 - rollout_corr/rollout_is_min:0.16989949345588684 - rollout_corr/rollout_is_std:0.028271863237023354 - rollout_corr/rollout_is_eff_sample_size:0.9992012210621306 - rollout_corr/rollout_is_seq_mean:1.0000169277191162 - rollout_corr/rollout_is_seq_std:0.0010278340196236968 - rollout_corr/rollout_is_seq_max:1.0032761096954346 - rollout_corr/rollout_is_seq_min:0.9966418147087097 - rollout_corr/rollout_is_seq_max_deviation:0.003358185291290283 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4229408679530025) - rollout_corr/training_log_ppl:np.float64(0.3282105611579027) - rollout_corr/kl:np.float64(0.0004335679858704644) - rollout_corr/k3_kl:np.float64(0.00045822281549590116) - rollout_corr/rollout_ppl:np.float64(1.422240549698472) - rollout_corr/rollout_log_ppl:np.float64(0.3277769897831604) - rollout_corr/log_ppl_diff:np.float64(0.0004335713747423142) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009576250740792602) - rollout_corr/log_ppl_diff_max:np.float64(0.004134416580200195) - rollout_corr/log_ppl_diff_min:np.float64(-0.004089929163455963) - rollout_corr/ppl_ratio:np.float64(1.000434412388131) - rollout_corr/chi2_token:np.float64(0.0009704537224024534) - rollout_corr/chi2_seq:np.float64(1.733142590848729) - actor/pg_loss:np.float64(2.68502626568079e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020757744624688712) - actor/ppo_kl:np.float64(6.416326252889348e-05) - actor/pg_clipfrac_lower:np.float64(1.8610052165968227e-06) - actor/grad_norm:np.float64(0.14784160628914833) - perf/mfu/actor:np.float64(0.1614180488812359) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.14054107666016) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:1 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005857898388057947 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005857898388057947 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1286.57421875 - response_length/max:7615.0 - response_length/min:164.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1286.57421875 - response_length_non_aborted/max:7615.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.15625 - prompt_length/max:460.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.9996037483215332e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0687543358653784) - timing_s/agent_loop/generate_sequences/max:np.float64(51.31303711421788) - timing_s/agent_loop/generate_sequences/mean:np.float64(12.701673694929923) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(51.31303711421788) -
(TaskRunner pid=2281424) 
Training Progress:  44%|████▍     | 44/100 [47:52<1:14:55, 80.27s/it]
(TaskRunner pid=2281424) step:45 - global_seqlen/min:29259 - global_seqlen/max:67289 - global_seqlen/minmax_diff:38030 - global_seqlen/balanced_min:45428 - global_seqlen/balanced_max:45525 - global_seqlen/mean:45499.625 - actor/entropy:0.39985886216163635 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25675493478775024 - training/rollout_probs_diff_mean:0.0032300371676683426 - training/rollout_probs_diff_std:0.00817596260458231 - training/rollout_actor_probs_pearson_corr:0.9996156692504883 - rollout_corr/rollout_is_mean:0.9999929666519165 - rollout_corr/rollout_is_ratio_fraction_high:3.4166423574788496e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.4166423574788496e-06 - rollout_corr/rollout_is_max:2.063965082168579 - rollout_corr/rollout_is_min:0.2600027620792389 - rollout_corr/rollout_is_std:0.028221219778060913 - rollout_corr/rollout_is_eff_sample_size:0.9992040775244214 - rollout_corr/rollout_is_seq_mean:0.999981701374054 - rollout_corr/rollout_is_seq_std:0.0011321797501295805 - rollout_corr/rollout_is_seq_max:1.00321364402771 - rollout_corr/rollout_is_seq_min:0.9935727119445801 - rollout_corr/rollout_is_seq_max_deviation:0.006427288055419922 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4365097098052502) - rollout_corr/training_log_ppl:np.float64(0.33815656184742693) - rollout_corr/kl:np.float64(0.0006105463397099165) - rollout_corr/k3_kl:np.float64(0.0004829082062087764) - rollout_corr/rollout_ppl:np.float64(1.4356286954134703) - rollout_corr/rollout_log_ppl:np.float64(0.3375460135139292) - rollout_corr/log_ppl_diff:np.float64(0.000610548333497718) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009788531460799277) - rollout_corr/log_ppl_diff_max:np.float64(0.007325723767280579) - rollout_corr/log_ppl_diff_min:np.float64(-0.0033019185066223145) - rollout_corr/ppl_ratio:np.float64(1.0006115036085248) - rollout_corr/chi2_token:np.float64(0.0007435702718794346) - rollout_corr/chi2_seq:np.float64(2.2746842054184526) - actor/pg_loss:np.float64(7.623340934514999e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001888360293378355) - actor/ppo_kl:np.float64(0.0001864508021274247) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12979234382510185) - perf/mfu/actor:np.float64(0.15628703385179005) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.251708984375) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:2 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008445513434708118 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008445513434708118 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1143.30078125 - response_length/max:3842.0 - response_length/min:192.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1143.30078125 - response_length_non_aborted/max:3842.0 - response_length_non_aborted/min:192.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.5625 - prompt_length/max:437.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002876799553632736 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4185115825384855) - timing_s/agent_loop/generate_sequences/max:np.float64(29.632253983989358) - timing_s/agent_loop/generate_sequences/mean:np.float64(11.899439175627776) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(29.632253983989358) - 
(TaskRunner pid=2281424) 
Training Progress:  45%|████▌     | 45/100 [48:49<1:07:07, 73.23s/it]
(TaskRunner pid=2281424) step:46 - global_seqlen/min:20945 - global_seqlen/max:94884 - global_seqlen/minmax_diff:73939 - global_seqlen/balanced_min:43085 - global_seqlen/balanced_max:44324 - global_seqlen/mean:43425.25 - actor/entropy:0.3762817084789276 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5543509721755981 - training/rollout_probs_diff_mean:0.0032524620182812214 - training/rollout_probs_diff_std:0.008445695042610168 - training/rollout_actor_probs_pearson_corr:0.9995606541633606 - rollout_corr/rollout_is_mean:0.999961793422699 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4132578144199215e-05 - rollout_corr/rollout_is_max:1.6447221040725708 - rollout_corr/rollout_is_min:0.002203931100666523 - rollout_corr/rollout_is_std:0.027791311964392662 - rollout_corr/rollout_is_eff_sample_size:0.999228120062589 - rollout_corr/rollout_is_seq_mean:0.9999802112579346 - rollout_corr/rollout_is_seq_std:0.0012473923852667212 - rollout_corr/rollout_is_seq_max:1.0055023431777954 - rollout_corr/rollout_is_seq_min:0.9959372878074646 - rollout_corr/rollout_is_seq_max_deviation:0.00550234317779541 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3645080188289285) - rollout_corr/training_log_ppl:np.float64(0.28979677851020824) - rollout_corr/kl:np.float64(0.0005104832347480048) - rollout_corr/k3_kl:np.float64(0.00045839784411327855) - rollout_corr/rollout_ppl:np.float64(1.3638080032542348) - rollout_corr/rollout_log_ppl:np.float64(0.2892862949083792) - rollout_corr/log_ppl_diff:np.float64(0.0005104836018290371) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010666771559044719) - rollout_corr/log_ppl_diff_max:np.float64(0.005919963121414185) - rollout_corr/log_ppl_diff_min:np.float64(-0.004801034927368164) - rollout_corr/ppl_ratio:np.float64(1.0005116499960423) - rollout_corr/chi2_token:np.float64(0.0008005544077605009) - rollout_corr/chi2_seq:np.float64(1.0131068772170693) - actor/pg_loss:np.float64(-1.758092548698187e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.805006137890814e-05) - actor/ppo_kl:np.float64(7.538383222893685e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.10063287615776062) - perf/mfu/actor:np.float64(0.14549136899234336) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.28108978271484) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:2 - critic/score/mean:0.7890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.027433983981609344 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.027433983981609344 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1105.6015625 - response_length/max:8192.0 - response_length/min:136.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:1105.6015625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:251.4375 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001056082546710968 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1990190539509058) - timing_s/agent_loop/generate_sequences/max:np.float64(59.229342663660645) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.871870818904426) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(59.229342663660645) - timing_
(TaskRunner pid=2281424) 
Training Progress:  46%|████▌     | 46/100 [50:15<1:09:24, 77.13s/it]
(TaskRunner pid=2281424) step:47 - global_seqlen/min:31035 - global_seqlen/max:67357 - global_seqlen/minmax_diff:36322 - global_seqlen/balanced_min:49399 - global_seqlen/balanced_max:49649 - global_seqlen/mean:49578.625 - actor/entropy:0.34646978974342346 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5756693482398987 - training/rollout_probs_diff_mean:0.0030399137176573277 - training/rollout_probs_diff_std:0.00816922727972269 - training/rollout_actor_probs_pearson_corr:0.9995539784431458 - rollout_corr/rollout_is_mean:0.9999984502792358 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.122617484536022e-06 - rollout_corr/rollout_is_max:1.9699045419692993 - rollout_corr/rollout_is_min:0.25761306285858154 - rollout_corr/rollout_is_std:0.026937464252114296 - rollout_corr/rollout_is_eff_sample_size:0.9992746611342851 - rollout_corr/rollout_is_seq_mean:1.0000650882720947 - rollout_corr/rollout_is_seq_std:0.001198867685161531 - rollout_corr/rollout_is_seq_max:1.005571961402893 - rollout_corr/rollout_is_seq_min:0.9964936971664429 - rollout_corr/rollout_is_seq_max_deviation:0.005571961402893066 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3529047267511487) - rollout_corr/training_log_ppl:np.float64(0.28667624859372154) - rollout_corr/kl:np.float64(0.0004597161929975613) - rollout_corr/k3_kl:np.float64(0.00046043861678413123) - rollout_corr/rollout_ppl:np.float64(1.3522660555317998) - rollout_corr/rollout_log_ppl:np.float64(0.2862165296683088) - rollout_corr/log_ppl_diff:np.float64(0.0004597189254127443) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009910539374686778) - rollout_corr/log_ppl_diff_max:np.float64(0.007551088929176331) - rollout_corr/log_ppl_diff_min:np.float64(-0.004575580358505249) - rollout_corr/ppl_ratio:np.float64(1.0004606684669852) - rollout_corr/chi2_token:np.float64(0.0009119359310716391) - rollout_corr/chi2_seq:np.float64(954.7251316532493) - actor/pg_loss:np.float64(-8.133123628795147e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001264397735667444) - actor/ppo_kl:np.float64(0.0001422462675852998) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11066612228751183) - perf/mfu/actor:np.float64(0.1590812815304962) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.31835174560547) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:2 - critic/score/mean:0.75 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.024808576330542564 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.024808576330542564 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1284.58203125 - response_length/max:7270.0 - response_length/min:165.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1284.58203125 - response_length_non_aborted/max:7270.0 - response_length_non_aborted/min:165.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.75 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.761533975601196e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.059980768710375) - timing_s/agent_loop/generate_sequences/max:np.float64(50.74220941402018) - timing_s/agent_loop/generate_sequences/mean:np.float64(12.601909340250131) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.74220941402018) - timing_s/agent_loop/slowest/tool_
(TaskRunner pid=2281424) 
Training Progress:  47%|████▋     | 47/100 [51:35<1:08:44, 77.83s/it]
(TaskRunner pid=2281424) step:48 - global_seqlen/min:22232 - global_seqlen/max:57431 - global_seqlen/minmax_diff:35199 - global_seqlen/balanced_min:41913 - global_seqlen/balanced_max:42517 - global_seqlen/mean:42023.25 - actor/entropy:0.34810274839401245 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2931787967681885 - training/rollout_probs_diff_mean:0.003345586359500885 - training/rollout_probs_diff_std:0.008651177398860455 - training/rollout_actor_probs_pearson_corr:0.9994678497314453 - rollout_corr/rollout_is_mean:1.0000156164169312 - rollout_corr/rollout_is_ratio_fraction_high:7.471328899555374e-06 - rollout_corr/rollout_is_ratio_fraction_low:7.471328899555374e-06 - rollout_corr/rollout_is_max:2.302506685256958 - rollout_corr/rollout_is_min:0.015659699216485023 - rollout_corr/rollout_is_std:0.02865091897547245 - rollout_corr/rollout_is_eff_sample_size:0.9991797981155129 - rollout_corr/rollout_is_seq_mean:1.0000426769256592 - rollout_corr/rollout_is_seq_std:0.0014892894541844726 - rollout_corr/rollout_is_seq_max:1.0078259706497192 - rollout_corr/rollout_is_seq_min:0.9916344881057739 - rollout_corr/rollout_is_seq_max_deviation:0.008365511894226074 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.361631895415485) - rollout_corr/training_log_ppl:np.float64(0.296743152604904) - rollout_corr/kl:np.float64(0.00044437152225995646) - rollout_corr/k3_kl:np.float64(0.0004750857856947732) - rollout_corr/rollout_ppl:np.float64(1.3610092177987099) - rollout_corr/rollout_log_ppl:np.float64(0.2962987821374554) - rollout_corr/log_ppl_diff:np.float64(0.0004443704674486071) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010612811020109802) - rollout_corr/log_ppl_diff_max:np.float64(0.009809985756874084) - rollout_corr/log_ppl_diff_min:np.float64(-0.005429580807685852) - rollout_corr/ppl_ratio:np.float64(1.0004456255119294) - rollout_corr/chi2_token:np.float64(0.0010020604822784662) - rollout_corr/chi2_seq:np.float64(0.4791911744978279) - actor/pg_loss:np.float64(3.651808947324753e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019356490452082653) - actor/ppo_kl:np.float64(3.127409698944206e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2193583231419325) - perf/mfu/actor:np.float64(0.14563750602154726) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.25163650512695) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:2 - critic/score/mean:0.765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.024314971640706062 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.024314971640706062 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1045.6640625 - response_length/max:5423.0 - response_length/min:153.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1045.6640625 - response_length_non_aborted/max:5423.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.5625 - prompt_length/max:365.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001386590301990509 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7352543752640486) - timing_s/agent_loop/generate_sequences/max:np.float64(38.22215409949422) - timing_s/agent_loop/generate_sequences/mean:np.float64(10.838098710140912) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(38.22215409949422) - timing_s
(TaskRunner pid=2281424) 
Training Progress:  48%|████▊     | 48/100 [52:40<1:04:05, 73.96s/it]
(TaskRunner pid=2281424) step:49 - global_seqlen/min:23092 - global_seqlen/max:51168 - global_seqlen/minmax_diff:28076 - global_seqlen/balanced_min:37111 - global_seqlen/balanced_max:37334 - global_seqlen/mean:37259.625 - actor/entropy:0.3179267346858978 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.20985785126686096 - training/rollout_probs_diff_mean:0.0031237066723406315 - training/rollout_probs_diff_std:0.008550681173801422 - training/rollout_actor_probs_pearson_corr:0.9993894100189209 - rollout_corr/rollout_is_mean:0.9999985098838806 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.58763451105915e-06 - rollout_corr/rollout_is_max:1.7586421966552734 - rollout_corr/rollout_is_min:0.3407052159309387 - rollout_corr/rollout_is_std:0.02756519615650177 - rollout_corr/rollout_is_eff_sample_size:0.9992407369068808 - rollout_corr/rollout_is_seq_mean:0.9999875426292419 - rollout_corr/rollout_is_seq_std:0.0011641301680356264 - rollout_corr/rollout_is_seq_max:1.0047152042388916 - rollout_corr/rollout_is_seq_min:0.9961039423942566 - rollout_corr/rollout_is_seq_max_deviation:0.0047152042388916016 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.328462516888976) - rollout_corr/training_log_ppl:np.float64(0.2660138398205163) - rollout_corr/kl:np.float64(0.00038798783518378954) - rollout_corr/k3_kl:np.float64(0.0004385801271951095) - rollout_corr/rollout_ppl:np.float64(1.3279090919531882) - rollout_corr/rollout_log_ppl:np.float64(0.26562585204374045) - rollout_corr/log_ppl_diff:np.float64(0.0003879877767758444) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009870426292764023) - rollout_corr/log_ppl_diff_max:np.float64(0.006948456168174744) - rollout_corr/log_ppl_diff_min:np.float64(-0.004289939999580383) - rollout_corr/ppl_ratio:np.float64(1.0003889380022883) - rollout_corr/chi2_token:np.float64(0.0009887455962598324) - rollout_corr/chi2_seq:np.float64(1.5676374214235693) - actor/pg_loss:np.float64(2.256419975310564e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012307778933973168) - actor/ppo_kl:np.float64(-3.0272832692190832e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.07936635101214051) - perf/mfu/actor:np.float64(0.13578834962618122) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.31588363647461) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:2 - critic/score/mean:0.859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006559879519045353 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006559879519045353 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:909.73828125 - response_length/max:4227.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:909.73828125 - response_length_non_aborted/max:4227.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.625 - prompt_length/max:362.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.641796350479126e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1448177974671125) - timing_s/agent_loop/generate_sequences/max:np.float64(29.94586299546063) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.983509431694984) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(29.94586299546063) - timing_s/agent_loop/slow
(TaskRunner pid=2281424) 
Training Progress:  49%|████▉     | 49/100 [53:35<58:09, 68.42s/it]  
(TaskRunner pid=2281424) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50}
(TaskRunner pid=2281424) validation generation end
(TaskRunner pid=2281424) [prompt] system
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2281424) <reasoning>
(TaskRunner pid=2281424) ...
(TaskRunner pid=2281424) </reasoning>
(TaskRunner pid=2281424) <answer>
(TaskRunner pid=2281424) ...
(TaskRunner pid=2281424) </answer>
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2281424) <answer>
(TaskRunner pid=2281424) A
(TaskRunner pid=2281424) </answer>
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) user
(TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$.
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) A: 50 pC
(TaskRunner pid=2281424) B: 56 pC
(TaskRunner pid=2281424) C: 60 pC
(TaskRunner pid=2281424) D: 64 pC
(TaskRunner pid=2281424) Please reason step by step.
(TaskRunner pid=2281424) assistant
(TaskRunner pid=2281424) <think>
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) </think>
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) [response] To find the magnitude of the charge $ q $, we use the formula for the electric field due to a point charge:
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) E = k \frac{q}{r^2}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) We are given:
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) - $ E = 2.0 \, \mathrm{N/C} $
(TaskRunner pid=2281424) - $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $
(TaskRunner pid=2281424) - $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) We need to solve for $ q $:
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q = \frac{E \cdot r^2}{k}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) Substituting the given values:
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q = \frac{0.5}{8.99 \times 10^9}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) Now convert to picocoulombs (1 pC = $ 10^{-12} $ C):
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) q \approx 55.6 \, \mathrm{pC}
(TaskRunner pid=2281424) $$
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) So the closest option is:
(TaskRunner pid=2281424) 
(TaskRunner pid=2281424) <answer>
(TaskRunner pid=2281424) B
(TaskRunner pid=2281424) </answer>
(TaskRunner pid=2281424) [ground_truth] B
(TaskRunner pid=2281424) [score] 1.0
(TaskRunner pid=2281424) [acc] 1.0
(TaskRunner pid=2281424) [pred] B
(TaskRunner pid=2281424) [incorrect_format] 1
(TaskRunner pid=2281424) [feedback] 
(TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50
(TaskRunner pid=2281424) step:50 - global_seqlen/min:34234 - global_seqlen/max:75152 - global_seqlen/minmax_diff:40918 - global_seqlen/balanced_min:51555 - global_seqlen/balanced_max:51907 - global_seqlen/mean:51805.25 - actor/entropy:0.4125359058380127 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19821900129318237 - training/rollout_probs_diff_mean:0.0032416246831417084 - training/rollout_probs_diff_std:0.008041433990001678 - training/rollout_actor_probs_pearson_corr:0.999610185623169 - rollout_corr/rollout_is_mean:1.0000464916229248 - rollout_corr/rollout_is_ratio_fraction_high:8.665861969348043e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.167043685913086 - rollout_corr/rollout_is_min:0.5511428117752075 - rollout_corr/rollout_is_std:0.028244443237781525 - rollout_corr/rollout_is_eff_sample_size:0.9992028873298154 - rollout_corr/rollout_is_seq_mean:0.9999783635139465 - rollout_corr/rollout_is_seq_std:0.0010451057460159063 - rollout_corr/rollout_is_seq_max:1.0044012069702148 - rollout_corr/rollout_is_seq_min:0.9962003827095032 - rollout_corr/rollout_is_seq_max_deviation:0.004401206970214844 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4209796008653939) - rollout_corr/training_log_ppl:np.float64(0.3332373514131177) - rollout_corr/kl:np.float64(0.0004479175979907879) - rollout_corr/k3_kl:np.float64(0.0004798759525215246) - rollout_corr/rollout_ppl:np.float64(1.4203380569815636) - rollout_corr/rollout_log_ppl:np.float64(0.3327894336543977) - rollout_corr/log_ppl_diff:np.float64(0.0004479177587199956) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009449601930100471) - rollout_corr/log_ppl_diff_max:np.float64(0.008978396654129028) - rollout_corr/log_ppl_diff_min:np.float64(-0.004376143217086792) - rollout_corr/ppl_ratio:np.float64(1.0004488993436098) - rollout_corr/chi2_token:np.float64(0.0010542639065533876) - rollout_corr/chi2_seq:np.float64(2.937424761708826) - actor/pg_loss:np.float64(0.00014853477478027344) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021279246669791974) - actor/ppo_kl:np.float64(1.2714361647070405e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1673903912305832) - perf/mfu/actor:np.float64(0.16524905376259622) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.29336929321289) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.65390625) - val-aux/sciknoweval/reward/std@16:np.float64(0.21345823452859586) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7464375000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.19899169203769165) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.55885) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1654343787430585) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6524) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.21312019792510917) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8329625) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.14911841799011322) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.49555000000000005) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.0995466061193929) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6846875000000001) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.19169144926745516) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8903625) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08902310096279493) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4632625) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.04804278474577391) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7100125) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.16088561753057692) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9214874999999999) - val-aux/sciknoweval/reward/best@16/std:np
(TaskRunner pid=2281424) 
Training Progress:  50%|█████     | 50/100 [57:56<1:45:03, 126.06s/it]
(TaskRunner pid=2281424) step:51 - global_seqlen/min:25192 - global_seqlen/max:76157 - global_seqlen/minmax_diff:50965 - global_seqlen/balanced_min:48271 - global_seqlen/balanced_max:48448 - global_seqlen/mean:48379.375 - actor/entropy:0.41202202439308167 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37214818596839905 - training/rollout_probs_diff_mean:0.003262621583417058 - training/rollout_probs_diff_std:0.008078004233539104 - training/rollout_actor_probs_pearson_corr:0.9995943903923035 - rollout_corr/rollout_is_mean:0.9999638795852661 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.276774456637213e-06 - rollout_corr/rollout_is_max:1.636783480644226 - rollout_corr/rollout_is_min:0.13366740942001343 - rollout_corr/rollout_is_std:0.027932504191994667 - rollout_corr/rollout_is_eff_sample_size:0.999220264452564 - rollout_corr/rollout_is_seq_mean:0.9999167919158936 - rollout_corr/rollout_is_seq_std:0.001000249176286161 - rollout_corr/rollout_is_seq_max:1.0028444528579712 - rollout_corr/rollout_is_seq_min:0.9964448809623718 - rollout_corr/rollout_is_seq_max_deviation:0.003555119037628174 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3898025089874864) - rollout_corr/training_log_ppl:np.float64(0.31166147111798637) - rollout_corr/kl:np.float64(0.0004372116775031021) - rollout_corr/k3_kl:np.float64(0.0003934571469699222) - rollout_corr/rollout_ppl:np.float64(1.3891857019625604) - rollout_corr/rollout_log_ppl:np.float64(0.3112242584757041) - rollout_corr/log_ppl_diff:np.float64(0.00043721264228224754) - rollout_corr/log_ppl_abs_diff:np.float64(0.000884955981746316) - rollout_corr/log_ppl_diff_max:np.float64(0.007482245564460754) - rollout_corr/log_ppl_diff_min:np.float64(-0.002665892243385315) - rollout_corr/ppl_ratio:np.float64(1.0004379730671644) - rollout_corr/chi2_token:np.float64(0.0006916895508766174) - rollout_corr/chi2_seq:np.float64(0.8691392296459526) - actor/pg_loss:np.float64(5.567236803472042e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015443053536046136) - actor/ppo_kl:np.float64(-3.0327652694950302e-05) - actor/pg_clipfrac_lower:np.float64(8.184056241589133e-07) - actor/grad_norm:np.float64(0.10533490870147943) - perf/mfu/actor:np.float64(0.1597418217861688) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.85708999633789) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:2 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014979914762079716 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014979914762079716 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1244.66796875 - response_length/max:5675.0 - response_length/min:200.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1244.66796875 - response_length_non_aborted/max:5675.0 - response_length_non_aborted/min:200.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.1875 - prompt_length/max:356.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011126697063446045 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4992719031870365) - timing_s/agent_loop/generate_sequences/max:np.float64(40.4043846745044) - timing_s/agent_loop/generate_sequences/mean:np.float64(11.760603568858642) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(40.4043846745044) - t
(TaskRunner pid=2281424) 
Training Progress:  51%|█████     | 51/100 [59:03<1:28:37, 108.52s/it]
(TaskRunner pid=2281424) step:52 - global_seqlen/min:38872 - global_seqlen/max:61135 - global_seqlen/minmax_diff:22263 - global_seqlen/balanced_min:50883 - global_seqlen/balanced_max:50962 - global_seqlen/mean:50922.75 - actor/entropy:0.4251244068145752 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31729912757873535 - training/rollout_probs_diff_mean:0.0034149810671806335 - training/rollout_probs_diff_std:0.008329342119395733 - training/rollout_actor_probs_pearson_corr:0.9995748996734619 - rollout_corr/rollout_is_mean:0.9999804496765137 - rollout_corr/rollout_is_ratio_fraction_high:2.9626644391100854e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.925328878220171e-06 - rollout_corr/rollout_is_max:2.0289995670318604 - rollout_corr/rollout_is_min:0.38479822874069214 - rollout_corr/rollout_is_std:0.02867795154452324 - rollout_corr/rollout_is_eff_sample_size:0.9991781319248341 - rollout_corr/rollout_is_seq_mean:1.0001072883605957 - rollout_corr/rollout_is_seq_std:0.0014209682121872902 - rollout_corr/rollout_is_seq_max:1.0089370012283325 - rollout_corr/rollout_is_seq_min:0.9960955381393433 - rollout_corr/rollout_is_seq_max_deviation:0.00893700122833252 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4353607962839305) - rollout_corr/training_log_ppl:np.float64(0.34465374518185854) - rollout_corr/kl:np.float64(0.00030015603282895764) - rollout_corr/k3_kl:np.float64(0.00044390009770722827) - rollout_corr/rollout_ppl:np.float64(1.4348906846717) - rollout_corr/rollout_log_ppl:np.float64(0.3443535875703674) - rollout_corr/log_ppl_diff:np.float64(0.0003001576114911586) - rollout_corr/log_ppl_abs_diff:np.float64(0.000979981996351853) - rollout_corr/log_ppl_diff_max:np.float64(0.006529062986373901) - rollout_corr/log_ppl_diff_min:np.float64(-0.006823122501373291) - rollout_corr/ppl_ratio:np.float64(1.0003011787775904) - rollout_corr/chi2_token:np.float64(0.0011839414946734905) - rollout_corr/chi2_seq:np.float64(2.5429790497291833) - actor/pg_loss:np.float64(3.4721335396170616e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.84185026279738e-05) - actor/ppo_kl:np.float64(-4.0347963830456024e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11110754683613777) - perf/mfu/actor:np.float64(0.16599907427815286) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.80552291870117) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:2 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004099216777831316 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004099216777831316 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1318.4921875 - response_length/max:5029.0 - response_length/min:132.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1318.4921875 - response_length_non_aborted/max:5029.0 - response_length_non_aborted/min:132.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.84375 - prompt_length/max:365.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001096632331609726 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7103512845933437) - timing_s/agent_loop/generate_sequences/max:np.float64(38.38058450073004) - timing_s/agent_loop/generate_sequences/mean:np.float64(13.082279307935096) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(38.38058450073004) - ti
(TaskRunner pid=2281424) 
Training Progress:  52%|█████▏    | 52/100 [1:00:09<1:16:34, 95.73s/it]
(TaskRunner pid=2281424) step:53 - global_seqlen/min:24839 - global_seqlen/max:55902 - global_seqlen/minmax_diff:31063 - global_seqlen/balanced_min:43329 - global_seqlen/balanced_max:43584 - global_seqlen/mean:43520.375 - actor/entropy:0.37126773595809937 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016398251056671 - training/rollout_probs_diff_mean:0.003143634181469679 - training/rollout_probs_diff_std:0.00820545107126236 - training/rollout_actor_probs_pearson_corr:0.9994932413101196 - rollout_corr/rollout_is_mean:0.9999392628669739 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.210143394331681e-06 - rollout_corr/rollout_is_max:1.7527995109558105 - rollout_corr/rollout_is_min:0.30146440863609314 - rollout_corr/rollout_is_std:0.027419937774538994 - rollout_corr/rollout_is_eff_sample_size:0.9992485928388086 - rollout_corr/rollout_is_seq_mean:0.9999774694442749 - rollout_corr/rollout_is_seq_std:0.0011340720811858773 - rollout_corr/rollout_is_seq_max:1.005629062652588 - rollout_corr/rollout_is_seq_min:0.9950371980667114 - rollout_corr/rollout_is_seq_max_deviation:0.005629062652587891 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3701058248989284) - rollout_corr/training_log_ppl:np.float64(0.2918059763032943) - rollout_corr/kl:np.float64(0.0003194374729091898) - rollout_corr/k3_kl:np.float64(0.00042779503252177165) - rollout_corr/rollout_ppl:np.float64(1.3696815236471593) - rollout_corr/rollout_log_ppl:np.float64(0.29148653568699956) - rollout_corr/log_ppl_diff:np.float64(0.00031944061629474163) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009462947491556406) - rollout_corr/log_ppl_diff_max:np.float64(0.006018996238708496) - rollout_corr/log_ppl_diff_min:np.float64(-0.003216482698917389) - rollout_corr/ppl_ratio:np.float64(1.0003202487714589) - rollout_corr/chi2_token:np.float64(0.0010781094897538424) - rollout_corr/chi2_seq:np.float64(1.596792209893465) - actor/pg_loss:np.float64(3.835768438875675e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021167921056530759) - actor/ppo_kl:np.float64(-7.656292218705119e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.13869349658489227) - perf/mfu/actor:np.float64(0.14950320003636675) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.62938690185547) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:2 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015568501316010952 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015568501316010952 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1083.54296875 - response_length/max:5024.0 - response_length/min:193.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1083.54296875 - response_length_non_aborted/max:5024.0 - response_length_non_aborted/min:193.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.46875 - prompt_length/max:375.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011430680751800537 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2967226803302765) - timing_s/agent_loop/generate_sequences/max:np.float64(36.389772513881326) - timing_s/agent_loop/generate_sequences/mean:np.float64(10.762310426936892) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(36.389772513881326) - timing_s/ag
(TaskRunner pid=2281424) 
Training Progress:  53%|█████▎    | 53/100 [1:01:12<1:07:19, 85.94s/it]
(TaskRunner pid=2281424) step:54 - global_seqlen/min:35066 - global_seqlen/max:65296 - global_seqlen/minmax_diff:30230 - global_seqlen/balanced_min:48016 - global_seqlen/balanced_max:48715 - global_seqlen/mean:48320.375 - actor/entropy:0.41513416171073914 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3469814658164978 - training/rollout_probs_diff_mean:0.0032787041272968054 - training/rollout_probs_diff_std:0.008095493540167809 - training/rollout_actor_probs_pearson_corr:0.999546229839325 - rollout_corr/rollout_is_mean:0.9998549818992615 - rollout_corr/rollout_is_ratio_fraction_high:3.120212340945727e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.360637704958208e-06 - rollout_corr/rollout_is_max:3.656163454055786 - rollout_corr/rollout_is_min:0.04287761077284813 - rollout_corr/rollout_is_std:0.027915427461266518 - rollout_corr/rollout_is_eff_sample_size:0.99922121664114 - rollout_corr/rollout_is_seq_mean:0.9998648166656494 - rollout_corr/rollout_is_seq_std:0.0012277925852686167 - rollout_corr/rollout_is_seq_max:1.0036733150482178 - rollout_corr/rollout_is_seq_min:0.9944618940353394 - rollout_corr/rollout_is_seq_max_deviation:0.0055381059646606445 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.386646915692836) - rollout_corr/training_log_ppl:np.float64(0.31141754146665335) - rollout_corr/kl:np.float64(0.00046588320381735215) - rollout_corr/k3_kl:np.float64(0.00041930354774422085) - rollout_corr/rollout_ppl:np.float64(1.3860031738877296) - rollout_corr/rollout_log_ppl:np.float64(0.31095165672013536) - rollout_corr/log_ppl_diff:np.float64(0.00046588474651798606) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009574951836839318) - rollout_corr/log_ppl_diff_max:np.float64(0.0065114498138427734) - rollout_corr/log_ppl_diff_min:np.float64(-0.0028219521045684814) - rollout_corr/ppl_ratio:np.float64(1.0004668282344937) - rollout_corr/chi2_token:np.float64(0.000739572336897254) - rollout_corr/chi2_seq:np.float64(1.1254641045816243) - actor/pg_loss:np.float64(-1.3741431757807732e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.72049774898187e-05) - actor/ppo_kl:np.float64(-8.353029136642931e-05) - actor/pg_clipfrac_lower:np.float64(1.2657970955842757e-06) - actor/grad_norm:np.float64(0.09051301795989275) - perf/mfu/actor:np.float64(0.15693746638014805) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.6057014465332) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:2 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015124449506402016 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015124449506402016 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1251.91796875 - response_length/max:8192.0 - response_length/min:151.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:1251.91796875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:151.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:258.09375 - prompt_length/max:363.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.027255535125732e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3670800887048244) - timing_s/agent_loop/generate_sequences/max:np.float64(57.39985219575465) - timing_s/agent_loop/generate_sequences/mean:np.float64(11.685668693418847) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_seq
(TaskRunner pid=2281424) 
Training Progress:  54%|█████▍    | 54/100 [1:02:38<1:05:43, 85.72s/it]
(TaskRunner pid=2281424) step:55 - global_seqlen/min:28028 - global_seqlen/max:53093 - global_seqlen/minmax_diff:25065 - global_seqlen/balanced_min:38416 - global_seqlen/balanced_max:38451 - global_seqlen/mean:38438.25 - actor/entropy:0.3102551996707916 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5002540349960327 - training/rollout_probs_diff_mean:0.0031136018224060535 - training/rollout_probs_diff_std:0.008566453121602535 - training/rollout_actor_probs_pearson_corr:0.9994353652000427 - rollout_corr/rollout_is_mean:0.9999341368675232 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.173030447418569e-06 - rollout_corr/rollout_is_max:1.8234162330627441 - rollout_corr/rollout_is_min:0.46666446328163147 - rollout_corr/rollout_is_std:0.02762567438185215 - rollout_corr/rollout_is_eff_sample_size:0.9992372850970654 - rollout_corr/rollout_is_seq_mean:0.999902606010437 - rollout_corr/rollout_is_seq_std:0.0013777980348095298 - rollout_corr/rollout_is_seq_max:1.0075701475143433 - rollout_corr/rollout_is_seq_min:0.9948606491088867 - rollout_corr/rollout_is_seq_max_deviation:0.007570147514343262 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.308470833580941) - rollout_corr/training_log_ppl:np.float64(0.2564443113806192) - rollout_corr/kl:np.float64(0.0004685787036251554) - rollout_corr/k3_kl:np.float64(0.0005076911484707125) - rollout_corr/rollout_ppl:np.float64(1.3078627600334585) - rollout_corr/rollout_log_ppl:np.float64(0.255975730920909) - rollout_corr/log_ppl_diff:np.float64(0.00046858045971021056) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011701608309522271) - rollout_corr/log_ppl_diff_max:np.float64(0.006679356098175049) - rollout_corr/log_ppl_diff_min:np.float64(-0.0075002312660217285) - rollout_corr/ppl_ratio:np.float64(1.000470134196803) - rollout_corr/chi2_token:np.float64(0.0011455377098172903) - rollout_corr/chi2_seq:np.float64(5.852095796493813) - actor/pg_loss:np.float64(-3.334670327603817e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001705758450043504) - actor/ppo_kl:np.float64(-4.5051564853793025e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.07974306493997574) - perf/mfu/actor:np.float64(0.1378203781579895) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.55471420288086) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:2 - critic/score/mean:0.84765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.84765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011153988540172577 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011153988540172577 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:936.0703125 - response_length/max:3898.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:936.0703125 - response_length_non_aborted/max:3898.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.125 - prompt_length/max:383.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001405421644449234 - timing_s/agent_loop/generate_sequences/min:np.float64(1.978264905512333) - timing_s/agent_loop/generate_sequences/max:np.float64(29.77366792783141) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.763206802737841) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(29.77366792783141) - timing_s/agent_loop/slowest
(TaskRunner pid=2281424) 
Training Progress:  55%|█████▌    | 55/100 [1:03:33<57:28, 76.64s/it]  
(TaskRunner pid=2281424) step:56 - global_seqlen/min:35222 - global_seqlen/max:67738 - global_seqlen/minmax_diff:32516 - global_seqlen/balanced_min:48431 - global_seqlen/balanced_max:48616 - global_seqlen/mean:48556.375 - actor/entropy:0.3693101406097412 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5576332211494446 - training/rollout_probs_diff_mean:0.003098106477409601 - training/rollout_probs_diff_std:0.008025160990655422 - training/rollout_actor_probs_pearson_corr:0.9995522499084473 - rollout_corr/rollout_is_mean:0.999983012676239 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.090129894189886e-06 - rollout_corr/rollout_is_max:1.6744037866592407 - rollout_corr/rollout_is_min:0.3452942669391632 - rollout_corr/rollout_is_std:0.02711610123515129 - rollout_corr/rollout_is_eff_sample_size:0.9992651383115954 - rollout_corr/rollout_is_seq_mean:0.9999608993530273 - rollout_corr/rollout_is_seq_std:0.0010731675429269671 - rollout_corr/rollout_is_seq_max:1.006026029586792 - rollout_corr/rollout_is_seq_min:0.9958376288414001 - rollout_corr/rollout_is_seq_max_deviation:0.006026029586791992 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.373812660574913) - rollout_corr/training_log_ppl:np.float64(0.3022938699577935) - rollout_corr/kl:np.float64(0.0004166627330413064) - rollout_corr/k3_kl:np.float64(0.00039845293167672935) - rollout_corr/rollout_ppl:np.float64(1.3732107440009713) - rollout_corr/rollout_log_ppl:np.float64(0.3018772063078359) - rollout_corr/log_ppl_diff:np.float64(0.00041666364995762706) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008859913214109838) - rollout_corr/log_ppl_diff_max:np.float64(0.00528949499130249) - rollout_corr/log_ppl_diff_min:np.float64(-0.005298733711242676) - rollout_corr/ppl_ratio:np.float64(1.0004174057394266) - rollout_corr/chi2_token:np.float64(0.0007626630831509829) - rollout_corr/chi2_seq:np.float64(3.9297358801122755) - actor/pg_loss:np.float64(-1.1299271136522293e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014216022026403152) - actor/ppo_kl:np.float64(-9.481449527726227e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08582276105880737) - perf/mfu/actor:np.float64(0.15478688954187692) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(117.41515350341797) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:2 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01221644226461649 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.01221644226461649 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:1264.10546875 - response_length/max:5770.0 - response_length/min:163.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1264.10546875 - response_length_non_aborted/max:5770.0 - response_length_non_aborted/min:163.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:253.28125 - prompt_length/max:355.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013057328760623932 - timing_s/agent_loop/generate_sequences/min:np.float64(2.215219648554921) - timing_s/agent_loop/generate_sequences/max:np.float64(45.5247247479856) - timing_s/agent_loop/generate_sequences/mean:np.float64(12.946847598039312) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(45.5247247479856) - timing_s/agent_loop/slowest
(TaskRunner pid=2281424) 
Training Progress:  56%|█████▌    | 56/100 [1:04:47<55:32, 75.74s/it]
(TaskRunner pid=2281424) step:57 - global_seqlen/min:50111 - global_seqlen/max:61804 - global_seqlen/minmax_diff:11693 - global_seqlen/balanced_min:54058 - global_seqlen/balanced_max:54380 - global_seqlen/mean:54299.125 - actor/entropy:0.3828109800815582 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26735612750053406 - training/rollout_probs_diff_mean:0.003163557732477784 - training/rollout_probs_diff_std:0.007935766130685806 - training/rollout_actor_probs_pearson_corr:0.9995775818824768 - rollout_corr/rollout_is_mean:0.9999682903289795 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.757548145382316e-06 - rollout_corr/rollout_is_max:1.6177020072937012 - rollout_corr/rollout_is_min:0.4519280195236206 - rollout_corr/rollout_is_std:0.027300119400024414 - rollout_corr/rollout_is_eff_sample_size:0.9992552585748898 - rollout_corr/rollout_is_seq_mean:0.9999948740005493 - rollout_corr/rollout_is_seq_std:0.0008971233037300408 - rollout_corr/rollout_is_seq_max:1.004016637802124 - rollout_corr/rollout_is_seq_min:0.9975495934486389 - rollout_corr/rollout_is_seq_max_deviation:0.0040166378021240234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4292010506615043) - rollout_corr/training_log_ppl:np.float64(0.3405683948658407) - rollout_corr/kl:np.float64(0.00034688604483079644) - rollout_corr/k3_kl:np.float64(0.00040535901956673115) - rollout_corr/rollout_ppl:np.float64(1.4287002789787948) - rollout_corr/rollout_log_ppl:np.float64(0.34022150505916215) - rollout_corr/log_ppl_diff:np.float64(0.00034688980667851865) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007467021641787142) - rollout_corr/log_ppl_diff_max:np.float64(0.0035548806190490723) - rollout_corr/log_ppl_diff_min:np.float64(-0.0031991004943847656) - rollout_corr/ppl_ratio:np.float64(1.0003474049735814) - rollout_corr/chi2_token:np.float64(0.0009404013399034739) - rollout_corr/chi2_seq:np.float64(1.3481947318650782) - actor/pg_loss:np.float64(-1.1662370525300503e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001704909616933037) - actor/ppo_kl:np.float64(-4.2620445977803456e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11611825972795486) - perf/mfu/actor:np.float64(0.13975899720323387) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(142.28152465820312) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:2 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0023128935135900974 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0023128935135900974 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1416.56640625 - response_length/max:6110.0 - response_length/min:138.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1416.56640625 - response_length_non_aborted/max:6110.0 - response_length_non_aborted/min:138.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.28125 - prompt_length/max:382.0 - prompt_length/min:209.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.9925798773765564e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7666615955531597) - timing_s/agent_loop/generate_sequences/max:np.float64(44.93153551965952) - timing_s/agent_loop/generate_sequences/mean:np.float64(14.538133056579682) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(44.93153551965952) - timing_s
(TaskRunner pid=2281424) 
Training Progress:  57%|█████▋    | 57/100 [1:06:04<54:35, 76.17s/it]