SeongryongJung/Qwen3-4B-Chemical-GRPO-TR
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
Files
main
Qwen3-4B-Chemical-GRPO-TR/artifacts/queue.log

1961 lines
730 KiB
Plaintext
Raw Permalink Normal View History

初始化项目，由ModelHub XC社区提供模型 Model: SeongryongJung/Qwen3-4B-Chemical-GRPO-TR Source: Original Platform
2026-08-09 20:57:18 +08:00
[2026-07-02 05:05:41] Starting L2T batch32 Qwen3-4B GRPO then RLSD_TR queue
[2026-07-02 05:05:42] Commit: ee6a667
[2026-07-02 05:05:42] Settings: model=Qwen/Qwen3-4B steps=100 train_batch=32 rlsd_tr_batch=32 rollout=8 mini_batch=8 tr_mix=0.1 max_model_len=10240 vllm=0.8
[2026-07-02 05:05:42] Starting chemistry grpo
Experiment: qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8
Dataset: chemistry
Method: grpo
Config: baseline_grpo
Model: Qwen/Qwen3-4B
Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet
Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet
Ray tmp: /tmp/ray_q3g_chemistry_grpo_Qwen3_4B
2026-07-02 05:05:43,546 INFO scripts.py:1364 -- Did not find any active Ray processes.
Experiment: qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8
Config: baseline_grpo
Task: datasets/sciknoweval/chemistry
Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-GRPO-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_chemistry_grpo_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/chemistry +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.actor.policy_loss.loss_mode=vanilla actor_rollout_ref.actor.kl_loss_coef=0.0
ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_chemistry_grpo_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/chemistry', 'EXPERIMENT': 'qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}}
2026-07-02 05:05:53,228 INFO worker.py:2007 -- Started a local Ray instance.
/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
warnings.warn(
(TaskRunner pid=2001144) TaskRunner hostname: mole-workspace-rw, PID: 2001144
(TaskRunner pid=2001144) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig',
(TaskRunner pid=2001144) 'calculate_entropy': False,
(TaskRunner pid=2001144) 'calculate_sum_pi_squared': False,
(TaskRunner pid=2001144) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig',
(TaskRunner pid=2001144) 'async_save': False,
(TaskRunner pid=2001144) 'load_contents': ['model',
(TaskRunner pid=2001144) 'optimizer',
(TaskRunner pid=2001144) 'extra'],
(TaskRunner pid=2001144) 'save_contents': ['model',
(TaskRunner pid=2001144) 'optimizer',
(TaskRunner pid=2001144) 'extra']},
(TaskRunner pid=2001144) 'clip_ratio': 0.2,
(TaskRunner pid=2001144) 'clip_ratio_c': 3.0,
(TaskRunner pid=2001144) 'clip_ratio_high': 0.28,
(TaskRunner pid=2001144) 'clip_ratio_low': 0.2,
(TaskRunner pid=2001144) 'data_loader_seed': 42,
(TaskRunner pid=2001144) 'entropy_checkpointing': False,
(TaskRunner pid=2001144) 'entropy_coeff': 0,
(TaskRunner pid=2001144) 'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2001144) 'freeze_vision_tower': False,
(TaskRunner pid=2001144) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2001144) 'dtype': 'bfloat16',
(TaskRunner pid=2001144) 'entropy_checkpointing': False,
(TaskRunner pid=2001144) 'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2001144) 'forward_only': False,
(TaskRunner pid=2001144) 'forward_prefetch': False,
(TaskRunner pid=2001144) 'fsdp_size': -1,
(TaskRunner pid=2001144) 'full_determinism': False,
(TaskRunner pid=2001144) 'model_dtype': 'fp32',
(TaskRunner pid=2001144) 'offload_policy': False,
(TaskRunner pid=2001144) 'optimizer_offload': False,
(TaskRunner pid=2001144) 'param_offload': False,
(TaskRunner pid=2001144) 'reshard_after_forward': True,
(TaskRunner pid=2001144) 'seed': 42,
(TaskRunner pid=2001144) 'strategy': 'fsdp',
(TaskRunner pid=2001144) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2001144) 'use_orig_params': False,
(TaskRunner pid=2001144) 'use_torch_compile': True,
(TaskRunner pid=2001144) 'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2001144) 'grad_clip': 1.0,
(TaskRunner pid=2001144) 'kl_loss_coef': 0.0,
(TaskRunner pid=2001144) 'kl_loss_type': 'low_var_kl',
(TaskRunner pid=2001144) 'loss_agg_mode': 'token-mean',
(TaskRunner pid=2001144) 'loss_scale_factor': None,
(TaskRunner pid=2001144) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig',
(TaskRunner pid=2001144) 'betas': [0.9, 0.999],
(TaskRunner pid=2001144) 'clip_grad': 1.0,
(TaskRunner pid=2001144) 'lr': 1e-06,
(TaskRunner pid=2001144) 'lr_scheduler_type': 'constant',
(TaskRunner pid=2001144) 'lr_warmup_steps': 10,
(TaskRunner pid=2001144) 'lr_warmup_steps_ratio': 0.0,
(TaskRunner pid=2001144) 'min_lr_ratio': 0.0,
(TaskRunner pid=2001144) 'num_cycles': 0.5,
(TaskRunner pid=2001144) 'optimizer': 'AdamW',
(TaskRunner pid=2001144) 'optimizer_impl': 'torch.optim',
(TaskRunner pid=2001144) 'override_optimizer_config': None,
(TaskRunner pid=2001144) 'total_training_steps': -1,
(TaskRunner pid=2001144) 'warmup_style': None,
(TaskRunner pid=2001144) 'weight_decay': 0.01},
(TaskRunner pid=2001144) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig',
(TaskRunner pid=2001144) 'clip_cov_lb': 1.0,
(TaskRunner pid=2001144) 'clip_cov_ratio': 0.0002,
(TaskRunner pid=2001144) 'clip_cov_ub': 5.0,
(TaskRunner pid=2001144) 'kl_cov_ratio': 0.0002,
(TaskRunner pid=2001144) 'loss_mode': 'vanilla',
(TaskRunner pid=2001144) 'ppo_kl_coef': 0.1},
(TaskRunner pid=2001144) 'ppo_epochs': 1,
(TaskRunner pid=2001144) 'ppo_max_token_len_per_gpu': 10240,
(TaskRunner pid=2001144) 'ppo_micro_batch_size': None,
(TaskRunner pid=2001144) 'ppo_micro_batch_size_per_gpu': 1,
(TaskRunner pid=2001144) 'ppo_mini_batch_size': 8,
(TaskRunner pid=2001144) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2001144) 'all_ranks': False,
(TaskRunner pid=2001144) 'enable': False,
(TaskRunner pid=2001144) 'ranks': [],
(TaskRunner pid=2001144) 'save_path': 'outputs/profile',
(TaskRunner pid=2001144) 'tool': None,
(TaskRunner pid=2001144) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2001144) 'analysis': True,
(TaskRunner pid=2001144) 'contents': [],
(TaskRunner pid=2001144) 'discrete': False,
(TaskRunner pid=2001144) 'level': 'level0'},
(TaskRunner pid=2001144) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2001144) 'discrete': False},
(TaskRunner pid=2001144) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2001144) 'step_end': None,
(TaskRunner pid=2001144) 'step_start': 0},
(TaskRunner pid=2001144) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2001144) 'stack_depth': 32,
(TaskRunner pid=2001144) 'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2001144) 'rollout_n': 8,
(TaskRunner pid=2001144) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig',
(TaskRunner pid=2001144) 'mode': 'disabled',
(TaskRunner pid=2001144) 'record_file': None,
(TaskRunner pid=2001144) 'replay_file': None},
(TaskRunner pid=2001144) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig',
(TaskRunner pid=2001144) 'alpha': 0.5,
(TaskRunner pid=2001144) 'distillation_add_tail': True,
(TaskRunner pid=2001144) 'distillation_topk': 100,
(TaskRunner pid=2001144) 'dont_reprompt_on_self_success': True,
(TaskRunner pid=2001144) 'environment_feedback_only_without_solution': True,
(TaskRunner pid=2001144) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig',
(TaskRunner pid=2001144) 'enable': False,
(TaskRunner pid=2001144) 'loss_weight': 0.0,
(TaskRunner pid=2001144) 'mask': 'all'},
(TaskRunner pid=2001144) 'feedback_template': '\n'
(TaskRunner pid=2001144) 'The '
(TaskRunner pid=2001144) 'following '
(TaskRunner pid=2001144) 'is '
(TaskRunner pid=2001144) 'feedback '
(TaskRunner pid=2001144) 'from '
(TaskRunner pid=2001144) 'your '
(TaskRunner pid=2001144) 'unsuccessful '
(TaskRunner pid=2001144) 'earlier '
(TaskRunner pid=2001144) 'attempt:\n'
(TaskRunner pid=2001144) '\n'
(TaskRunner pid=2001144) '{feedback_raw}',
(TaskRunner pid=2001144) 'full_logit_distillation': True,
(TaskRunner pid=2001144) 'include_environment_feedback': True,
(TaskRunner pid=2001144) 'is_clip': 2,
(TaskRunner pid=2001144) 'max_reprompt_len': 22528,
(TaskRunner pid=2001144) 'remove_thinking_from_demonstration': False,
(TaskRunner pid=2001144) 'reprompt_template': '{prompt}{solution}{feedback}\n'
(TaskRunner pid=2001144) '\n'
(TaskRunner pid=2001144) 'Correctly '
(TaskRunner pid=2001144) 'solve '
(TaskRunner pid=2001144) 'the '
(TaskRunner pid=2001144) 'original '
(TaskRunner pid=2001144) 'question.',
(TaskRunner pid=2001144) 'reprompt_truncation': 'right',
(TaskRunner pid=2001144) 'solution_template': '\n'
(TaskRunner pid=2001144) 'Correct '
(TaskRunner pid=2001144) 'solution:\n'
(TaskRunner pid=2001144) '\n'
(TaskRunner pid=2001144) '{successful_previous_attempt}',
(TaskRunner pid=2001144) 'srpo_dynamic_weighting': False,
(TaskRunner pid=2001144) 'srpo_dynamic_weighting_temperature': 1.0,
(TaskRunner pid=2001144) 'success_reward_threshold': 0.5,
(TaskRunner pid=2001144) 'teacher_regularization': 'ema',
(TaskRunner pid=2001144) 'teacher_update_rate': 0.0,
(TaskRunner pid=2001144) 'token_reweight_decay_steps': None,
(TaskRunner pid=2001144) 'token_reweight_eps_w': 0.2,
(TaskRunner pid=2001144) 'token_reweight_lambda': 0.5},
(TaskRunner pid=2001144) 'shuffle': False,
(TaskRunner pid=2001144) 'strategy': 'fsdp',
(TaskRunner pid=2001144) 'sum_pi_squared_checkpointing': False,
(TaskRunner pid=2001144) 'tau_neg': 1.05,
(TaskRunner pid=2001144) 'tau_pos': 1.0,
(TaskRunner pid=2001144) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2001144) 'use_dynamic_bsz': False,
(TaskRunner pid=2001144) 'use_fused_kernels': False,
(TaskRunner pid=2001144) 'use_kl_loss': False,
(TaskRunner pid=2001144) 'use_prefix_grouper': False,
(TaskRunner pid=2001144) 'use_remove_padding': True,
(TaskRunner pid=2001144) 'use_torch_compile': True},
(TaskRunner pid=2001144) 'hybrid_engine': True,
(TaskRunner pid=2001144) 'model': {'_target_': 'verl.workers.config.HFModelConfig',
(TaskRunner pid=2001144) 'custom_chat_template': None,
(TaskRunner pid=2001144) 'enable_activation_offload': False,
(TaskRunner pid=2001144) 'enable_gradient_checkpointing': True,
(TaskRunner pid=2001144) 'exclude_modules': None,
(TaskRunner pid=2001144) 'external_lib': None,
(TaskRunner pid=2001144) 'fused_kernel_options': {'impl_backend': 'torch'},
(TaskRunner pid=2001144) 'hf_config_path': None,
(TaskRunner pid=2001144) 'lora_adapter_path': None,
(TaskRunner pid=2001144) 'lora_alpha': 16,
(TaskRunner pid=2001144) 'lora_rank': 0,
(TaskRunner pid=2001144) 'override_config': {},
(TaskRunner pid=2001144) 'path': 'Qwen/Qwen3-4B',
(TaskRunner pid=2001144) 'target_modules': 'all-linear',
(TaskRunner pid=2001144) 'tiled_mlp': {'enabled': False,
(TaskRunner pid=2001144) 'num_shards': 4},
(TaskRunner pid=2001144) 'tokenizer_path': None,
(TaskRunner pid=2001144) 'trust_remote_code': True,
(TaskRunner pid=2001144) 'use_fused_kernels': False,
(TaskRunner pid=2001144) 'use_liger': False,
(TaskRunner pid=2001144) 'use_remove_padding': True,
(TaskRunner pid=2001144) 'use_shm': False},
(TaskRunner pid=2001144) 'nccl_timeout': 600,
(TaskRunner pid=2001144) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig',
(TaskRunner pid=2001144) 'entropy_checkpointing': False,
(TaskRunner pid=2001144) 'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2001144) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2001144) 'dtype': 'bfloat16',
(TaskRunner pid=2001144) 'entropy_checkpointing': False,
(TaskRunner pid=2001144) 'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2001144) 'forward_only': True,
(TaskRunner pid=2001144) 'forward_prefetch': False,
(TaskRunner pid=2001144) 'fsdp_size': -1,
(TaskRunner pid=2001144) 'full_determinism': False,
(TaskRunner pid=2001144) 'model_dtype': 'fp32',
(TaskRunner pid=2001144) 'offload_policy': False,
(TaskRunner pid=2001144) 'optimizer_offload': False,
(TaskRunner pid=2001144) 'param_offload': False,
(TaskRunner pid=2001144) 'reshard_after_forward': True,
(TaskRunner pid=2001144) 'seed': 42,
(TaskRunner pid=2001144) 'strategy': 'fsdp',
(TaskRunner pid=2001144) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2001144) 'use_orig_params': False,
(TaskRunner pid=2001144) 'use_torch_compile': True,
(TaskRunner pid=2001144) 'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2001144) 'log_prob_max_token_len_per_gpu': 10240,
(TaskRunner pid=2001144) 'log_prob_micro_batch_size': None,
(TaskRunner pid=2001144) 'log_prob_micro_batch_size_per_gpu': 1,
(TaskRunner pid=2001144) 'log_prob_use_dynamic_bsz': False,
(TaskRunner pid=2001144) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2001144) 'all_ranks': False,
(TaskRunner pid=2001144) 'enable': False,
(TaskRunner pid=2001144) 'ranks': [],
(TaskRunner pid=2001144) 'save_path': 'outputs/profile',
(TaskRunner pid=2001144) 'tool': None,
(TaskRunner pid=2001144) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2001144) 'analysis': True,
(TaskRunner pid=2001144) 'contents': [],
(TaskRunner pid=2001144) 'discrete': False,
(TaskRunner pid=2001144) 'level': 'level0'},
(TaskRunner pid=2001144) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2001144) 'discrete': False},
(TaskRunner pid=2001144) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2001144) 'step_end': None,
(TaskRunner pid=2001144) 'step_start': 0},
(TaskRunner pid=2001144) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2001144) 'stack_depth': 32,
(TaskRunner pid=2001144) 'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2001144) 'rollout_n': 8,
(TaskRunner pid=2001144) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig',
(TaskRunner pid=2001144) 'mode': 'disabled',
(TaskRunner pid=2001144) 'record_file': None,
(TaskRunner pid=2001144) 'replay_file': None},
(TaskRunner pid=2001144) 'strategy': 'fsdp',
(TaskRunner pid=2001144) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2001144) 'use_torch_compile': True},
(TaskRunner pid=2001144) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig',
(TaskRunner pid=2001144) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig',
(TaskRunner pid=2001144) 'agent_loop_config_path': None,
(TaskRunner pid=2001144) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig',
(TaskRunner pid=2001144) 'name': None,
(TaskRunner pid=2001144) 'path': None},
(TaskRunner pid=2001144) 'default_agent_loop': 'single_turn_agent',
(TaskRunner pid=2001144) 'num_workers': 8},
(TaskRunner pid=2001144) 'calculate_log_probs': True,
(TaskRunner pid=2001144) 'cudagraph_capture_sizes': None,
(TaskRunner pid=2001144) 'data_parallel_size': 1,
(TaskRunner pid=2001144) 'disable_log_stats': True,
(TaskRunner pid=2001144) 'do_sample': True,
(TaskRunner pid=2001144) 'dtype': 'bfloat16',
(TaskRunner pid=2001144) 'enable_chunked_prefill': True,
(TaskRunner pid=2001144) 'enable_prefix_caching': True,
(TaskRunner pid=2001144) 'enable_rollout_routing_replay': False,
(TaskRunner pid=2001144) 'enforce_eager': False,
(TaskRunner pid=2001144) 'engine_kwargs': {'sglang': {}, 'vllm': {}},
(TaskRunner pid=2001144) 'expert_parallel_size': 1,
(TaskRunner pid=2001144) 'free_cache_engine': True,
(TaskRunner pid=2001144) 'gpu_memory_utilization': 0.8,
(TaskRunner pid=2001144) 'ignore_eos': False,
(TaskRunner pid=2001144) 'layered_summon': False,
(TaskRunner pid=2001144) 'load_format': 'dummy',
(TaskRunner pid=2001144) 'log_prob_max_token_len_per_gpu': 10240,
(TaskRunner pid=2001144) 'log_prob_micro_batch_size': None,
(TaskRunner pid=2001144) 'log_prob_micro_batch_size_per_gpu': 1,
(TaskRunner pid=2001144) 'log_prob_use_dynamic_bsz': False,
(TaskRunner pid=2001144) 'logprobs_mode': 'processed_logprobs',
(TaskRunner pid=2001144) 'max_model_len': 10240,
(TaskRunner pid=2001144) 'max_num_batched_tokens': 10240,
(TaskRunner pid=2001144) 'max_num_seqs': 1024,
(TaskRunner pid=2001144) 'mode': 'async',
(TaskRunner pid=2001144) 'multi_stage_wake_up': False,
(TaskRunner pid=2001144) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig',
(TaskRunner pid=2001144) 'enable': False,
(TaskRunner pid=2001144) 'format': 'hermes',
(TaskRunner pid=2001144) 'interaction_config_path': None,
(TaskRunner pid=2001144) 'max_assistant_turns': None,
(TaskRunner pid=2001144) 'max_parallel_calls': 1,
(TaskRunner pid=2001144) 'max_tool_response_length': 256,
(TaskRunner pid=2001144) 'max_user_turns': None,
(TaskRunner pid=2001144) 'num_repeat_rollouts': None,
(TaskRunner pid=2001144) 'tokenization_sanity_check_mode': 'strict',
(TaskRunner pid=2001144) 'tool_config_path': None,
(TaskRunner pid=2001144) 'tool_response_truncate_side': 'middle',
(TaskRunner pid=2001144) 'use_inference_chat_template': False},
(TaskRunner pid=2001144) 'n': 8,
(TaskRunner pid=2001144) 'name': 'vllm',
(TaskRunner pid=2001144) 'over_sample_rate': 0,
(TaskRunner pid=2001144) 'pipeline_model_parallel_size': 1,
(TaskRunner pid=2001144) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2001144) 'all_ranks': False,
(TaskRunner pid=2001144) 'enable': False,
(TaskRunner pid=2001144) 'ranks': [],
(TaskRunner pid=2001144) 'save_path': 'outputs/profile',
(TaskRunner pid=2001144) 'tool': None,
(TaskRunner pid=2001144) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2001144) 'analysis': True,
(TaskRunner pid=2001144) 'contents': [],
(TaskRunner pid=2001144) 'discrete': False,
(TaskRunner pid=2001144) 'level': 'level0'},
(TaskRunner pid=2001144) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2001144) 'discrete': False},
(TaskRunner pid=2001144) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2001144) 'step_end': None,
(TaskRunner pid=2001144) 'step_start': 0},
(TaskRunner pid=2001144) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2001144) 'stack_depth': 32,
(TaskRunner pid=2001144) 'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2001144) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig',
(TaskRunner pid=2001144) 'enable': False,
(TaskRunner pid=2001144) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml',
(TaskRunner pid=2001144) 'port': 9090,
(TaskRunner pid=2001144) 'served_model_name': 'Qwen/Qwen3-4B'},
(TaskRunner pid=2001144) 'prompt_length': 2048,
(TaskRunner pid=2001144) 'quantization': None,
(TaskRunner pid=2001144) 'quantization_config_file': None,
(TaskRunner pid=2001144) 'response_length': 8192,
(TaskRunner pid=2001144) 'scheduling_policy': 'fcfs',
(TaskRunner pid=2001144) 'skip_dump_dir': '/tmp/rollout_dump',
(TaskRunner pid=2001144) 'skip_rollout': False,
(TaskRunner pid=2001144) 'skip_tokenizer_init': True,
(TaskRunner pid=2001144) 'temperature': 1.0,
(TaskRunner pid=2001144) 'tensor_model_parallel_size': 2,
(TaskRunner pid=2001144) 'top_k': -1,
(TaskRunner pid=2001144) 'top_p': 1.0,
(TaskRunner pid=2001144) 'trace': {'_target_': 'verl.workers.config.TraceConfig',
(TaskRunner pid=2001144) 'backend': None,
(TaskRunner pid=2001144) 'max_samples_per_step_per_worker': None,
(TaskRunner pid=2001144) 'token2text': False},
(TaskRunner pid=2001144) 'update_weights_bucket_megabytes': 512,
(TaskRunner pid=2001144) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig',
(TaskRunner pid=2001144) 'do_sample': True,
(TaskRunner pid=2001144) 'n': 16,
(TaskRunner pid=2001144) 'temperature': 0.6,
(TaskRunner pid=2001144) 'top_k': -1,
(TaskRunner pid=2001144) 'top_p': 0.95}}},
(TaskRunner pid=2001144) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig',
(TaskRunner pid=2001144) 'adv_estimator': 'grpo',
(TaskRunner pid=2001144) 'gamma': 1.0,
(TaskRunner pid=2001144) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig',
(TaskRunner pid=2001144) 'horizon': 10000,
(TaskRunner pid=2001144) 'kl_coef': 0.001,
(TaskRunner pid=2001144) 'target_kl': 0.1,
(TaskRunner pid=2001144) 'type': 'fixed'},
(TaskRunner pid=2001144) 'kl_penalty': 'kl',
(TaskRunner pid=2001144) 'lam': 1.0,
(TaskRunner pid=2001144) 'norm_adv_by_std_in_grpo': False,
(TaskRunner pid=2001144) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0},
(TaskRunner pid=2001144) 'rollout_correction': {'bypass_mode': False,
(TaskRunner pid=2001144) 'loss_type': 'ppo_clip',
(TaskRunner pid=2001144) 'rollout_is': 'token',
(TaskRunner pid=2001144) 'rollout_is_batch_normalize': False,
(TaskRunner pid=2001144) 'rollout_is_threshold': 2.0,
(TaskRunner pid=2001144) 'rollout_rs': None,
(TaskRunner pid=2001144) 'rollout_rs_threshold': None},
(TaskRunner pid=2001144) 'use_kl_in_reward': False,
(TaskRunner pid=2001144) 'use_pf_ppo': False},
(TaskRunner pid=2001144) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig',
(TaskRunner pid=2001144) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig',
(TaskRunner pid=2001144) 'async_save': False,
(TaskRunner pid=2001144) 'load_contents': ['model', 'optimizer', 'extra'],
(TaskRunner pid=2001144) 'save_contents': ['model', 'optimizer', 'extra']},
(TaskRunner pid=2001144) 'cliprange_value': 0.5,
(TaskRunner pid=2001144) 'data_loader_seed': 42,
(TaskRunner pid=2001144) 'enable': None,
(TaskRunner pid=2001144) 'forward_max_token_len_per_gpu': 32768,
(TaskRunner pid=2001144) 'forward_micro_batch_size': None,
(TaskRunner pid=2001144) 'forward_micro_batch_size_per_gpu': None,
(TaskRunner pid=2001144) 'grad_clip': 1.0,
(TaskRunner pid=2001144) 'loss_agg_mode': 'token-mean',
(TaskRunner pid=2001144) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg',
(TaskRunner pid=2001144) 'enable_activation_offload': False,
(TaskRunner pid=2001144) 'enable_gradient_checkpointing': True,
(TaskRunner pid=2001144) 'external_lib': None,
(TaskRunner pid=2001144) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2001144) 'dtype': 'bfloat16',
(TaskRunner pid=2001144) 'entropy_checkpointing': False,
(TaskRunner pid=2001144) 'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2001144) 'forward_only': False,
(TaskRunner pid=2001144) 'forward_prefetch': False,
(TaskRunner pid=2001144) 'fsdp_size': -1,
(TaskRunner pid=2001144) 'full_determinism': False,
(TaskRunner pid=2001144) 'model_dtype': 'fp32',
(TaskRunner pid=2001144) 'offload_policy': False,
(TaskRunner pid=2001144) 'optimizer_offload': False,
(TaskRunner pid=2001144) 'param_offload': False,
(TaskRunner pid=2001144) 'reshard_after_forward': True,
(TaskRunner pid=2001144) 'seed': 42,
(TaskRunner pid=2001144) 'strategy': 'fsdp',
(TaskRunner pid=2001144) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2001144) 'use_orig_params': False,
(TaskRunner pid=2001144) 'use_torch_compile': True,
(TaskRunner pid=2001144) 'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2001144) 'lora_alpha': 16,
(TaskRunner pid=2001144) 'lora_rank': 0,
(TaskRunner pid=2001144) 'override_config': {},
(TaskRunner pid=2001144) 'path': 'Qwen/Qwen3-8B',
(TaskRunner pid=2001144) 'target_modules': 'all-linear',
(TaskRunner pid=2001144) 'tiled_mlp': {'enabled': False, 'num_shards': 4},
(TaskRunner pid=2001144) 'tokenizer_path': 'Qwen/Qwen3-4B',
(TaskRunner pid=2001144) 'trust_remote_code': True,
(TaskRunner pid=2001144) 'use_remove_padding': False,
(TaskRunner pid=2001144) 'use_shm': False},
(TaskRunner pid=2001144) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig',
(TaskRunner pid=2001144) 'betas': [0.9, 0.999],
(TaskRunner pid=2001144) 'clip_grad': 1.0,
(TaskRunner pid=2001144) 'lr': 1e-05,
(TaskRunner pid=2001144) 'lr_scheduler_type': 'constant',
(TaskRunner pid=2001144) 'lr_warmup_steps': -1,
(TaskRunner pid=2001144) 'lr_warmup_steps_ratio': 0.0,
(TaskRunner pid=2001144) 'min_lr_ratio': 0.0,
(TaskRunner pid=2001144) 'num_cycles': 0.5,
(TaskRunner pid=2001144) 'optimizer': 'AdamW',
(TaskRunner pid=2001144) 'optimizer_impl': 'torch.optim',
(TaskRunner pid=2001144) 'override_optimizer_config': None,
(TaskRunner pid=2001144) 'total_training_steps': -1,
(TaskRunner pid=2001144) 'warmup_style': None,
(TaskRunner pid=2001144) 'weight_decay': 0.01},
(TaskRunner pid=2001144) 'ppo_epochs': 1,
(TaskRunner pid=2001144) 'ppo_max_token_len_per_gpu': 32768,
(TaskRunner pid=2001144) 'ppo_micro_batch_size': None,
(TaskRunner pid=2001144) 'ppo_micro_batch_size_per_gpu': None,
(TaskRunner pid=2001144) 'ppo_mini_batch_size': 8,
(TaskRunner pid=2001144) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2001144) 'all_ranks': False,
(TaskRunner pid=2001144) 'enable': False,
(TaskRunner pid=2001144) 'ranks': [],
(TaskRunner pid=2001144) 'save_path': 'outputs/profile',
(TaskRunner pid=2001144) 'tool': None,
(TaskRunner pid=2001144) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2001144) 'analysis': True,
(TaskRunner pid=2001144) 'contents': [],
(TaskRunner pid=2001144) 'discrete': False,
(TaskRunner pid=2001144) 'level': 'level0'},
(TaskRunner pid=2001144) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2001144) 'discrete': False},
(TaskRunner pid=2001144) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2001144) 'step_end': None,
(TaskRunner pid=2001144) 'step_start': 0},
(TaskRunner pid=2001144) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2001144) 'stack_depth': 32,
(TaskRunner pid=2001144) 'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2001144) 'rollout_n': 8,
(TaskRunner pid=2001144) 'shuffle': False,
(TaskRunner pid=2001144) 'strategy': 'fsdp',
(TaskRunner pid=2001144) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2001144) 'use_dynamic_bsz': False},
(TaskRunner pid=2001144) 'custom_reward_function': {'name': 'compute_score',
(TaskRunner pid=2001144) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'},
(TaskRunner pid=2001144) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False},
(TaskRunner pid=2001144) 'custom_cls': {'name': None, 'path': None},
(TaskRunner pid=2001144) 'datagen': {'name': None, 'path': None},
(TaskRunner pid=2001144) 'dataloader_num_workers': 8,
(TaskRunner pid=2001144) 'filter_overlong_prompts': True,
(TaskRunner pid=2001144) 'filter_overlong_prompts_workers': 1,
(TaskRunner pid=2001144) 'image_key': 'images',
(TaskRunner pid=2001144) 'image_patch_size': 14,
(TaskRunner pid=2001144) 'max_prompt_length': 2048,
(TaskRunner pid=2001144) 'max_response_length': 8192,
(TaskRunner pid=2001144) 'prompt_key': 'prompt',
(TaskRunner pid=2001144) 'return_full_prompt': False,
(TaskRunner pid=2001144) 'return_multi_modal_inputs': True,
(TaskRunner pid=2001144) 'return_raw_chat': True,
(TaskRunner pid=2001144) 'return_raw_input_ids': False,
(TaskRunner pid=2001144) 'reward_fn_key': 'data_source',
(TaskRunner pid=2001144) 'sampler': {'class_name': None, 'class_path': None},
(TaskRunner pid=2001144) 'seed': None,
(TaskRunner pid=2001144) 'shuffle': True,
(TaskRunner pid=2001144) 'tokenizer': None,
(TaskRunner pid=2001144) 'tool_config_path': None,
(TaskRunner pid=2001144) 'train_batch_size': 32,
(TaskRunner pid=2001144) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet'],
(TaskRunner pid=2001144) 'train_max_samples': 3200,
(TaskRunner pid=2001144) 'truncation': 'error',
(TaskRunner pid=2001144) 'trust_remote_code': True,
(TaskRunner pid=2001144) 'use_shm': False,
(TaskRunner pid=2001144) 'val_batch_size': None,
(TaskRunner pid=2001144) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet'],
(TaskRunner pid=2001144) 'val_max_samples': -1,
(TaskRunner pid=2001144) 'validation_shuffle': False,
(TaskRunner pid=2001144) 'video_key': 'videos'},
(TaskRunner pid=2001144) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2001144) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2001144) 'controller_nsight_options': {'cuda-graph-trace': 'graph',
(TaskRunner pid=2001144) 'cuda-memory-usage': 'true',
(TaskRunner pid=2001144) 'trace': 'cuda,nvtx,cublas,ucx'},
(TaskRunner pid=2001144) 'discrete': False,
(TaskRunner pid=2001144) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi',
(TaskRunner pid=2001144) 'capture-range-end': None,
(TaskRunner pid=2001144) 'cuda-graph-trace': 'graph',
(TaskRunner pid=2001144) 'cuda-memory-usage': 'true',
(TaskRunner pid=2001144) 'kill': 'none',
(TaskRunner pid=2001144) 'trace': 'cuda,nvtx,cublas,ucx'}},
(TaskRunner pid=2001144) 'torch_memory': {'context': 'all',
(TaskRunner pid=2001144) 'kw_args': {},
(TaskRunner pid=2001144) 'stack_depth': 32,
(TaskRunner pid=2001144) 'stacks': 'all',
(TaskRunner pid=2001144) 'trace_alloc_max_entries': 100000}},
(TaskRunner pid=2001144) 'profile_continuous_steps': False,
(TaskRunner pid=2001144) 'save_path': 'outputs/profile',
(TaskRunner pid=2001144) 'steps': None,
(TaskRunner pid=2001144) 'tool': None},
(TaskRunner pid=2001144) 'max_model_len': 10240,
(TaskRunner pid=2001144) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_chemistry_grpo_Qwen3_4B',
(TaskRunner pid=2001144) 'include_dashboard': False,
(TaskRunner pid=2001144) 'num_cpus': None,
(TaskRunner pid=2001144) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8',
(TaskRunner pid=2001144) 'TASK': 'datasets/sciknoweval/chemistry',
(TaskRunner pid=2001144) 'USER': 'root'}}},
(TaskRunner pid=2001144) 'timeline_json_file': None},
(TaskRunner pid=2001144) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig',
(TaskRunner pid=2001144) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig',
(TaskRunner pid=2001144) 'name': 'custom_reward_manager',
(TaskRunner pid=2001144) 'path': None},
(TaskRunner pid=2001144) 'name': 'naive',
(TaskRunner pid=2001144) 'source': 'register'},
(TaskRunner pid=2001144) 'reward_model': {'enable': False,
(TaskRunner pid=2001144) 'enable_resource_pool': False,
(TaskRunner pid=2001144) 'forward_max_token_len_per_gpu': 32768,
(TaskRunner pid=2001144) 'launch_reward_fn_async': False,
(TaskRunner pid=2001144) 'max_length': None,
(TaskRunner pid=2001144) 'micro_batch_size': None,
(TaskRunner pid=2001144) 'micro_batch_size_per_gpu': None,
(TaskRunner pid=2001144) 'model': {'external_lib': None,
(TaskRunner pid=2001144) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2001144) 'forward_prefetch': False,
(TaskRunner pid=2001144) 'fsdp_size': -1,
(TaskRunner pid=2001144) 'param_offload': False,
(TaskRunner pid=2001144) 'reshard_after_forward': True,
(TaskRunner pid=2001144) 'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2001144) 'input_tokenizer': 'Qwen/Qwen3-4B',
(TaskRunner pid=2001144) 'override_config': {},
(TaskRunner pid=2001144) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1',
(TaskRunner pid=2001144) 'trust_remote_code': False,
(TaskRunner pid=2001144) 'use_fused_kernels': False,
(TaskRunner pid=2001144) 'use_remove_padding': False,
(TaskRunner pid=2001144) 'use_shm': False},
(TaskRunner pid=2001144) 'n_gpus_per_node': 8,
(TaskRunner pid=2001144) 'nnodes': 0,
(TaskRunner pid=2001144) 'num_workers': 1,
(TaskRunner pid=2001144) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2001144) 'all_ranks': False,
(TaskRunner pid=2001144) 'enable': False,
(TaskRunner pid=2001144) 'ranks': [],
(TaskRunner pid=2001144) 'save_path': 'outputs/profile',
(TaskRunner pid=2001144) 'tool': None,
(TaskRunner pid=2001144) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2001144) 'analysis': True,
(TaskRunner pid=2001144) 'contents': [],
(TaskRunner pid=2001144) 'discrete': False,
(TaskRunner pid=2001144) 'level': 'level0'},
(TaskRunner pid=2001144) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2001144) 'discrete': False},
(TaskRunner pid=2001144) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2001144) 'step_end': None,
(TaskRunner pid=2001144) 'step_start': 0},
(TaskRunner pid=2001144) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2001144) 'stack_depth': 32,
(TaskRunner pid=2001144) 'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2001144) 'reward_loop_class_name': None,
(TaskRunner pid=2001144) 'reward_loop_module_path': None,
(TaskRunner pid=2001144) 'reward_loop_source': 'register',
(TaskRunner pid=2001144) 'reward_manager': 'naive',
(TaskRunner pid=2001144) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig',
(TaskRunner pid=2001144) 'cudagraph_capture_sizes': None,
(TaskRunner pid=2001144) 'data_parallel_size': 1,
(TaskRunner pid=2001144) 'disable_log_stats': True,
(TaskRunner pid=2001144) 'dtype': 'bfloat16',
(TaskRunner pid=2001144) 'enable_chunked_prefill': True,
(TaskRunner pid=2001144) 'enable_prefix_caching': True,
(TaskRunner pid=2001144) 'enforce_eager': True,
(TaskRunner pid=2001144) 'engine_kwargs': {},
(TaskRunner pid=2001144) 'expert_parallel_size': 1,
(TaskRunner pid=2001144) 'free_cache_engine': True,
(TaskRunner pid=2001144) 'gpu_memory_utilization': 0.5,
(TaskRunner pid=2001144) 'limit_images': None,
(TaskRunner pid=2001144) 'load_format': 'auto',
(TaskRunner pid=2001144) 'max_model_len': None,
(TaskRunner pid=2001144) 'max_num_batched_tokens': 8192,
(TaskRunner pid=2001144) 'max_num_seqs': 1024,
(TaskRunner pid=2001144) 'name': '???',
(TaskRunner pid=2001144) 'prompt_length': 2048,
(TaskRunner pid=2001144) 'response_length': 2048,
(TaskRunner pid=2001144) 'skip_tokenizer_init': False,
(TaskRunner pid=2001144) 'tensor_model_parallel_size': 2},
(TaskRunner pid=2001144) 'sandbox_fusion': {'max_concurrent': 64,
(TaskRunner pid=2001144) 'memory_limit_mb': 1024,
(TaskRunner pid=2001144) 'url': None},
(TaskRunner pid=2001144) 'strategy': 'fsdp',
(TaskRunner pid=2001144) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2001144) 'use_dynamic_bsz': False,
(TaskRunner pid=2001144) 'use_reward_loop': False},
(TaskRunner pid=2001144) 'trainer': {'balance_batch': True,
(TaskRunner pid=2001144) 'critic_warmup': 0,
(TaskRunner pid=2001144) 'default_hdfs_dir': None,
(TaskRunner pid=2001144) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8',
(TaskRunner pid=2001144) 'del_local_ckpt_after_load': False,
(TaskRunner pid=2001144) 'device': 'cuda',
(TaskRunner pid=2001144) 'esi_redundant_time': 0,
(TaskRunner pid=2001144) 'experiment_name': 'qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8',
(TaskRunner pid=2001144) 'group_name': 'QWEN3-GRPO-generalization',
(TaskRunner pid=2001144) 'log_val_generations': 0,
(TaskRunner pid=2001144) 'logger': ['console', 'wandb'],
(TaskRunner pid=2001144) 'max_actor_ckpt_to_keep': 1,
(TaskRunner pid=2001144) 'max_critic_ckpt_to_keep': None,
(TaskRunner pid=2001144) 'n_gpus_per_node': 8,
(TaskRunner pid=2001144) 'nnodes': 1,
(TaskRunner pid=2001144) 'project_name': 'SDPO-root',
(TaskRunner pid=2001144) 'ray_wait_register_center_timeout': 300,
(TaskRunner pid=2001144) 'resume_from_path': None,
(TaskRunner pid=2001144) 'resume_mode': 'auto',
(TaskRunner pid=2001144) 'rollout_data_dir': None,
(TaskRunner pid=2001144) 'save_freq': 10,
(TaskRunner pid=2001144) 'test_freq': 10,
(TaskRunner pid=2001144) 'total_epochs': 30,
(TaskRunner pid=2001144) 'total_training_steps': 100,
(TaskRunner pid=2001144) 'use_legacy_worker_impl': 'auto',
(TaskRunner pid=2001144) 'val_before_train': False,
(TaskRunner pid=2001144) 'val_only': False,
(TaskRunner pid=2001144) 'validation_data_dir': None},
(TaskRunner pid=2001144) 'transfer_queue': {'enable': False},
(TaskRunner pid=2001144) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/chemistry',
(TaskRunner pid=2001144) 'dir': '/users/root/SDPO',
(TaskRunner pid=2001144) 'log_dir': '/users/root/output',
(TaskRunner pid=2001144) 'task': 'datasets/sciknoweval/chemistry'}}
(TaskRunner pid=2001144) [validate_config] All configuration checks passed successfully!
(TaskRunner pid=2001144) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True
(TaskRunner pid=2001144) use_critic=need_critic(config),
(TaskRunner pid=2001144) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now.
(TaskRunner pid=2001144) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1)
(TaskRunner pid=2001144) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(TaskRunner pid=2001144) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(TaskRunner pid=2001144) Using dataset class: RLHFDataset
(TaskRunner pid=2001144) dataset len: 1890
(TaskRunner pid=2001144) Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2001144) WARNING:2026-07-02 05:06:02,228:Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2001144) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/1890 [00:00<?, ? examples/s]
(TaskRunner pid=2001144) Filtering prompts longer than 2048 tokens (num_proc=1): 53%|█████▎ | 1000/1890 [00:00<00:00, 1082.92 examples/s]
(TaskRunner pid=2001144) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 1890/1890 [00:01<00:00, 1537.63 examples/s]
(TaskRunner pid=2001144) filter dataset len: 1890
(TaskRunner pid=2001144) Using dataset class: RLHFDataset
(TaskRunner pid=2001144) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 1890/1890 [00:01<00:00, 1333.84 examples/s]
(TaskRunner pid=2001144) dataset len: 210
(TaskRunner pid=2001144) Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2001144) WARNING:2026-07-02 05:06:04,078:Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2001144) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/210 [00:00<?, ? examples/s]
(TaskRunner pid=2001144) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 210/210 [00:00<00:00, 363.62 examples/s]
(TaskRunner pid=2001144) filter dataset len: 210
(TaskRunner pid=2001144) Size of train dataloader: 59, Size of val dataloader: 1
(TaskRunner pid=2001144) Total training steps: 100
(TaskRunner pid=2001144) colocated worker base class <class 'verl.single_controller.base.worker.Worker'>
(TaskRunner pid=2001144) bind role actor_rollout method chat_completion to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2001144) bind role actor_rollout method generate to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2001144) bind role actor_rollout method get_zeromq_address to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2001144) bind role actor_rollout method sleep to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2001144) bind role actor_rollout method wake_up to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2001144) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 210/210 [00:00<00:00, 313.37 examples/s]
(TaskRunner pid=2001144) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True
(TaskRunner pid=2001144) self.use_critic = need_critic(self.config)
(WorkerDict pid=2001538) [W702 05:06:10.102061737 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:50097 (errno: 97 - Address family not supported by protocol).
(WorkerDict pid=2001534) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7
(WorkerDict pid=2001541) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(WorkerDict pid=2001541) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(WorkerDict pid=2001544) [W702 05:06:12.623076441 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:50097 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.)
(WorkerDict pid=2001534) Model config after override: Qwen3Config {
(WorkerDict pid=2001534) "architectures": [
(WorkerDict pid=2001534) "Qwen3ForCausalLM"
(WorkerDict pid=2001534) ],
(WorkerDict pid=2001534) "attention_bias": false,
(WorkerDict pid=2001534) "attention_dropout": 0.0,
(WorkerDict pid=2001534) "dtype": "bfloat16",
(WorkerDict pid=2001534) "eos_token_id": 151645,
(WorkerDict pid=2001534) "head_dim": 128,
(WorkerDict pid=2001534) "hidden_act": "silu",
(WorkerDict pid=2001534) "hidden_size": 2560,
(WorkerDict pid=2001534) "initializer_range": 0.02,
(WorkerDict pid=2001534) "intermediate_size": 9728,
(WorkerDict pid=2001534) "layer_types": [
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention",
(WorkerDict pid=2001534) "full_attention"
(WorkerDict pid=2001534) ],
(WorkerDict pid=2001534) "max_position_embeddings": 40960,
(WorkerDict pid=2001534) "max_window_layers": 36,
(WorkerDict pid=2001534) "model_type": "qwen3",
(WorkerDict pid=2001534) "num_attention_heads": 32,
(WorkerDict pid=2001534) "num_hidden_layers": 36,
(WorkerDict pid=2001534) "num_key_value_heads": 8,
(WorkerDict pid=2001534) "pad_token_id": 151643,
(WorkerDict pid=2001534) "rms_norm_eps": 1e-06,
(WorkerDict pid=2001534) "rope_scaling": null,
(WorkerDict pid=2001534) "rope_theta": 1000000,
(WorkerDict pid=2001534) "sliding_window": null,
(WorkerDict pid=2001534) "tie_word_embeddings": true,
(WorkerDict pid=2001534) "transformers_version": "4.57.1",
(WorkerDict pid=2001534) "use_cache": true,
(WorkerDict pid=2001534) "use_sliding_window": false,
(WorkerDict pid=2001534) "vocab_size": 151936
(WorkerDict pid=2001534) }
(WorkerDict pid=2001534)
(WorkerDict pid=2001542) `torch_dtype` is deprecated! Use `dtype` instead!
(WorkerDict pid=2001542) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)`
(WorkerDict pid=2001542) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)`
(WorkerDict pid=2001541) Loading checkpoint shards: 0%| | 0/3 [00:00<?, ?it/s]
(WorkerDict pid=2001542) Loading checkpoint shards: 33%|███▎ | 1/3 [00:02<00:05, 2.88s/it]
(WorkerDict pid=2001545) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster]
(WorkerDict pid=2001545) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster]
(WorkerDict pid=2001545) `torch_dtype` is deprecated! Use `dtype` instead! [repeated 7x across cluster]
(WorkerDict pid=2001545) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` [repeated 14x across cluster]
(WorkerDict pid=2001538) Loading checkpoint shards: 0%| | 0/3 [00:00<?, ?it/s] [repeated 7x across cluster]
(WorkerDict pid=2001546) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.94s/it]
(WorkerDict pid=2001534) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:03, 3.00s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:06<00:00, 2.02s/it]
(WorkerDict pid=2001542) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention
(WorkerDict pid=2001542) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch
(WorkerDict pid=2001544) [Gloo] Rank 5 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 [repeated 7x across cluster]
(WorkerDict pid=2001534) Qwen3ForCausalLM contains 4.02B parameters
(WorkerDict pid=2001534) wrap_policy: functools.partial(<function _or_policy at 0x7e78cc1705e0>, policies=[functools.partial(<function transformer_auto_wrap_policy at 0x7e78cc1704a0>, transformer_layer_cls={<class 'transformers.models.qwen3.modeling_qwen3.Qwen3DecoderLayer'>})])
(WorkerDict pid=2001534) NCCL version 2.27.5+cuda12.9
(WorkerDict pid=2001538) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now.
(WorkerDict pid=2001538) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1)
(WorkerDict pid=2001544) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.91s/it] [repeated 12x across cluster]
(WorkerDict pid=2001544) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.96s/it] [repeated 4x across cluster]
(WorkerDict pid=2001545) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.86s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.93s/it] [repeated 2x across cluster]
(WorkerDict pid=2001534) Total steps: 100, num_warmup_steps: 10
(WorkerDict pid=2001534) Actor use_remove_padding=True
(WorkerDict pid=2001534) Actor use_fused_kernels=False
(WorkerDict pid=2001534) Actor use_prefix_grouper=False
(WorkerDict pid=2001538) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster]
(WorkerDict pid=2001538) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster]
(WorkerDict pid=2001538) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(WorkerDict pid=2001538) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(WorkerDict pid=2001538) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3
(WorkerDict pid=2001534) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html .
(WorkerDict pid=2001534) warnings.warn(
(WorkerDict pid=2001546) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster]
(WorkerDict pid=2001546) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster]
(WorkerDict pid=2001541) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster]
(WorkerDict pid=2001541) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster]
(WorkerDict pid=2001543) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster]
(WorkerDict pid=2001543) warnings.warn( [repeated 7x across cluster]
(TaskRunner pid=2001144) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now.
(TaskRunner pid=2001144) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1)
(vLLMHttpServer pid=2002422) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(vLLMHttpServer pid=2002422) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(vLLMHttpServer pid=2002422) ['serve',
(vLLMHttpServer pid=2002422) 'Qwen/Qwen3-4B',
(vLLMHttpServer pid=2002422) '--dtype',
(vLLMHttpServer pid=2002422) 'bfloat16',
(vLLMHttpServer pid=2002422) '--load_format',
(vLLMHttpServer pid=2002422) 'dummy',
(vLLMHttpServer pid=2002422) '--trust_remote_code',
(vLLMHttpServer pid=2002422) '--max_model_len',
(vLLMHttpServer pid=2002422) '40960',
(vLLMHttpServer pid=2002422) '--max_num_seqs',
(vLLMHttpServer pid=2002422) '1024',
(vLLMHttpServer pid=2002422) '--enable_chunked_prefill',
(vLLMHttpServer pid=2002422) '--max_num_batched_tokens',
(vLLMHttpServer pid=2002422) '10240',
(vLLMHttpServer pid=2002422) '--enable_prefix_caching',
(vLLMHttpServer pid=2002422) '--enable_sleep_mode',
(vLLMHttpServer pid=2002422) '--logprobs_mode',
(vLLMHttpServer pid=2002422) 'processed_logprobs',
(vLLMHttpServer pid=2002422) '--disable_custom_all_reduce',
(vLLMHttpServer pid=2002422) '--gpu_memory_utilization',
(vLLMHttpServer pid=2002422) '0.8',
(vLLMHttpServer pid=2002422) '--disable_log_stats',
(vLLMHttpServer pid=2002422) '--tensor_parallel_size',
(vLLMHttpServer pid=2002422) '2',
(vLLMHttpServer pid=2002422) '--seed',
(vLLMHttpServer pid=2002422) '0',
(vLLMHttpServer pid=2002422) '--override_generation_config',
(vLLMHttpServer pid=2002422) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, '
(vLLMHttpServer pid=2002422) '"max_new_tokens": 8192}',
(vLLMHttpServer pid=2002422) '--hf_overrides',
(vLLMHttpServer pid=2002422) '{}',
(vLLMHttpServer pid=2002422) '--scheduling_policy',
(vLLMHttpServer pid=2002422) 'fcfs']
(WorkerDict pid=2001544) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster]
(vLLMHttpServer pid=2002422) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead.
(vLLMHttpServer pid=2002422) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(vLLMHttpServer pid=2002423) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(vLLMHttpServer pid=2002423) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(vLLMHttpServer pid=2002422) WARNING 07-02 05:06:54 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned
(WorkerDict pid=2001534) WARNING 07-02 05:07:02 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'.
(vLLMHttpServer pid=2002425) WARNING 07-02 05:06:55 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster]
(WorkerDict pid=2001543) NCCL version 2.27.5+cuda12.9
(WorkerDict pid=2001543) [rank4]:W0702 05:07:14.209000 2001543 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph
(WorkerDict pid=2001543) [rank4]:W0702 05:07:14.209000 2001543 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last):
(WorkerDict pid=2001543) [rank4]:W0702 05:07:14.209000 2001543 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates
(WorkerDict pid=2001543) [rank4]:W0702 05:07:14.209000 2001543 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f:
(WorkerDict pid=2001543) [rank4]:W0702 05:07:14.209000 2001543 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(WorkerDict pid=2001543) [rank4]:W0702 05:07:14.209000 2001543 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/fxgraph/vs/fvscv5dg7y3y3dcup4ddfpgbmpglg73u5zi54vpf2q4orgekbtq7/.2001542.127996398335680.tmp'
(vLLMHttpServer pid=2002425) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster]
(vLLMHttpServer pid=2002425) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster]
(vLLMHttpServer pid=2002425) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 2x across cluster]
(vLLMHttpServer pid=2002425) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 2x across cluster]
(WorkerDict pid=2001545) 2026-07-02 05:07:16,641 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ...
(WorkerDict pid=2001545) 2026-07-02 05:07:16,661 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00<?, ?it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 4%|▍ | 2/51 [00:00<00:02, 17.93it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 8%|▊ | 4/51 [00:00<00:02, 18.25it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 12%|█▏ | 6/51 [00:00<00:02, 18.01it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 16%|█▌ | 8/51 [00:00<00:02, 18.31it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 20%|█▉ | 10/51 [00:00<00:02, 17.76it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 24%|██▎ | 12/51 [00:00<00:02, 17.45it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 27%|██▋ | 14/51 [00:00<00:02, 17.22it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 31%|███▏ | 16/51 [00:00<00:02, 17.08it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 35%|███▌ | 18/51 [00:01<00:01, 16.75it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 39%|███▉ | 20/51 [00:01<00:01, 16.30it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 43%|████▎ | 22/51 [00:01<00:01, 16.12it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 47%|████▋ | 24/51 [00:01<00:01, 16.01it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 51%|█████ | 26/51 [00:01<00:01, 15.73it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 55%|█████▍ | 28/51 [00:01<00:01, 15.73it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 59%|█████▉ | 30/51 [00:01<00:01, 15.71it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 63%|██████▎ | 32/51 [00:01<00:01, 15.59it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 67%|██████▋ | 34/51 [00:02<00:01, 15.21it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 71%|███████ | 36/51 [00:02<00:00, 15.11it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 75%|███████▍ | 38/51 [00:02<00:00, 15.01it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 78%|███████▊ | 40/51 [00:02<00:00, 14.51it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 82%|████████▏ | 42/51 [00:02<00:00, 14.76it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 86%|████████▋ | 44/51 [00:02<00:00, 14.81it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 90%|█████████ | 46/51 [00:02<00:00, 14.75it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 94%|█████████▍| 48/51 [00:03<00:00, 14.78it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 98%|█████████▊| 50/51 [00:03<00:00, 15.28it/s] Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 51/51 [00:03<00:00, 15.78it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (decode, FULL): 0%| | 0/51 [00:00<?, ?it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (decode, FULL): 6%|▌ | 3/51 [00:00<00:02, 21.44it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (decode, FULL): 12%|█▏ | 6/51 [00:00<00:01, 22.88it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (decode, FULL): 18%|█▊ | 9/51 [00:00<00:01, 23.28it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (decode, FULL): 24%|██▎ | 12/51 [00:00<00:01, 23.00it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (decode, FULL): 29%|██▉ | 15/51 [00:00<00:01, 23.09it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (decode, FULL): 35%|███▌ | 18/51 [00:00<00:01, 23.14it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (decode, FULL): 41%|████ | 21/51 [00:00<00:01, 23.22it/s]
(WorkerDict pid=2001544) 2026-07-02 05:07:18,031 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... [repeated 7x across cluster]
(WorkerDict pid=2001544) 2026-07-02 05:07:18,066 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends [repeated 7x across cluster]
(WorkerDict pid=2001534) Capturing CUDA graphs (decode, FULL): 47%|████▋ | 24/51 [00:01<00:01, 22.81it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (decode, FULL): 53%|█████▎ | 27/51 [00:01<00:01, 22.99it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (decode, FULL): 59%|█████▉ | 30/51 [00:01<00:00, 23.10it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (decode, FULL): 65%|██████▍ | 33/51 [00:01<00:00, 22.87it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (decode, FULL): 71%|███████ | 36/51 [00:01<00:00, 21.40it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (decode, FULL): 76%|███████▋ | 39/51 [00:01<00:00, 21.17it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (decode, FULL): 82%|████████▏ | 42/51 [00:01<00:00, 21.86it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (decode, FULL): 88%|████████▊ | 45/51 [00:02<00:00, 20.38it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (decode, FULL): 94%|█████████▍| 48/51 [00:02<00:00, 20.96it/s]
(WorkerDict pid=2001534) Capturing CUDA graphs (decode, FULL): 100%|██████████| 51/51 [00:02<00:00, 21.63it/s] Capturing CUDA graphs (decode, FULL): 100%|██████████| 51/51 [00:02<00:00, 22.14it/s]
(vLLMHttpServer pid=2002425) WARNING 07-02 05:07:24 [api_server.py:1358] LoRA dynamic loading & unloading is enabled in the API server. This should ONLY be used for local development!
(WorkerDict pid=2001541) WARNING 07-02 05:07:03 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. [repeated 7x across cluster]
(WorkerDict pid=2001541) NCCL version 2.27.5+cuda12.9 [repeated 2x across cluster]
(vLLMHttpServer pid=2002425) WARNING 07-02 05:07:25 [model.py:1576] Default sampling parameters have been overridden by the model's Hugging Face generation config recommended from the model creator. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`.
(TaskRunner pid=2001144) AgentLoopManager: ['198.19.44.212:33781', '198.19.44.212:38613', '198.19.44.212:45203', '198.19.44.212:42559']
(TaskRunner pid=2001144) wandb: Currently logged in as: seongryongjung (seongryongjung-chung-ang-university) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
(TaskRunner pid=2001144) wandb: setting up run 0ozhfamy
(TaskRunner pid=2001144) wandb: Tracking run with wandb version 0.23.1
(TaskRunner pid=2001144) wandb: Run data is saved locally in /mnt/mole/SDPO/L2T/wandb/run-20260702_050732-0ozhfamy
(TaskRunner pid=2001144) wandb: Run `wandb offline` to turn off syncing.
(TaskRunner pid=2001144) wandb: Syncing run qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8
(TaskRunner pid=2001144) wandb: ⭐️ View project at https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root
(TaskRunner pid=2001144) wandb: 🚀 View run at https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/0ozhfamy
(TaskRunner pid=2001144) Checkpoint tracker file does not exist: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/latest_checkpointed_iteration.txt
(TaskRunner pid=2001144) Training from scratch
(vLLMHttpServer pid=2002424) WARNING 07-02 05:07:26 [api_server.py:1358] LoRA dynamic loading & unloading is enabled in the API server. This should ONLY be used for local development! [repeated 3x across cluster]
(vLLMHttpServer pid=2002424) WARNING 07-02 05:07:27 [model.py:1576] Default sampling parameters have been overridden by the model's Hugging Face generation config recommended from the model creator. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`. [repeated 3x across cluster]
(TaskRunner pid=2001144) wandb: Detected [openai] in use.
(TaskRunner pid=2001144) wandb: Use W&B Weave for improved LLM call tracing. Install Weave with `pip install weave` then add `import weave` to the top of your script.
(TaskRunner pid=2001144) wandb: For more information, check out the docs at: https://weave-docs.wandb.ai/
(TaskRunner pid=2001144) Training Progress: 0%| | 0/100 [00:00<?, ?it/s]
(AgentLoopWorker pid=2003383) Using dataset class: RLHFDataset
(AgentLoopWorker pid=2003383) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(AgentLoopWorker pid=2003383) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(AgentLoopWorker pid=2003389) You're using a Qwen2TokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.
(AgentLoopWorker pid=2003391) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster]
(AgentLoopWorker pid=2003391) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster]
(TaskRunner pid=2001144) step:1 - global_seqlen/min:15341 - global_seqlen/max:19975 - global_seqlen/minmax_diff:4634 - global_seqlen/balanced_min:18570 - global_seqlen/balanced_max:18575 - global_seqlen/mean:18573.125 - actor/entropy:0.21792373061180115 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.560697078704834 - training/rollout_probs_diff_mean:0.00600972305983305 - training/rollout_probs_diff_std:0.01622072234749794 - training/rollout_actor_probs_pearson_corr:0.9973068237304688 - rollout_corr/rollout_is_mean:0.9999498128890991 - rollout_corr/rollout_is_ratio_fraction_high:3.6347333661979064e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014538933464791626 - rollout_corr/rollout_is_max:3.8316073417663574 - rollout_corr/rollout_is_min:0.1732623279094696 - rollout_corr/rollout_is_std:0.04737083241343498 - rollout_corr/rollout_is_eff_sample_size:0.9977609098996091 - rollout_corr/rollout_is_seq_mean:0.9999146461486816 - rollout_corr/rollout_is_seq_std:0.0028308711480349302 - rollout_corr/rollout_is_seq_max:1.0065586566925049 - rollout_corr/rollout_is_seq_min:0.9891539216041565 - rollout_corr/rollout_is_seq_max_deviation:0.010846078395843506 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2644061427563429) - rollout_corr/training_log_ppl:np.float64(0.23130364282405935) - rollout_corr/kl:np.float64(0.0013095591846621346) - rollout_corr/k3_kl:np.float64(0.0012242524317116477) - rollout_corr/rollout_ppl:np.float64(1.2627015500329435) - rollout_corr/rollout_log_ppl:np.float64(0.22999408110626973) - rollout_corr/log_ppl_diff:np.float64(0.0013095617177896202) - rollout_corr/log_ppl_abs_diff:np.float64(0.002432231092825532) - rollout_corr/log_ppl_diff_max:np.float64(0.014287501573562622) - rollout_corr/log_ppl_diff_min:np.float64(-0.00489102303981781) - rollout_corr/ppl_ratio:np.float64(1.0013147233985364) - rollout_corr/chi2_token:np.float64(0.0022797714918851852) - rollout_corr/chi2_seq:np.float64(0.6155039784498513) - actor/pg_loss:np.float64(0.00010700547136366367) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(0.0) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6368567198514938) - perf/mfu/actor:np.float64(0.05112901652756981) - perf/max_memory_allocated_gb:np.float64(116.22064924240112) - perf/max_memory_reserved_gb:np.float64(119.734375) - perf/cpu_memory_used_gb:np.float64(98.35462951660156) - actor/lr:np.float64(0.0) - training/global_step:1 - training/epoch:0 - critic/score/mean:0.57421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.57421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002030907431617379 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.002030907431617379 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:322.41015625 - response_length/max:823.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:322.41015625 - response_length_non_aborted/max:823.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.0 - prompt_length/max:491.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00026675499975681305 - timing_s/agent_loop/generate_sequences/min:np.float64(11.024557879194617) - timing_s/agent_loop/generate_sequences/max:np.float64(22.03078863210976) - timing_s/agent_loop/generate_sequences/mean:np.float64(16.95798079094675) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.03078863210976) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0)
(AgentLoopWorker pid=2003388) Using dataset class: RLHFDataset [repeated 7x across cluster]
(TaskRunner pid=2001144) Training Progress: 1%| | 1/100 [00:47<1:18:52, 47.81s/it]
(AgentLoopWorker pid=2003383) You're using a Qwen2TokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding. [repeated 7x across cluster]
(TaskRunner pid=2001144) step:2 - global_seqlen/min:17671 - global_seqlen/max:26891 - global_seqlen/minmax_diff:9220 - global_seqlen/balanced_min:20942 - global_seqlen/balanced_max:27804 - global_seqlen/mean:21800.5 - actor/entropy:0.1998014748096466 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4668242037296295 - training/rollout_probs_diff_mean:0.005503688473254442 - training/rollout_probs_diff_std:0.015470650047063828 - training/rollout_actor_probs_pearson_corr:0.9974149465560913 - rollout_corr/rollout_is_mean:0.999813437461853 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00010796804417623207 - rollout_corr/rollout_is_max:1.9347293376922607 - rollout_corr/rollout_is_min:0.10411839932203293 - rollout_corr/rollout_is_std:0.045086104422807693 - rollout_corr/rollout_is_eff_sample_size:0.9979706545561053 - rollout_corr/rollout_is_seq_mean:0.9998338222503662 - rollout_corr/rollout_is_seq_std:0.0029471437446773052 - rollout_corr/rollout_is_seq_max:1.0096304416656494 - rollout_corr/rollout_is_seq_min:0.9908748269081116 - rollout_corr/rollout_is_seq_max_deviation:0.009630441665649414 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2753069633617997) - rollout_corr/training_log_ppl:np.float64(0.23976701798255817) - rollout_corr/kl:np.float64(0.001319143736814965) - rollout_corr/k3_kl:np.float64(0.0011949871332888051) - rollout_corr/rollout_ppl:np.float64(1.2736200345680118) - rollout_corr/rollout_log_ppl:np.float64(0.23844787387542965) - rollout_corr/log_ppl_diff:np.float64(0.0013191441071285226) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024896592835830234) - rollout_corr/log_ppl_diff_max:np.float64(0.010564923286437988) - rollout_corr/log_ppl_diff_min:np.float64(-0.006165146827697754) - rollout_corr/ppl_ratio:np.float64(1.0013240850530565) - rollout_corr/chi2_token:np.float64(0.002125972881913185) - rollout_corr/chi2_seq:np.float64(0.5237184260040522) - actor/pg_loss:np.float64(-0.0002447298029437661) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000750460975268652) - actor/ppo_kl:np.float64(-6.13481997966403e-05) - actor/pg_clipfrac_lower:np.float64(1.2765523024427239e-05) - actor/grad_norm:np.float64(0.46729355305433273) - perf/mfu/actor:np.float64(0.05800634257584497) - perf/max_memory_allocated_gb:np.float64(123.23046827316284) - perf/max_memory_reserved_gb:np.float64(128.9921875) - perf/cpu_memory_used_gb:np.float64(98.91415882110596) - actor/lr:np.float64(1e-07) - training/global_step:2 - training/epoch:0 - critic/score/mean:0.3515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03626241534948349 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.03626241534948349 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:361.796875 - response_length/max:8192.0 - response_length/min:132.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:361.796875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:132.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:319.46875 - prompt_length/max:818.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00019069761037826538 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6073354799300432) - timing_s/agent_loop/generate_sequences/max:np.float64(50.03751926869154) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.823697601859749) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.03751926869154) - t
(TaskRunner pid=2001144) Training Progress: 2%|▏ | 2/100 [01:58<1:39:46, 61.09s/it]
(TaskRunner pid=2001144) step:3 - global_seqlen/min:17281 - global_seqlen/max:24260 - global_seqlen/minmax_diff:6979 - global_seqlen/balanced_min:20395 - global_seqlen/balanced_max:20398 - global_seqlen/mean:20397.125 - actor/entropy:0.23295670747756958 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9241909980773926 - training/rollout_probs_diff_mean:0.006103870924562216 - training/rollout_probs_diff_std:0.016071822494268417 - training/rollout_actor_probs_pearson_corr:0.9974639415740967 - rollout_corr/rollout_is_mean:1.0000509023666382 - rollout_corr/rollout_is_ratio_fraction_high:1.1662487850117031e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015161234478000551 - rollout_corr/rollout_is_max:2.08491849899292 - rollout_corr/rollout_is_min:0.033831071108579636 - rollout_corr/rollout_is_std:0.04714380204677582 - rollout_corr/rollout_is_eff_sample_size:0.9977828654513343 - rollout_corr/rollout_is_seq_mean:1.0001158714294434 - rollout_corr/rollout_is_seq_std:0.0027241308707743883 - rollout_corr/rollout_is_seq_max:1.0088725090026855 - rollout_corr/rollout_is_seq_min:0.9909912943840027 - rollout_corr/rollout_is_seq_max_deviation:0.009008705615997314 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2909106509760022) - rollout_corr/training_log_ppl:np.float64(0.25240021632635035) - rollout_corr/kl:np.float64(0.001061959089172837) - rollout_corr/k3_kl:np.float64(0.0012607591598907675) - rollout_corr/rollout_ppl:np.float64(1.2894891127943993) - rollout_corr/rollout_log_ppl:np.float64(0.2513382558245212) - rollout_corr/log_ppl_diff:np.float64(0.0010619605018291622) - rollout_corr/log_ppl_abs_diff:np.float64(0.002370319707551971) - rollout_corr/log_ppl_diff_max:np.float64(0.011879503726959229) - rollout_corr/log_ppl_diff_min:np.float64(-0.010377109050750732) - rollout_corr/ppl_ratio:np.float64(1.0010669911280274) - rollout_corr/chi2_token:np.float64(0.002813283819705248) - rollout_corr/chi2_seq:np.float64(1.7621169579215348) - actor/pg_loss:np.float64(-0.00013136141933500767) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009750944991537835) - actor/ppo_kl:np.float64(-7.251462441715262e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6780977249145508) - perf/mfu/actor:np.float64(0.05816728476221922) - perf/max_memory_allocated_gb:np.float64(123.23046827316284) - perf/max_memory_reserved_gb:np.float64(128.9921875) - perf/cpu_memory_used_gb:np.float64(98.82655382156372) - actor/lr:np.float64(2e-07) - training/global_step:3 - training/epoch:0 - critic/score/mean:0.4453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00393608957529068 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00393608957529068 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:334.94140625 - response_length/max:822.0 - response_length/min:130.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:334.94140625 - response_length_non_aborted/max:822.0 - response_length_non_aborted/min:130.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.46875 - prompt_length/max:557.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014040805399417877 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5433837957680225) - timing_s/agent_loop/generate_sequences/max:np.float64(6.467613570392132) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.614001794565411) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.467613570392132) - timing_s/agen
(TaskRunner pid=2001144) Training Progress: 3%|▎ | 3/100 [02:23<1:12:23, 44.77s/it]
(TaskRunner pid=2001144) step:4 - global_seqlen/min:15030 - global_seqlen/max:22807 - global_seqlen/minmax_diff:7777 - global_seqlen/balanced_min:19523 - global_seqlen/balanced_max:19527 - global_seqlen/mean:19526.0 - actor/entropy:0.2535804808139801 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29003456234931946 - training/rollout_probs_diff_mean:0.006516153924167156 - training/rollout_probs_diff_std:0.015992887318134308 - training/rollout_actor_probs_pearson_corr:0.997599720954895 - rollout_corr/rollout_is_mean:1.0002198219299316 - rollout_corr/rollout_is_ratio_fraction_high:3.8616002711933106e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.010400390252471e-05 - rollout_corr/rollout_is_max:3.050083637237549 - rollout_corr/rollout_is_min:0.38555577397346497 - rollout_corr/rollout_is_std:0.048566192388534546 - rollout_corr/rollout_is_eff_sample_size:0.9976477057966052 - rollout_corr/rollout_is_seq_mean:1.0001745223999023 - rollout_corr/rollout_is_seq_std:0.002962638158351183 - rollout_corr/rollout_is_seq_max:1.0089842081069946 - rollout_corr/rollout_is_seq_min:0.9875988364219666 - rollout_corr/rollout_is_seq_max_deviation:0.012401163578033447 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.309306774288416) - rollout_corr/training_log_ppl:np.float64(0.2669931099808309) - rollout_corr/kl:np.float64(0.001383627707099322) - rollout_corr/k3_kl:np.float64(0.0012187714478386624) - rollout_corr/rollout_ppl:np.float64(1.3074305579066277) - rollout_corr/rollout_log_ppl:np.float64(0.2656094823614694) - rollout_corr/log_ppl_diff:np.float64(0.001383627619361505) - rollout_corr/log_ppl_abs_diff:np.float64(0.002610180847113952) - rollout_corr/log_ppl_diff_max:np.float64(0.015292882919311523) - rollout_corr/log_ppl_diff_min:np.float64(-0.007626235485076904) - rollout_corr/ppl_ratio:np.float64(1.0013895512092859) - rollout_corr/chi2_token:np.float64(0.002081405371427536) - rollout_corr/chi2_seq:np.float64(0.8863824785221368) - actor/pg_loss:np.float64(0.00011153123341500759) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001363606254926708) - actor/ppo_kl:np.float64(0.0003070539827580432) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6546329408884048) - perf/mfu/actor:np.float64(0.056622341128102674) - perf/max_memory_allocated_gb:np.float64(123.23046827316284) - perf/max_memory_reserved_gb:np.float64(128.9921875) - perf/cpu_memory_used_gb:np.float64(98.90660190582275) - actor/lr:np.float64(3e-07) - training/global_step:4 - training/epoch:0 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0012003140291199088 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0012003140291199088 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:303.46875 - response_length/max:660.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:303.46875 - response_length_non_aborted/max:660.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:306.71875 - prompt_length/max:604.0 - prompt_length/min:173.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014789216220378876 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2032876536250114) - timing_s/agent_loop/generate_sequences/max:np.float64(5.484291851520538) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.278224767615029) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.484291851520538) - timing_s/agent_loop/slowest/tool
(TaskRunner pid=2001144) Training Progress: 4%|▍ | 4/100 [02:47<58:35, 36.62s/it]
(TaskRunner pid=2001144) step:5 - global_seqlen/min:16082 - global_seqlen/max:22947 - global_seqlen/minmax_diff:6865 - global_seqlen/balanced_min:19226 - global_seqlen/balanced_max:19237 - global_seqlen/mean:19235.0 - actor/entropy:0.2292356938123703 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5328755378723145 - training/rollout_probs_diff_mean:0.006111367605626583 - training/rollout_probs_diff_std:0.01571831665933132 - training/rollout_actor_probs_pearson_corr:0.9975342154502869 - rollout_corr/rollout_is_mean:1.0003525018692017 - rollout_corr/rollout_is_ratio_fraction_high:4.771902968059294e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011929757602047175 - rollout_corr/rollout_is_max:5.221855163574219 - rollout_corr/rollout_is_min:0.25228720903396606 - rollout_corr/rollout_is_std:0.04753286764025688 - rollout_corr/rollout_is_eff_sample_size:0.9977469063226616 - rollout_corr/rollout_is_seq_mean:1.000296711921692 - rollout_corr/rollout_is_seq_std:0.0032089175656437874 - rollout_corr/rollout_is_seq_max:1.0130149126052856 - rollout_corr/rollout_is_seq_min:0.98602294921875 - rollout_corr/rollout_is_seq_max_deviation:0.01397705078125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2860177187249064) - rollout_corr/training_log_ppl:np.float64(0.24845740979071707) - rollout_corr/kl:np.float64(0.0009744246037328708) - rollout_corr/k3_kl:np.float64(0.0011901776415470522) - rollout_corr/rollout_ppl:np.float64(1.2847146200947464) - rollout_corr/rollout_log_ppl:np.float64(0.24748298374470323) - rollout_corr/log_ppl_diff:np.float64(0.0009744260460138321) - rollout_corr/log_ppl_abs_diff:np.float64(0.002458791306708008) - rollout_corr/log_ppl_diff_max:np.float64(0.019990086555480957) - rollout_corr/log_ppl_diff_min:np.float64(-0.011164993047714233) - rollout_corr/ppl_ratio:np.float64(1.000980426557362) - rollout_corr/chi2_token:np.float64(0.002840849570930004) - rollout_corr/chi2_seq:np.float64(2.9528152886778116) - actor/pg_loss:np.float64(-0.00020826375111937523) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008727045728846861) - actor/ppo_kl:np.float64(4.3875641402735255e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6006891652941704) - perf/mfu/actor:np.float64(0.055138926398487595) - perf/max_memory_allocated_gb:np.float64(123.23046827316284) - perf/max_memory_reserved_gb:np.float64(128.9921875) - perf/cpu_memory_used_gb:np.float64(99.03239822387695) - actor/lr:np.float64(4e-07) - training/global_step:5 - training/epoch:0 - critic/score/mean:0.4609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0007963113021105528 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0007963113021105528 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:327.4375 - response_length/max:983.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:327.4375 - response_length_non_aborted/max:983.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.65625 - prompt_length/max:520.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001307167112827301 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3612507972866297) - timing_s/agent_loop/generate_sequences/max:np.float64(7.172659207135439) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.559622299297189) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.172659207135439) - timing_s/agent_loop/slowes
(TaskRunner pid=2001144) Training Progress: 5%|▌ | 5/100 [03:13<51:51, 32.75s/it]
(TaskRunner pid=2001144) step:6 - global_seqlen/min:17384 - global_seqlen/max:21706 - global_seqlen/minmax_diff:4322 - global_seqlen/balanced_min:19590 - global_seqlen/balanced_max:19800 - global_seqlen/mean:19621.75 - actor/entropy:0.21373490989208221 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4621105492115021 - training/rollout_probs_diff_mean:0.005837094504386187 - training/rollout_probs_diff_std:0.015570911578834057 - training/rollout_actor_probs_pearson_corr:0.9974234104156494 - rollout_corr/rollout_is_mean:0.9998706579208374 - rollout_corr/rollout_is_ratio_fraction_high:2.3090955437510274e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001270002539968118 - rollout_corr/rollout_is_max:3.3360953330993652 - rollout_corr/rollout_is_min:0.28757569193840027 - rollout_corr/rollout_is_std:0.04587505757808685 - rollout_corr/rollout_is_eff_sample_size:0.9978994240724717 - rollout_corr/rollout_is_seq_mean:0.99991375207901 - rollout_corr/rollout_is_seq_std:0.0029123693238943815 - rollout_corr/rollout_is_seq_max:1.0116981267929077 - rollout_corr/rollout_is_seq_min:0.9917416572570801 - rollout_corr/rollout_is_seq_max_deviation:0.011698126792907715 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2590826102532446) - rollout_corr/training_log_ppl:np.float64(0.22722638328559697) - rollout_corr/kl:np.float64(0.0013997482810399475) - rollout_corr/k3_kl:np.float64(0.001158072938096666) - rollout_corr/rollout_ppl:np.float64(1.25725998589769) - rollout_corr/rollout_log_ppl:np.float64(0.22582663492357824) - rollout_corr/log_ppl_diff:np.float64(0.0013997483620187268) - rollout_corr/log_ppl_abs_diff:np.float64(0.002493032763595693) - rollout_corr/log_ppl_diff_max:np.float64(0.011750966310501099) - rollout_corr/log_ppl_diff_min:np.float64(-0.009251482784748077) - rollout_corr/ppl_ratio:np.float64(1.0014047641307116) - rollout_corr/chi2_token:np.float64(0.0018572814296931028) - rollout_corr/chi2_seq:np.float64(0.7247052339371294) - actor/pg_loss:np.float64(0.00011766655370593071) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001063216413513146) - actor/ppo_kl:np.float64(0.00016442567110885875) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6680105775594711) - perf/mfu/actor:np.float64(0.05739404785112723) - perf/max_memory_allocated_gb:np.float64(123.23046827316284) - perf/max_memory_reserved_gb:np.float64(128.9921875) - perf/cpu_memory_used_gb:np.float64(99.0004653930664) - actor/lr:np.float64(5e-07) - training/global_step:6 - training/epoch:0 - critic/score/mean:0.4453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004941464401781559 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004941464401781559 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:338.3359375 - response_length/max:1335.0 - response_length/min:133.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:338.3359375 - response_length_non_aborted/max:1335.0 - response_length_non_aborted/min:133.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.84375 - prompt_length/max:450.0 - prompt_length/min:173.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001075156033039093 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6150045990943909) - timing_s/agent_loop/generate_sequences/max:np.float64(8.919308114796877) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6153250775169) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.919308114796877) - timing_s/agent_loo
(TaskRunner pid=2001144) Training Progress: 6%|▌ | 6/100 [03:40<48:26, 30.92s/it]
(TaskRunner pid=2001144) step:7 - global_seqlen/min:15844 - global_seqlen/max:21641 - global_seqlen/minmax_diff:5797 - global_seqlen/balanced_min:18734 - global_seqlen/balanced_max:18736 - global_seqlen/mean:18735.25 - actor/entropy:0.24987167119979858 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24823123216629028 - training/rollout_probs_diff_mean:0.0062087057158350945 - training/rollout_probs_diff_std:0.015159662812948227 - training/rollout_actor_probs_pearson_corr:0.9978258609771729 - rollout_corr/rollout_is_mean:1.0000073909759521 - rollout_corr/rollout_is_ratio_fraction_high:1.3175577805668581e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.222904191119596e-05 - rollout_corr/rollout_is_max:2.0243802070617676 - rollout_corr/rollout_is_min:0.2674410939216614 - rollout_corr/rollout_is_std:0.04672598838806152 - rollout_corr/rollout_is_eff_sample_size:0.9978214383550319 - rollout_corr/rollout_is_seq_mean:1.0000078678131104 - rollout_corr/rollout_is_seq_std:0.0028761806897819042 - rollout_corr/rollout_is_seq_max:1.0079717636108398 - rollout_corr/rollout_is_seq_min:0.9920334219932556 - rollout_corr/rollout_is_seq_max_deviation:0.007971763610839844 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3118886887095869) - rollout_corr/training_log_ppl:np.float64(0.26835201296489686) - rollout_corr/kl:np.float64(0.0014748186453408252) - rollout_corr/k3_kl:np.float64(0.0011655904238523362) - rollout_corr/rollout_ppl:np.float64(1.3099367562681437) - rollout_corr/rollout_log_ppl:np.float64(0.266877191781532) - rollout_corr/log_ppl_diff:np.float64(0.0014748211833648384) - rollout_corr/log_ppl_abs_diff:np.float64(0.002653281146194786) - rollout_corr/log_ppl_diff_max:np.float64(0.010297298431396484) - rollout_corr/log_ppl_diff_min:np.float64(-0.006454944610595703) - rollout_corr/ppl_ratio:np.float64(1.001480347942561) - rollout_corr/chi2_token:np.float64(0.0017184854950755835) - rollout_corr/chi2_seq:np.float64(0.814037763280794) - actor/pg_loss:np.float64(-0.00013944506645202637) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009932554166880436) - actor/ppo_kl:np.float64(0.00032581822456734244) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.673604927957058) - perf/mfu/actor:np.float64(0.05468275306454441) - perf/max_memory_allocated_gb:np.float64(123.23046827316284) - perf/max_memory_reserved_gb:np.float64(128.9921875) - perf/cpu_memory_used_gb:np.float64(99.08334875106812) - actor/lr:np.float64(6e-07) - training/global_step:7 - training/epoch:0 - critic/score/mean:0.38671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.38671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019911592826247215 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0019911592826247215 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:296.4765625 - response_length/max:653.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:296.4765625 - response_length_non_aborted/max:653.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:289.0 - prompt_length/max:662.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010829232633113861 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4806529209017754) - timing_s/agent_loop/generate_sequences/max:np.float64(5.109449569135904) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.167691953116446) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.109449569135904) - timing_s/agent_
(TaskRunner pid=2001144) Training Progress: 7%|▋ | 7/100 [04:04<44:19, 28.59s/it]
(TaskRunner pid=2001144) step:8 - global_seqlen/min:16782 - global_seqlen/max:29009 - global_seqlen/minmax_diff:12227 - global_seqlen/balanced_min:20378 - global_seqlen/balanced_max:27526 - global_seqlen/mean:21273.5 - actor/entropy:0.1980297565460205 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5599467158317566 - training/rollout_probs_diff_mean:0.005174444522708654 - training/rollout_probs_diff_std:0.01463010348379612 - training/rollout_actor_probs_pearson_corr:0.9976457357406616 - rollout_corr/rollout_is_mean:1.000063180923462 - rollout_corr/rollout_is_ratio_fraction_high:4.197271846351214e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010493179433979094 - rollout_corr/rollout_is_max:5.700270175933838 - rollout_corr/rollout_is_min:0.3046276867389679 - rollout_corr/rollout_is_std:0.043574828654527664 - rollout_corr/rollout_is_eff_sample_size:0.9981049516719247 - rollout_corr/rollout_is_seq_mean:1.0000544786453247 - rollout_corr/rollout_is_seq_std:0.0028321563731878996 - rollout_corr/rollout_is_seq_max:1.0116652250289917 - rollout_corr/rollout_is_seq_min:0.9911968111991882 - rollout_corr/rollout_is_seq_max_deviation:0.0116652250289917 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2691756379790604) - rollout_corr/training_log_ppl:np.float64(0.23452277634714846) - rollout_corr/kl:np.float64(0.0009898367117955331) - rollout_corr/k3_kl:np.float64(0.0011807819291504984) - rollout_corr/rollout_ppl:np.float64(1.2679196307435632) - rollout_corr/rollout_log_ppl:np.float64(0.2335329395136796) - rollout_corr/log_ppl_diff:np.float64(0.0009898368334688712) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023592572724737693) - rollout_corr/log_ppl_diff_max:np.float64(0.010740295052528381) - rollout_corr/log_ppl_diff_min:np.float64(-0.008645445108413696) - rollout_corr/ppl_ratio:np.float64(1.0009945032652467) - rollout_corr/chi2_token:np.float64(0.003041410120204091) - rollout_corr/chi2_seq:np.float64(1.8656591917388141) - actor/pg_loss:np.float64(-0.00011384394019842148) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000983607562375255) - actor/ppo_kl:np.float64(-0.00010213372407541499) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.615655317902565) - perf/mfu/actor:np.float64(0.05799213283080365) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(98.98189926147461) - actor/lr:np.float64(7e-07) - training/global_step:8 - training/epoch:0 - critic/score/mean:0.52734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.52734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.04217733442783356 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.04217733442783356 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:372.265625 - response_length/max:8192.0 - response_length/min:126.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:372.265625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:292.53125 - prompt_length/max:720.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012424588203430176 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8992805797606707) - timing_s/agent_loop/generate_sequences/max:np.float64(49.95310030132532) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5988047708087834) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.95310030132532) - tim
(TaskRunner pid=2001144) Training Progress: 8%|▊ | 8/100 [05:14<1:03:42, 41.55s/it]
(TaskRunner pid=2001144) step:9 - global_seqlen/min:16403 - global_seqlen/max:22002 - global_seqlen/minmax_diff:5599 - global_seqlen/balanced_min:18531 - global_seqlen/balanced_max:18532 - global_seqlen/mean:18531.625 - actor/entropy:0.2494705617427826 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.470685750246048 - training/rollout_probs_diff_mean:0.0063387444242835045 - training/rollout_probs_diff_std:0.015942737460136414 - training/rollout_actor_probs_pearson_corr:0.997621476650238 - rollout_corr/rollout_is_mean:1.0000908374786377 - rollout_corr/rollout_is_ratio_fraction_high:5.382928065955639e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.074392098933458e-05 - rollout_corr/rollout_is_max:3.7972795963287354 - rollout_corr/rollout_is_min:0.2831878662109375 - rollout_corr/rollout_is_std:0.04843469336628914 - rollout_corr/rollout_is_eff_sample_size:0.9976599268055171 - rollout_corr/rollout_is_seq_mean:1.000196933746338 - rollout_corr/rollout_is_seq_std:0.0032875696197152138 - rollout_corr/rollout_is_seq_max:1.0117636919021606 - rollout_corr/rollout_is_seq_min:0.9898091554641724 - rollout_corr/rollout_is_seq_max_deviation:0.011763691902160645 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.30543770827353) - rollout_corr/training_log_ppl:np.float64(0.2632649658189621) - rollout_corr/kl:np.float64(0.0009418832014631562) - rollout_corr/k3_kl:np.float64(0.001251085950457309) - rollout_corr/rollout_ppl:np.float64(1.3042354141362011) - rollout_corr/rollout_log_ppl:np.float64(0.26232308059115894) - rollout_corr/log_ppl_diff:np.float64(0.0009418852278031409) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025248671299777925) - rollout_corr/log_ppl_diff_max:np.float64(0.011189579963684082) - rollout_corr/log_ppl_diff_min:np.float64(-0.009953498840332031) - rollout_corr/ppl_ratio:np.float64(1.0009476765990257) - rollout_corr/chi2_token:np.float64(0.0031711403280496597) - rollout_corr/chi2_seq:np.float64(1.6533140249084681) - actor/pg_loss:np.float64(-2.5617191568017006e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00141446490169983) - actor/ppo_kl:np.float64(-0.0001171964302635331) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.7313621640205383) - perf/mfu/actor:np.float64(0.05392485374524558) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(99.09900951385498) - actor/lr:np.float64(8e-07) - training/global_step:9 - training/epoch:0 - critic/score/mean:0.3671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0024694183375686407 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0024694183375686407 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:290.26953125 - response_length/max:833.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:290.26953125 - response_length_non_aborted/max:833.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:288.84375 - prompt_length/max:735.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010447949171066284 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9076374713331461) - timing_s/agent_loop/generate_sequences/max:np.float64(6.183150801807642) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9941010520487907) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.183150801807642) - timing_s/agent
(TaskRunner pid=2001144) Training Progress: 9%|▉ | 9/100 [05:38<54:56, 36.23s/it]
(TaskRunner pid=2001144) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 10}
(TaskRunner pid=2001144) validation generation end
(TaskRunner pid=2001144) [prompt] system
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2001144) <reasoning>
(TaskRunner pid=2001144) ...
(TaskRunner pid=2001144) </reasoning>
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) ...
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) A
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) user
(TaskRunner pid=2001144) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2001144) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2001144) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2001144) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2001144) Please reason step by step.
(TaskRunner pid=2001144) assistant
(TaskRunner pid=2001144) <think>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) </think>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) [response] <reasoning>
(TaskRunner pid=2001144) The question asks which molecule is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". This product contains a complex structure with a benzene ring substituted with a chlorine atom and a trifluoromethyl group (C(F)(F)F). The key part of the product is the "Cc2ccc(Cl)cc2C(F)(F)F" fragment, which suggests that the reactant must contain a similar structure.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Looking at the options:
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) A: Contains a ketone group (O=C) and a nitrogen-containing ring, but does not match the specific substituents in the product.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) B: Contains a chlorine atom and a nitrogen-containing ring, but the structure does not match the specific substituents in the product.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) C: Contains a ketone group (O=C), a nitrogen-containing ring, and a substituent with a chlorine atom and a trifluoromethyl group (C(F)(F)F), which matches the substituents in the product.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) D: Contains a ketone group (O=C), a nitrogen-containing ring, and a substituent with a trifluoromethyl group (C(F)(F)F), but does not contain a chlorine atom, which is present in the product.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Therefore, option C is the only one that contains the correct substituents (chlorine and trifluoromethyl group) and matches the structure of the product.
(TaskRunner pid=2001144) </reasoning>
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) C
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144) [ground_truth] A
(TaskRunner pid=2001144) [score] 0.0
(TaskRunner pid=2001144) [acc] 0.0
(TaskRunner pid=2001144) [pred] C
(TaskRunner pid=2001144) [incorrect_format] 1
(TaskRunner pid=2001144) [feedback]
(TaskRunner pid=2001144) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10
(TaskRunner pid=2001144) step:10 - global_seqlen/min:16499 - global_seqlen/max:20634 - global_seqlen/minmax_diff:4135 - global_seqlen/balanced_min:18668 - global_seqlen/balanced_max:18671 - global_seqlen/mean:18669.625 - actor/entropy:0.23466506600379944 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102376401424408 - training/rollout_probs_diff_mean:0.005939433351159096 - training/rollout_probs_diff_std:0.015314425341784954 - training/rollout_actor_probs_pearson_corr:0.9977052211761475 - rollout_corr/rollout_is_mean:0.9997667074203491 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00011194447870366275 - rollout_corr/rollout_is_max:1.8923181295394897 - rollout_corr/rollout_is_min:0.1828954964876175 - rollout_corr/rollout_is_std:0.04555103927850723 - rollout_corr/rollout_is_eff_sample_size:0.997928627340882 - rollout_corr/rollout_is_seq_mean:0.9997364282608032 - rollout_corr/rollout_is_seq_std:0.0029734699055552483 - rollout_corr/rollout_is_seq_max:1.0109349489212036 - rollout_corr/rollout_is_seq_min:0.9885185360908508 - rollout_corr/rollout_is_seq_max_deviation:0.01148146390914917 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2886819131672382) - rollout_corr/training_log_ppl:np.float64(0.2502548612537794) - rollout_corr/kl:np.float64(0.0012091508927172256) - rollout_corr/k3_kl:np.float64(0.0011430539511252391) - rollout_corr/rollout_ppl:np.float64(1.2871073153801262) - rollout_corr/rollout_log_ppl:np.float64(0.24904570922080893) - rollout_corr/log_ppl_diff:np.float64(0.0012091520329704508) - rollout_corr/log_ppl_abs_diff:np.float64(0.002516082939109765) - rollout_corr/log_ppl_diff_max:np.float64(0.013573050498962402) - rollout_corr/log_ppl_diff_min:np.float64(-0.008949816226959229) - rollout_corr/ppl_ratio:np.float64(1.0012146937660873) - rollout_corr/chi2_token:np.float64(0.0021457152906805277) - rollout_corr/chi2_seq:np.float64(0.8803747321944684) - actor/pg_loss:np.float64(0.0005458357045426965) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009766517282514542) - actor/ppo_kl:np.float64(-0.00015984548297964807) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6216555684804916) - perf/mfu/actor:np.float64(0.05389931420407375) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(98.98427391052246) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.4398809523809524) - val-aux/sciknoweval/reward/std@16:np.float64(0.2531525674276788) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5452428571428571) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.22649699915845536) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.3350523809523809) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.20395124061665057) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.43977619047619054) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.253434096775805) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6373428571428571) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.17734368583388807) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.2564666666666667) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.13947206052940614) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.4509047619047619) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.20914979153477942) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7085333333333333) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.11850900812548211) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.2045857142857143) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08650800772441575) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.4578190476190476) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.15974105012253428) - val-aux/sciknoweval/reward/best@16/mean:np.float64
(TaskRunner pid=2001144) Training Progress: 10%|█ | 10/100 [08:09<1:47:41, 71.80s/it]
(TaskRunner pid=2001144) step:11 - global_seqlen/min:16972 - global_seqlen/max:23304 - global_seqlen/minmax_diff:6332 - global_seqlen/balanced_min:19878 - global_seqlen/balanced_max:19881 - global_seqlen/mean:19879.5 - actor/entropy:0.2568400800228119 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3175737261772156 - training/rollout_probs_diff_mean:0.006205270532518625 - training/rollout_probs_diff_std:0.015006247907876968 - training/rollout_actor_probs_pearson_corr:0.9978901147842407 - rollout_corr/rollout_is_mean:1.0000262260437012 - rollout_corr/rollout_is_ratio_fraction_high:3.637774716480635e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.8503661673748866e-05 - rollout_corr/rollout_is_max:2.5544419288635254 - rollout_corr/rollout_is_min:0.3556796610355377 - rollout_corr/rollout_is_std:0.04575927555561066 - rollout_corr/rollout_is_eff_sample_size:0.9979107015548458 - rollout_corr/rollout_is_seq_mean:1.0000146627426147 - rollout_corr/rollout_is_seq_std:0.0029180385172367096 - rollout_corr/rollout_is_seq_max:1.0130001306533813 - rollout_corr/rollout_is_seq_min:0.9923868179321289 - rollout_corr/rollout_is_seq_max_deviation:0.013000130653381348 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.309125877916813) - rollout_corr/training_log_ppl:np.float64(0.26654446820612065) - rollout_corr/kl:np.float64(0.001024794482830771) - rollout_corr/k3_kl:np.float64(0.0011288186734645933) - rollout_corr/rollout_ppl:np.float64(1.307792722247541) - rollout_corr/rollout_log_ppl:np.float64(0.2655196722771507) - rollout_corr/log_ppl_diff:np.float64(0.0010247959289699793) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024583893246017396) - rollout_corr/log_ppl_diff_max:np.float64(0.010337024927139282) - rollout_corr/log_ppl_diff_min:np.float64(-0.008593171834945679) - rollout_corr/ppl_ratio:np.float64(1.0010299547575414) - rollout_corr/chi2_token:np.float64(0.002454928820952773) - rollout_corr/chi2_seq:np.float64(1.2113799394574016) - actor/pg_loss:np.float64(0.0005805137334391475) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013821643965457042) - actor/ppo_kl:np.float64(-6.95000276493829e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.734000563621521) - perf/mfu/actor:np.float64(0.05680744081325592) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(103.87120914459229) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.4609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0014657200081273913 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0014657200081273913 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:322.140625 - response_length/max:747.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:322.140625 - response_length_non_aborted/max:747.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.09375 - prompt_length/max:620.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.674656808376312e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0329370480030775) - timing_s/agent_loop/generate_sequences/max:np.float64(5.834429504349828) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2267463311582105) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.834429504349828) - timing_s/agent_l
(TaskRunner pid=2001144) Training Progress: 11%|█ | 11/100 [08:34<1:24:59, 57.30s/it]
(TaskRunner pid=2001144) step:12 - global_seqlen/min:16616 - global_seqlen/max:21054 - global_seqlen/minmax_diff:4438 - global_seqlen/balanced_min:18539 - global_seqlen/balanced_max:18546 - global_seqlen/mean:18542.75 - actor/entropy:0.22723700106143951 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2932344377040863 - training/rollout_probs_diff_mean:0.0056840586476027966 - training/rollout_probs_diff_std:0.01489218883216381 - training/rollout_actor_probs_pearson_corr:0.9977792501449585 - rollout_corr/rollout_is_mean:1.0001059770584106 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.725786028662696e-05 - rollout_corr/rollout_is_max:1.9842201471328735 - rollout_corr/rollout_is_min:0.3704789876937866 - rollout_corr/rollout_is_std:0.045186467468738556 - rollout_corr/rollout_is_eff_sample_size:0.9979626999810725 - rollout_corr/rollout_is_seq_mean:1.0000935792922974 - rollout_corr/rollout_is_seq_std:0.0029613038059324026 - rollout_corr/rollout_is_seq_max:1.009190320968628 - rollout_corr/rollout_is_seq_min:0.9909468293190002 - rollout_corr/rollout_is_seq_max_deviation:0.00919032096862793 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2808634196408093) - rollout_corr/training_log_ppl:np.float64(0.24432541536225472) - rollout_corr/kl:np.float64(0.0009018880355071701) - rollout_corr/k3_kl:np.float64(0.0011332543278967933) - rollout_corr/rollout_ppl:np.float64(1.2797023290768266) - rollout_corr/rollout_log_ppl:np.float64(0.2434235254477244) - rollout_corr/log_ppl_diff:np.float64(0.0009018899145303294) - rollout_corr/log_ppl_abs_diff:np.float64(0.002466833670041524) - rollout_corr/log_ppl_diff_max:np.float64(0.009635865688323975) - rollout_corr/log_ppl_diff_min:np.float64(-0.009274661540985107) - rollout_corr/ppl_ratio:np.float64(1.0009068062063307) - rollout_corr/chi2_token:np.float64(0.0027368178125470877) - rollout_corr/chi2_seq:np.float64(1.1611458365805447) - actor/pg_loss:np.float64(2.365687396377325e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007770666093165346) - actor/ppo_kl:np.float64(-6.309558004602422e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6073131710290909) - perf/mfu/actor:np.float64(0.05349557021952155) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(103.51963424682617) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.48046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0016843883786350489 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0016843883786350489 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:313.3671875 - response_length/max:944.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:313.3671875 - response_length_non_aborted/max:944.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.09375 - prompt_length/max:665.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001513790339231491 - timing_s/agent_loop/generate_sequences/min:np.float64(1.105068003758788) - timing_s/agent_loop/generate_sequences/max:np.float64(6.365561559796333) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1331619364136714) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.365561559796333) - timing_s/agent_loop/slowes
(TaskRunner pid=2001144) Training Progress: 12%|█▏ | 12/100 [08:59<1:09:39, 47.49s/it]
(TaskRunner pid=2001144) step:13 - global_seqlen/min:16504 - global_seqlen/max:19938 - global_seqlen/minmax_diff:3434 - global_seqlen/balanced_min:18354 - global_seqlen/balanced_max:18365 - global_seqlen/mean:18356.25 - actor/entropy:0.26043570041656494 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5599740743637085 - training/rollout_probs_diff_mean:0.006170737091451883 - training/rollout_probs_diff_std:0.01526377908885479 - training/rollout_actor_probs_pearson_corr:0.997814953327179 - rollout_corr/rollout_is_mean:1.000131368637085 - rollout_corr/rollout_is_ratio_fraction_high:2.6342824639868923e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.902847573859617e-05 - rollout_corr/rollout_is_max:2.745440721511841 - rollout_corr/rollout_is_min:0.29307273030281067 - rollout_corr/rollout_is_std:0.04572148248553276 - rollout_corr/rollout_is_eff_sample_size:0.99791438163057 - rollout_corr/rollout_is_seq_mean:1.0001637935638428 - rollout_corr/rollout_is_seq_std:0.0029542227275669575 - rollout_corr/rollout_is_seq_max:1.0101301670074463 - rollout_corr/rollout_is_seq_min:0.9904539585113525 - rollout_corr/rollout_is_seq_max_deviation:0.010130167007446289 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3110852953977883) - rollout_corr/training_log_ppl:np.float64(0.2679355984146241) - rollout_corr/kl:np.float64(0.001122537189274908) - rollout_corr/k3_kl:np.float64(0.0011619261935038594) - rollout_corr/rollout_ppl:np.float64(1.309649194125086) - rollout_corr/rollout_log_ppl:np.float64(0.26681306047248654) - rollout_corr/log_ppl_diff:np.float64(0.0011225379421375692) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026728353113867342) - rollout_corr/log_ppl_diff_max:np.float64(0.011994540691375732) - rollout_corr/log_ppl_diff_min:np.float64(-0.009199783205986023) - rollout_corr/ppl_ratio:np.float64(1.0011286756489426) - rollout_corr/chi2_token:np.float64(0.002395269460976124) - rollout_corr/chi2_seq:np.float64(1.0188483067322522) - actor/pg_loss:np.float64(3.5404693335294724e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012823700794797332) - actor/ppo_kl:np.float64(0.00018168347935443308) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6756055504083633) - perf/mfu/actor:np.float64(0.05365240280751715) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.96561050415039) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004651155322790146 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004651155322790146 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:296.5703125 - response_length/max:874.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:296.5703125 - response_length_non_aborted/max:874.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.0625 - prompt_length/max:612.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.747750401496887e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3406916372478008) - timing_s/agent_loop/generate_sequences/max:np.float64(6.296166500076652) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2052804644044954) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.296166500076652) - timing_s/agent_loo
(TaskRunner pid=2001144) Training Progress: 13%|█▎ | 13/100 [09:24<58:54, 40.62s/it]
(TaskRunner pid=2001144) step:14 - global_seqlen/min:16538 - global_seqlen/max:20578 - global_seqlen/minmax_diff:4040 - global_seqlen/balanced_min:17931 - global_seqlen/balanced_max:17935 - global_seqlen/mean:17933.875 - actor/entropy:0.25489386916160583 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.42451077699661255 - training/rollout_probs_diff_mean:0.006314896512776613 - training/rollout_probs_diff_std:0.015762560069561005 - training/rollout_actor_probs_pearson_corr:0.9976977109909058 - rollout_corr/rollout_is_mean:0.9999311566352844 - rollout_corr/rollout_is_ratio_fraction_high:5.332835644367151e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010665671288734302 - rollout_corr/rollout_is_max:4.368031024932861 - rollout_corr/rollout_is_min:0.18324820697307587 - rollout_corr/rollout_is_std:0.047335587441921234 - rollout_corr/rollout_is_eff_sample_size:0.9977636394561464 - rollout_corr/rollout_is_seq_mean:1.0000313520431519 - rollout_corr/rollout_is_seq_std:0.0029517030343413353 - rollout_corr/rollout_is_seq_max:1.0110210180282593 - rollout_corr/rollout_is_seq_min:0.9917967915534973 - rollout_corr/rollout_is_seq_max_deviation:0.011021018028259277 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3141490640118718) - rollout_corr/training_log_ppl:np.float64(0.2699595228477847) - rollout_corr/kl:np.float64(0.001397545010334511) - rollout_corr/k3_kl:np.float64(0.0013356649056959213) - rollout_corr/rollout_ppl:np.float64(1.3122873408719897) - rollout_corr/rollout_log_ppl:np.float64(0.26856197745655663) - rollout_corr/log_ppl_diff:np.float64(0.00139754539122805) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026808647089637816) - rollout_corr/log_ppl_diff_max:np.float64(0.011007845401763916) - rollout_corr/log_ppl_diff_min:np.float64(-0.008663326501846313) - rollout_corr/ppl_ratio:np.float64(1.0014033836778253) - rollout_corr/chi2_token:np.float64(0.002597767859697342) - rollout_corr/chi2_seq:np.float64(0.9163431329652667) - actor/pg_loss:np.float64(0.00030385772697627544) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014312766675175226) - actor/ppo_kl:np.float64(0.00020630698352164245) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5935972481966019) - perf/mfu/actor:np.float64(0.05202053672815645) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.96749782562256) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.3828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005056194961071014 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005056194961071014 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:292.99609375 - response_length/max:690.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:292.99609375 - response_length_non_aborted/max:690.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.4375 - prompt_length/max:666.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.713135659694672e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6927519552409649) - timing_s/agent_loop/generate_sequences/max:np.float64(5.228574378415942) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8809800103117595) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.228574378415942) - timing_s/ag
(TaskRunner pid=2001144) Training Progress: 14%|█▍ | 14/100 [09:47<50:49, 35.46s/it]
(TaskRunner pid=2001144) step:15 - global_seqlen/min:16414 - global_seqlen/max:24183 - global_seqlen/minmax_diff:7769 - global_seqlen/balanced_min:19637 - global_seqlen/balanced_max:19646 - global_seqlen/mean:19643.5 - actor/entropy:0.2467813342809677 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4452105164527893 - training/rollout_probs_diff_mean:0.0059585063718259335 - training/rollout_probs_diff_std:0.015233662910759449 - training/rollout_actor_probs_pearson_corr:0.9978125691413879 - rollout_corr/rollout_is_mean:0.9999366998672485 - rollout_corr/rollout_is_ratio_fraction_high:2.3894863261375576e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.168458978412673e-05 - rollout_corr/rollout_is_max:2.333303928375244 - rollout_corr/rollout_is_min:0.05626031383872032 - rollout_corr/rollout_is_std:0.04593608155846596 - rollout_corr/rollout_is_eff_sample_size:0.9978940821961024 - rollout_corr/rollout_is_seq_mean:0.9999532699584961 - rollout_corr/rollout_is_seq_std:0.0028823891188949347 - rollout_corr/rollout_is_seq_max:1.0088497400283813 - rollout_corr/rollout_is_seq_min:0.992048442363739 - rollout_corr/rollout_is_seq_max_deviation:0.008849740028381348 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3019497729837894) - rollout_corr/training_log_ppl:np.float64(0.2605938920751214) - rollout_corr/kl:np.float64(0.0015000761722845368) - rollout_corr/k3_kl:np.float64(0.0014059990195391947) - rollout_corr/rollout_ppl:np.float64(1.2999335206113756) - rollout_corr/rollout_log_ppl:np.float64(0.25909381291421596) - rollout_corr/log_ppl_diff:np.float64(0.0015000791609054431) - rollout_corr/log_ppl_abs_diff:np.float64(0.002593038443592377) - rollout_corr/log_ppl_diff_max:np.float64(0.013034999370574951) - rollout_corr/log_ppl_diff_min:np.float64(-0.005962371826171875) - rollout_corr/ppl_ratio:np.float64(1.0015062093734741) - rollout_corr/chi2_token:np.float64(0.002502645133063197) - rollout_corr/chi2_seq:np.float64(0.5845161369070411) - actor/pg_loss:np.float64(0.00025862420443445444) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013049286999375909) - actor/ppo_kl:np.float64(0.00032604149287962514) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6387124732136726) - perf/mfu/actor:np.float64(0.057379223993279604) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.89004135131836) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009287634864449501 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009287634864449501 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:326.953125 - response_length/max:878.0 - response_length/min:129.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:326.953125 - response_length_non_aborted/max:878.0 - response_length_non_aborted/min:129.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.90625 - prompt_length/max:494.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011238642036914825 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2711260039359331) - timing_s/agent_loop/generate_sequences/max:np.float64(6.429199520498514) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3255655286775436) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.429199520498514) - timing_s/agen
(TaskRunner pid=2001144) Training Progress: 15%|█▌ | 15/100 [10:13<45:55, 32.42s/it]
(TaskRunner pid=2001144) step:16 - global_seqlen/min:15128 - global_seqlen/max:21306 - global_seqlen/minmax_diff:6178 - global_seqlen/balanced_min:18212 - global_seqlen/balanced_max:18217 - global_seqlen/mean:18214.5 - actor/entropy:0.2534453868865967 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8919227719306946 - training/rollout_probs_diff_mean:0.00648665614426136 - training/rollout_probs_diff_std:0.016380665823817253 - training/rollout_actor_probs_pearson_corr:0.9974845051765442 - rollout_corr/rollout_is_mean:0.9999707937240601 - rollout_corr/rollout_is_ratio_fraction_high:4.062178413732909e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010832475527422503 - rollout_corr/rollout_is_max:3.4635722637176514 - rollout_corr/rollout_is_min:0.0501820407807827 - rollout_corr/rollout_is_std:0.049311570823192596 - rollout_corr/rollout_is_eff_sample_size:0.9975739115037218 - rollout_corr/rollout_is_seq_mean:0.9999513626098633 - rollout_corr/rollout_is_seq_std:0.0030612836126238108 - rollout_corr/rollout_is_seq_max:1.015810251235962 - rollout_corr/rollout_is_seq_min:0.9910605549812317 - rollout_corr/rollout_is_seq_max_deviation:0.015810251235961914 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3079908890649676) - rollout_corr/training_log_ppl:np.float64(0.2653956447902601) - rollout_corr/kl:np.float64(0.0015181687596950155) - rollout_corr/k3_kl:np.float64(0.0014056676951668123) - rollout_corr/rollout_ppl:np.float64(1.3059758823364973) - rollout_corr/rollout_log_ppl:np.float64(0.2638774759543594) - rollout_corr/log_ppl_diff:np.float64(0.001518168835900724) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029001745861023664) - rollout_corr/log_ppl_diff_max:np.float64(0.01136597990989685) - rollout_corr/log_ppl_diff_min:np.float64(-0.009734898805618286) - rollout_corr/ppl_ratio:np.float64(1.001524917082861) - rollout_corr/chi2_token:np.float64(0.0025894674472510815) - rollout_corr/chi2_seq:np.float64(2.101719085825607) - actor/pg_loss:np.float64(-1.1261901818215847e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016824579834064934) - actor/ppo_kl:np.float64(0.00018151451919656836) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6559636294841766) - perf/mfu/actor:np.float64(0.05199215735894495) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.92138290405273) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0013794481055811048 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0013794481055811048 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:288.484375 - response_length/max:670.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:288.484375 - response_length_non_aborted/max:670.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.71875 - prompt_length/max:859.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013069622218608856 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7501519974321127) - timing_s/agent_loop/generate_sequences/max:np.float64(5.151654185727239) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.730000512499828) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.151654185727239) - timing_s/agent
(TaskRunner pid=2001144) Training Progress: 16%|█▌ | 16/100 [10:37<41:48, 29.87s/it]
(TaskRunner pid=2001144) step:17 - global_seqlen/min:17229 - global_seqlen/max:24109 - global_seqlen/minmax_diff:6880 - global_seqlen/balanced_min:21032 - global_seqlen/balanced_max:21049 - global_seqlen/mean:21043.875 - actor/entropy:0.24237079918384552 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48335814476013184 - training/rollout_probs_diff_mean:0.005934419110417366 - training/rollout_probs_diff_std:0.015368545427918434 - training/rollout_actor_probs_pearson_corr:0.9977557063102722 - rollout_corr/rollout_is_mean:0.9996291399002075 - rollout_corr/rollout_is_ratio_fraction_high:3.6124121834291145e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010837236914085224 - rollout_corr/rollout_is_max:2.384920835494995 - rollout_corr/rollout_is_min:0.17634271085262299 - rollout_corr/rollout_is_std:0.04588804766535759 - rollout_corr/rollout_is_eff_sample_size:0.9978970498981358 - rollout_corr/rollout_is_seq_mean:0.9996278285980225 - rollout_corr/rollout_is_seq_std:0.0029213442467153072 - rollout_corr/rollout_is_seq_max:1.010084867477417 - rollout_corr/rollout_is_seq_min:0.9916045665740967 - rollout_corr/rollout_is_seq_max_deviation:0.010084867477416992 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3161271321587265) - rollout_corr/training_log_ppl:np.float64(0.2709654231730383) - rollout_corr/kl:np.float64(0.0015956489069308688) - rollout_corr/k3_kl:np.float64(0.0012857737199283292) - rollout_corr/rollout_ppl:np.float64(1.3140167291276157) - rollout_corr/rollout_log_ppl:np.float64(0.2693697740032803) - rollout_corr/log_ppl_diff:np.float64(0.0015956491697579622) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026653365639504045) - rollout_corr/log_ppl_diff_max:np.float64(0.012520521879196167) - rollout_corr/log_ppl_diff_min:np.float64(-0.008787065744400024) - rollout_corr/ppl_ratio:np.float64(1.0016017749439925) - rollout_corr/chi2_token:np.float64(0.0018776548095047474) - rollout_corr/chi2_seq:np.float64(0.16753704147413373) - actor/pg_loss:np.float64(0.0004364902852103114) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001323818467426463) - actor/ppo_kl:np.float64(6.438546964204761e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.475775346159935) - perf/mfu/actor:np.float64(0.06116319690839021) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.92072677612305) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0059153251349925995 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0059153251349925995 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:324.40234375 - response_length/max:1206.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:324.40234375 - response_length_non_aborted/max:1206.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:333.21875 - prompt_length/max:617.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.671039879322052e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7061740309000015) - timing_s/agent_loop/generate_sequences/max:np.float64(7.931276587769389) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.704336658178363) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.931276587769389) - timing
(TaskRunner pid=2001144) Training Progress: 17%|█▋ | 17/100 [11:03<39:53, 28.84s/it]
(TaskRunner pid=2001144) step:18 - global_seqlen/min:14987 - global_seqlen/max:19635 - global_seqlen/minmax_diff:4648 - global_seqlen/balanced_min:17643 - global_seqlen/balanced_max:17645 - global_seqlen/mean:17644.0 - actor/entropy:0.26970869302749634 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5830113887786865 - training/rollout_probs_diff_mean:0.006448776926845312 - training/rollout_probs_diff_std:0.015335245057940483 - training/rollout_actor_probs_pearson_corr:0.9978418350219727 - rollout_corr/rollout_is_mean:1.000036597251892 - rollout_corr/rollout_is_ratio_fraction_high:4.2139567085541785e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.832565410761163e-05 - rollout_corr/rollout_is_max:2.183253765106201 - rollout_corr/rollout_is_min:0.20251059532165527 - rollout_corr/rollout_is_std:0.048081450164318085 - rollout_corr/rollout_is_eff_sample_size:0.9976937437789896 - rollout_corr/rollout_is_seq_mean:0.9999678730964661 - rollout_corr/rollout_is_seq_std:0.0029498955700546503 - rollout_corr/rollout_is_seq_max:1.0073635578155518 - rollout_corr/rollout_is_seq_min:0.9901910424232483 - rollout_corr/rollout_is_seq_max_deviation:0.009808957576751709 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3322614277713) - rollout_corr/training_log_ppl:np.float64(0.2836042442067992) - rollout_corr/kl:np.float64(0.001448770982129588) - rollout_corr/k3_kl:np.float64(0.0012904692449637878) - rollout_corr/rollout_ppl:np.float64(1.330310198944062) - rollout_corr/rollout_log_ppl:np.float64(0.282155472174054) - rollout_corr/log_ppl_diff:np.float64(0.0014487720327451825) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025887844967655838) - rollout_corr/log_ppl_diff_max:np.float64(0.01330643892288208) - rollout_corr/log_ppl_diff_min:np.float64(-0.007128685712814331) - rollout_corr/ppl_ratio:np.float64(1.0014543796423823) - rollout_corr/chi2_token:np.float64(0.0022857701405882835) - rollout_corr/chi2_seq:np.float64(1.108610187890008) - actor/pg_loss:np.float64(-4.5623164623975754e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015221664089040132) - actor/ppo_kl:np.float64(0.00021617239783999764) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6824703365564346) - perf/mfu/actor:np.float64(0.05163383872473892) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.88374328613281) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0016434431308880448 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0016434431308880448 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:278.09375 - response_length/max:702.0 - response_length/min:124.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:278.09375 - response_length_non_aborted/max:702.0 - response_length_non_aborted/min:124.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.28125 - prompt_length/max:540.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014663487672805786 - timing_s/agent_loop/generate_sequences/min:np.float64(1.150055268779397) - timing_s/agent_loop/generate_sequences/max:np.float64(5.1164687890559435) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9099226215621457) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.1164687890559435) - timing_s/agent_l
(TaskRunner pid=2001144) Training Progress: 18%|█▊ | 18/100 [11:27<37:11, 27.21s/it]
(TaskRunner pid=2001144) step:19 - global_seqlen/min:16965 - global_seqlen/max:27062 - global_seqlen/minmax_diff:10097 - global_seqlen/balanced_min:19414 - global_seqlen/balanced_max:26375 - global_seqlen/mean:20284.375 - actor/entropy:0.23812763392925262 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29032203555107117 - training/rollout_probs_diff_mean:0.005633227992802858 - training/rollout_probs_diff_std:0.014483724720776081 - training/rollout_actor_probs_pearson_corr:0.9979185461997986 - rollout_corr/rollout_is_mean:0.9999683499336243 - rollout_corr/rollout_is_ratio_fraction_high:1.1774263839470223e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.709705535788089e-05 - rollout_corr/rollout_is_max:2.0693747997283936 - rollout_corr/rollout_is_min:0.058139823377132416 - rollout_corr/rollout_is_std:0.043480340391397476 - rollout_corr/rollout_is_eff_sample_size:0.9981129085148519 - rollout_corr/rollout_is_seq_mean:1.0001128911972046 - rollout_corr/rollout_is_seq_std:0.00288040516898036 - rollout_corr/rollout_is_seq_max:1.0079009532928467 - rollout_corr/rollout_is_seq_min:0.9920950531959534 - rollout_corr/rollout_is_seq_max_deviation:0.00790494680404663 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3143577151931822) - rollout_corr/training_log_ppl:np.float64(0.269440776151896) - rollout_corr/kl:np.float64(0.0011703791883235226) - rollout_corr/k3_kl:np.float64(0.0011921048244118992) - rollout_corr/rollout_ppl:np.float64(1.312834210228175) - rollout_corr/rollout_log_ppl:np.float64(0.268270394609317) - rollout_corr/log_ppl_diff:np.float64(0.0011703815425789799) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024647157288200106) - rollout_corr/log_ppl_diff_max:np.float64(0.012380748987197876) - rollout_corr/log_ppl_diff_min:np.float64(-0.007736772298812866) - rollout_corr/ppl_ratio:np.float64(1.0011754070874304) - rollout_corr/chi2_token:np.float64(0.0024726022966206074) - rollout_corr/chi2_seq:np.float64(0.6414078404195607) - actor/pg_loss:np.float64(8.146546315401793e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005772276381321717) - actor/ppo_kl:np.float64(0.0001913630728651583) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4756404012441635) - perf/mfu/actor:np.float64(0.055944168489773044) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.84822082519531) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010490869171917439 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010490869171917439 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:331.76171875 - response_length/max:8192.0 - response_length/min:126.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:331.76171875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:302.125 - prompt_length/max:836.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013879314064979553 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4014530777931213) - timing_s/agent_loop/generate_sequences/max:np.float64(48.83390277065337) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3988357019479736) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(48.833902770653
(TaskRunner pid=2001144) Training Progress: 19%|█▉ | 19/100 [12:34<53:14, 39.44s/it]
(TaskRunner pid=2001144) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20}
(TaskRunner pid=2001144) validation generation end
(TaskRunner pid=2001144) [prompt] system
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2001144) <reasoning>
(TaskRunner pid=2001144) ...
(TaskRunner pid=2001144) </reasoning>
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) ...
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) A
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) user
(TaskRunner pid=2001144) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2001144) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2001144) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2001144) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2001144) Please reason step by step.
(TaskRunner pid=2001144) assistant
(TaskRunner pid=2001144) <think>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) </think>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) [response] <reasoning>
(TaskRunner pid=2001144) The question asks which of the given molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". This product contains a complex structure with a substituted benzene ring and a trifluoromethyl group (C(F)(F)F). The key is to identify which of the options matches the structure of the reactant that would be used to synthesize this product.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Looking at the product, it contains a nitrile group (–n) and a substituted benzene ring with a chlorine atom and a trifluoromethyl group. The correct reactant should have a structure that can lead to the formation of this product through a chemical reaction, such as a nucleophilic substitution or a condensation reaction.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Option A contains a structure that includes a nitrile group and a substituted benzene ring with a chlorine atom and a trifluoromethyl group. It also includes a carbonyl group (C=O) and a nitrogen atom, which suggests it could be a suitable reactant in a reaction that forms the product. The structure of option A matches the key features of the product, including the presence of a nitrile group and the substituted benzene ring.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Option B contains a structure with a chlorine atom and a benzene ring, but it does not include the trifluoromethyl group or the nitrile group, which are present in the product. Therefore, it is less likely to be the correct reactant.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Option C contains a structure with a substituted benzene ring and a trifluoromethyl group, but it does not include the nitrile group, which is a key feature of the product. Therefore, it is not the correct reactant.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Option D contains a structure with a substituted benzene ring and a trifluoromethyl group, but it does not include the nitrile group, which is a key feature of the product. Therefore, it is not the correct reactant.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Based on the analysis, option A is the only one that contains the key features of the product, including the nitrile group and the substituted benzene ring with a chlorine atom and a trifluoromethyl group. Therefore, it is the correct reactant.
(TaskRunner pid=2001144) </reasoning>
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) A
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144) [ground_truth] A
(TaskRunner pid=2001144) [score] 1.0
(TaskRunner pid=2001144) [acc] 1.0
(TaskRunner pid=2001144) [pred] A
(TaskRunner pid=2001144) [incorrect_format] 1
(TaskRunner pid=2001144) [feedback]
(TaskRunner pid=2001144) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20
(WorkerDict pid=2001534) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10/actor
(TaskRunner pid=2001144) step:20 - global_seqlen/min:17276 - global_seqlen/max:22218 - global_seqlen/minmax_diff:4942 - global_seqlen/balanced_min:20235 - global_seqlen/balanced_max:20243 - global_seqlen/mean:20241.0 - actor/entropy:0.28219664096832275 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7399708032608032 - training/rollout_probs_diff_mean:0.006355004850775003 - training/rollout_probs_diff_std:0.015308930538594723 - training/rollout_actor_probs_pearson_corr:0.997978687286377 - rollout_corr/rollout_is_mean:0.9999727010726929 - rollout_corr/rollout_is_ratio_fraction_high:4.4778793380828574e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.83628856879659e-05 - rollout_corr/rollout_is_max:4.32278299331665 - rollout_corr/rollout_is_min:0.15594974160194397 - rollout_corr/rollout_is_std:0.04674001783132553 - rollout_corr/rollout_is_eff_sample_size:0.9978200140717158 - rollout_corr/rollout_is_seq_mean:1.000002384185791 - rollout_corr/rollout_is_seq_std:0.0030107342172414064 - rollout_corr/rollout_is_seq_max:1.0149316787719727 - rollout_corr/rollout_is_seq_min:0.9921532273292542 - rollout_corr/rollout_is_seq_max_deviation:0.014931678771972656 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3529459624551237) - rollout_corr/training_log_ppl:np.float64(0.2993746734573506) - rollout_corr/kl:np.float64(0.0010688752403762436) - rollout_corr/k3_kl:np.float64(0.00127186873487517) - rollout_corr/rollout_ppl:np.float64(1.3514942764304578) - rollout_corr/rollout_log_ppl:np.float64(0.2983057960518636) - rollout_corr/log_ppl_diff:np.float64(0.001068877405487001) - rollout_corr/log_ppl_abs_diff:np.float64(0.00255649903556332) - rollout_corr/log_ppl_diff_max:np.float64(0.009956508874893188) - rollout_corr/log_ppl_diff_min:np.float64(-0.012415051460266113) - rollout_corr/ppl_ratio:np.float64(1.0010744866449386) - rollout_corr/chi2_token:np.float64(0.0029958051163703203) - rollout_corr/chi2_seq:np.float64(1.5550862767267972) - actor/pg_loss:np.float64(6.246170960366726e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001650054843594262) - actor/ppo_kl:np.float64(-0.00011158695716773082) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6792175024747849) - perf/mfu/actor:np.float64(0.05850701352285697) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.9379768371582) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.5059523809523809) - val-aux/sciknoweval/reward/std@16:np.float64(0.23778516270573505) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6026523809523809) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1932973883686548) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.40893809523809527) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.20875811188155774) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5054571428571429) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.23696401290067473) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.676952380952381) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1373711977692103) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.32501428571428564) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1616758407656812) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.5247809523809523) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.18888641270937753) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7311047619047618) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09103040867235787) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.25953333333333334) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.11246263927053048) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.5349142857142857) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13427708237799116) - val-aux/sciknoweval/reward/best@16/mean:n
(TaskRunner pid=2001144) Training Progress: 20%|██ | 20/100 [15:06<1:37:29, 73.12s/it]
(TaskRunner pid=2001144) step:21 - global_seqlen/min:17686 - global_seqlen/max:26196 - global_seqlen/minmax_diff:8510 - global_seqlen/balanced_min:20180 - global_seqlen/balanced_max:27133 - global_seqlen/mean:21051.0 - actor/entropy:0.26770225167274475 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37976235151290894 - training/rollout_probs_diff_mean:0.005613590124994516 - training/rollout_probs_diff_std:0.014045730233192444 - training/rollout_actor_probs_pearson_corr:0.9982647895812988 - rollout_corr/rollout_is_mean:1.0001658201217651 - rollout_corr/rollout_is_ratio_fraction_high:2.085244886984583e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.213111944613047e-05 - rollout_corr/rollout_is_max:2.2679336071014404 - rollout_corr/rollout_is_min:0.33563682436943054 - rollout_corr/rollout_is_std:0.043513230979442596 - rollout_corr/rollout_is_eff_sample_size:0.9981108896023968 - rollout_corr/rollout_is_seq_mean:1.0002033710479736 - rollout_corr/rollout_is_seq_std:0.002796340500935912 - rollout_corr/rollout_is_seq_max:1.009148120880127 - rollout_corr/rollout_is_seq_min:0.9921823143959045 - rollout_corr/rollout_is_seq_max_deviation:0.009148120880126953 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3646910316310823) - rollout_corr/training_log_ppl:np.float64(0.30655970462976256) - rollout_corr/kl:np.float64(0.001067804822410423) - rollout_corr/k3_kl:np.float64(0.0013406110108462599) - rollout_corr/rollout_ppl:np.float64(1.3631728258915246) - rollout_corr/rollout_log_ppl:np.float64(0.30549189744488103) - rollout_corr/log_ppl_diff:np.float64(0.0010678071848815307) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025666777073638514) - rollout_corr/log_ppl_diff_max:np.float64(0.013620197772979736) - rollout_corr/log_ppl_diff_min:np.float64(-0.007804185152053833) - rollout_corr/ppl_ratio:np.float64(1.0010736973490566) - rollout_corr/chi2_token:np.float64(0.003265139413997531) - rollout_corr/chi2_seq:np.float64(1.1484402937348932) - actor/pg_loss:np.float64(-1.9012950360774994e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0018589328310554265) - actor/ppo_kl:np.float64(0.00015795662651640896) - actor/pg_clipfrac_lower:np.float64(5.160171895113308e-06) - actor/grad_norm:np.float64(0.7667596787214279) - perf/mfu/actor:np.float64(0.056962691555772035) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(103.04821491241455) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.021594014018774033 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.021594014018774033 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:374.65625 - response_length/max:8192.0 - response_length/min:104.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:374.65625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:283.1875 - prompt_length/max:669.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013934634625911713 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2651803772896528) - timing_s/agent_loop/generate_sequences/max:np.float64(50.0744674410671) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.779767223255476) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(
(TaskRunner pid=2001144) Training Progress: 21%|██ | 21/100 [16:16<1:34:55, 72.09s/it]
(TaskRunner pid=2001144) step:22 - global_seqlen/min:15847 - global_seqlen/max:19566 - global_seqlen/minmax_diff:3719 - global_seqlen/balanced_min:17806 - global_seqlen/balanced_max:17809 - global_seqlen/mean:17808.0 - actor/entropy:0.3063713610172272 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21458029747009277 - training/rollout_probs_diff_mean:0.006359769497066736 - training/rollout_probs_diff_std:0.014371220022439957 - training/rollout_actor_probs_pearson_corr:0.9982636570930481 - rollout_corr/rollout_is_mean:0.9997437596321106 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.5960540369851515e-05 - rollout_corr/rollout_is_max:1.950512170791626 - rollout_corr/rollout_is_min:0.36278146505355835 - rollout_corr/rollout_is_std:0.045671265572309494 - rollout_corr/rollout_is_eff_sample_size:0.9979173494533444 - rollout_corr/rollout_is_seq_mean:0.9996254444122314 - rollout_corr/rollout_is_seq_std:0.0029419215861707926 - rollout_corr/rollout_is_seq_max:1.0080243349075317 - rollout_corr/rollout_is_seq_min:0.9865586161613464 - rollout_corr/rollout_is_seq_max_deviation:0.013441383838653564 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.373158656526357) - rollout_corr/training_log_ppl:np.float64(0.31456172408070415) - rollout_corr/kl:np.float64(0.001310535317575301) - rollout_corr/k3_kl:np.float64(0.0011426664440250534) - rollout_corr/rollout_ppl:np.float64(1.3713175556622446) - rollout_corr/rollout_log_ppl:np.float64(0.3132511889853049) - rollout_corr/log_ppl_diff:np.float64(0.0013105350953992456) - rollout_corr/log_ppl_abs_diff:np.float64(0.00267272288328968) - rollout_corr/log_ppl_diff_max:np.float64(0.014944106340408325) - rollout_corr/log_ppl_diff_min:np.float64(-0.007561683654785156) - rollout_corr/ppl_ratio:np.float64(1.0013165678828955) - rollout_corr/chi2_token:np.float64(0.0019434010609984398) - rollout_corr/chi2_seq:np.float64(1.2117313593626022) - actor/pg_loss:np.float64(0.0001189985778182745) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009317383892266662) - actor/ppo_kl:np.float64(-0.0001350449852282054) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.49233904480934143) - perf/mfu/actor:np.float64(0.05169195387938463) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.73754215240479) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.34765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006569639313966036 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:-0.006569639313966036 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:300.9375 - response_length/max:751.0 - response_length/min:116.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:300.9375 - response_length_non_aborted/max:751.0 - response_length_non_aborted/min:116.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.5625 - prompt_length/max:592.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011088140308856964 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3909703567624092) - timing_s/agent_loop/generate_sequences/max:np.float64(5.8826358411461115) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2543173192534596) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.8826358411461115) - timing_s/agent_loop/slowest/too
(TaskRunner pid=2001144) Training Progress: 22%|██▏ | 22/100 [16:40<1:15:05, 57.77s/it]
(TaskRunner pid=2001144) step:23 - global_seqlen/min:16424 - global_seqlen/max:24003 - global_seqlen/minmax_diff:7579 - global_seqlen/balanced_min:19785 - global_seqlen/balanced_max:19791 - global_seqlen/mean:19790.0 - actor/entropy:0.28699958324432373 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31861215829849243 - training/rollout_probs_diff_mean:0.0062312777154147625 - training/rollout_probs_diff_std:0.014602634124457836 - training/rollout_actor_probs_pearson_corr:0.9981580376625061 - rollout_corr/rollout_is_mean:0.9998175501823425 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.585893839248456e-05 - rollout_corr/rollout_is_max:1.8458908796310425 - rollout_corr/rollout_is_min:0.47339972853660583 - rollout_corr/rollout_is_std:0.045659519731998444 - rollout_corr/rollout_is_eff_sample_size:0.9979188927281635 - rollout_corr/rollout_is_seq_mean:0.9999505281448364 - rollout_corr/rollout_is_seq_std:0.0028406144119799137 - rollout_corr/rollout_is_seq_max:1.0108468532562256 - rollout_corr/rollout_is_seq_min:0.9906536936759949 - rollout_corr/rollout_is_seq_max_deviation:0.010846853256225586 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3477039081044495) - rollout_corr/training_log_ppl:np.float64(0.2955174881790299) - rollout_corr/kl:np.float64(0.0010167940752410232) - rollout_corr/k3_kl:np.float64(0.0011501972365977053) - rollout_corr/rollout_ppl:np.float64(1.3463539532385767) - rollout_corr/rollout_log_ppl:np.float64(0.2945006933878176) - rollout_corr/log_ppl_diff:np.float64(0.0010167947912123054) - rollout_corr/log_ppl_abs_diff:np.float64(0.002272587764309719) - rollout_corr/log_ppl_diff_max:np.float64(0.00822228193283081) - rollout_corr/log_ppl_diff_min:np.float64(-0.009223580360412598) - rollout_corr/ppl_ratio:np.float64(1.0010209241881967) - rollout_corr/chi2_token:np.float64(0.0026352107524871826) - rollout_corr/chi2_seq:np.float64(0.8436703111510724) - actor/pg_loss:np.float64(0.000612961477600038) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001325598243965942) - actor/ppo_kl:np.float64(-0.00010477143022491475) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6717194765806198) - perf/mfu/actor:np.float64(0.05668413331361297) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.2077407836914) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:0 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007040780037641525 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007040780037641525 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:340.71875 - response_length/max:896.0 - response_length/min:121.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:340.71875 - response_length_non_aborted/max:896.0 - response_length_non_aborted/min:121.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.71875 - prompt_length/max:766.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010039843618869781 - timing_s/agent_loop/generate_sequences/min:np.float64(1.498599922284484) - timing_s/agent_loop/generate_sequences/max:np.float64(6.821983525529504) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6944557968745357) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.821983525529504) - timing_s/agent_loop/slowest/tool_
(TaskRunner pid=2001144) Training Progress: 23%|██▎ | 23/100 [17:05<1:01:39, 48.04s/it]
(TaskRunner pid=2001144) step:24 - global_seqlen/min:18442 - global_seqlen/max:21530 - global_seqlen/minmax_diff:3088 - global_seqlen/balanced_min:20056 - global_seqlen/balanced_max:20060 - global_seqlen/mean:20057.875 - actor/entropy:0.29705044627189636 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4429609775543213 - training/rollout_probs_diff_mean:0.006329038180410862 - training/rollout_probs_diff_std:0.014686203561723232 - training/rollout_actor_probs_pearson_corr:0.9981698393821716 - rollout_corr/rollout_is_mean:0.9996858835220337 - rollout_corr/rollout_is_ratio_fraction_high:1.1275609722360969e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.020487777888775e-05 - rollout_corr/rollout_is_max:2.2383830547332764 - rollout_corr/rollout_is_min:0.37353411316871643 - rollout_corr/rollout_is_std:0.045886747539043427 - rollout_corr/rollout_is_eff_sample_size:0.9978975247320991 - rollout_corr/rollout_is_seq_mean:0.9996603727340698 - rollout_corr/rollout_is_seq_std:0.0026348524261265993 - rollout_corr/rollout_is_seq_max:1.0088465213775635 - rollout_corr/rollout_is_seq_min:0.9900056719779968 - rollout_corr/rollout_is_seq_max_deviation:0.009994328022003174 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.358842563815415) - rollout_corr/training_log_ppl:np.float64(0.30429567405371927) - rollout_corr/kl:np.float64(0.0015967130124918683) - rollout_corr/k3_kl:np.float64(0.0012503337802627357) - rollout_corr/rollout_ppl:np.float64(1.356625474523753) - rollout_corr/rollout_log_ppl:np.float64(0.3026989574136678) - rollout_corr/log_ppl_diff:np.float64(0.0015967166400514543) - rollout_corr/log_ppl_abs_diff:np.float64(0.002631262585055083) - rollout_corr/log_ppl_diff_max:np.float64(0.011732339859008789) - rollout_corr/log_ppl_diff_min:np.float64(-0.007120221853256226) - rollout_corr/ppl_ratio:np.float64(1.0016025137156248) - rollout_corr/chi2_token:np.float64(0.0018508185166865587) - rollout_corr/chi2_seq:np.float64(0.7879260319750756) - actor/pg_loss:np.float64(0.00023575290106236935) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010448106727380946) - actor/ppo_kl:np.float64(0.0001596968348245298) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.49844516813755035) - perf/mfu/actor:np.float64(0.058504430481614356) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.06549835205078) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:0 - critic/score/mean:0.5703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0021705548278987408 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0021705548278987408 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:346.43359375 - response_length/max:665.0 - response_length/min:140.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:346.43359375 - response_length_non_aborted/max:665.0 - response_length_non_aborted/min:140.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.375 - prompt_length/max:848.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014815852046012878 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6617038361728191) - timing_s/agent_loop/generate_sequences/max:np.float64(5.792027268558741) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7918190722484724) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.792027268558741) - tim
(TaskRunner pid=2001144) Training Progress: 24%|██▍ | 24/100 [17:30<51:47, 40.89s/it]
(TaskRunner pid=2001144) step:25 - global_seqlen/min:18726 - global_seqlen/max:24479 - global_seqlen/minmax_diff:5753 - global_seqlen/balanced_min:21474 - global_seqlen/balanced_max:21479 - global_seqlen/mean:21477.25 - actor/entropy:0.30059802532196045 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4104272723197937 - training/rollout_probs_diff_mean:0.006044417154043913 - training/rollout_probs_diff_std:0.014332070015370846 - training/rollout_actor_probs_pearson_corr:0.9983054399490356 - rollout_corr/rollout_is_mean:0.9999333024024963 - rollout_corr/rollout_is_ratio_fraction_high:2.120576027664356e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.482304110657424e-05 - rollout_corr/rollout_is_max:3.3156065940856934 - rollout_corr/rollout_is_min:0.19203001260757446 - rollout_corr/rollout_is_std:0.04466639459133148 - rollout_corr/rollout_is_eff_sample_size:0.9980086483054951 - rollout_corr/rollout_is_seq_mean:0.9998590350151062 - rollout_corr/rollout_is_seq_std:0.0025852108374238014 - rollout_corr/rollout_is_seq_max:1.0069289207458496 - rollout_corr/rollout_is_seq_min:0.988861083984375 - rollout_corr/rollout_is_seq_max_deviation:0.011138916015625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.374245997518301) - rollout_corr/training_log_ppl:np.float64(0.3143928272766061) - rollout_corr/kl:np.float64(0.000824890810179113) - rollout_corr/k3_kl:np.float64(0.001163450876333627) - rollout_corr/rollout_ppl:np.float64(1.3731129108928144) - rollout_corr/rollout_log_ppl:np.float64(0.3135679374099709) - rollout_corr/log_ppl_diff:np.float64(0.0008248898666352034) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023570185294374824) - rollout_corr/log_ppl_diff_max:np.float64(0.011101752519607544) - rollout_corr/log_ppl_diff_min:np.float64(-0.009776294231414795) - rollout_corr/ppl_ratio:np.float64(1.000829664291814) - rollout_corr/chi2_token:np.float64(0.0031572484876960516) - rollout_corr/chi2_seq:np.float64(2.9503266639076173) - actor/pg_loss:np.float64(-5.722884088754654e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011451504933575052) - actor/ppo_kl:np.float64(-0.0003840430452992649) - actor/pg_clipfrac_lower:np.float64(1.1034604540327564e-05) - actor/grad_norm:np.float64(0.576805867254734) - perf/mfu/actor:np.float64(0.06308374010023463) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.13753128051758) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:0 - critic/score/mean:0.48046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0072536952793598175 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0072536952793598175 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:368.4140625 - response_length/max:933.0 - response_length/min:148.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:368.4140625 - response_length_non_aborted/max:933.0 - response_length_non_aborted/min:148.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.75 - prompt_length/max:646.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001402292400598526 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7118809763342142) - timing_s/agent_loop/generate_sequences/max:np.float64(7.255187263712287) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.907090678498207) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.255187263712287) -
(TaskRunner pid=2001144) Training Progress: 25%|██▌ | 25/100 [17:55<45:23, 36.31s/it]
(TaskRunner pid=2001144) step:26 - global_seqlen/min:18804 - global_seqlen/max:22969 - global_seqlen/minmax_diff:4165 - global_seqlen/balanced_min:20662 - global_seqlen/balanced_max:20668 - global_seqlen/mean:20665.25 - actor/entropy:0.32698380947113037 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102371037006378 - training/rollout_probs_diff_mean:0.006528716068714857 - training/rollout_probs_diff_std:0.014776989817619324 - training/rollout_actor_probs_pearson_corr:0.9982529878616333 - rollout_corr/rollout_is_mean:1.0001572370529175 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00012298472574912012 - rollout_corr/rollout_is_max:1.849212646484375 - rollout_corr/rollout_is_min:0.2505255937576294 - rollout_corr/rollout_is_std:0.04611092060804367 - rollout_corr/rollout_is_eff_sample_size:0.9978790065423667 - rollout_corr/rollout_is_seq_mean:1.000186562538147 - rollout_corr/rollout_is_seq_std:0.0025366947520524263 - rollout_corr/rollout_is_seq_max:1.0072381496429443 - rollout_corr/rollout_is_seq_min:0.9920287132263184 - rollout_corr/rollout_is_seq_max_deviation:0.00797128677368164 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4002767349593341) - rollout_corr/training_log_ppl:np.float64(0.3331134519830812) - rollout_corr/kl:np.float64(0.0012892596767173359) - rollout_corr/k3_kl:np.float64(0.0012173368421599662) - rollout_corr/rollout_ppl:np.float64(1.3984757866710424) - rollout_corr/rollout_log_ppl:np.float64(0.3318241920496803) - rollout_corr/log_ppl_diff:np.float64(0.0012892599334008992) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022995691397227347) - rollout_corr/log_ppl_diff_max:np.float64(0.009872853755950928) - rollout_corr/log_ppl_diff_min:np.float64(-0.00582653284072876) - rollout_corr/ppl_ratio:np.float64(1.0012937453575432) - rollout_corr/chi2_token:np.float64(0.002282004104927182) - rollout_corr/chi2_seq:np.float64(0.6940536317415535) - actor/pg_loss:np.float64(-4.73198015242815e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008514723272128322) - actor/ppo_kl:np.float64(0.00035901207717614625) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.573113739490509) - perf/mfu/actor:np.float64(0.060727617808351154) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(101.87111282348633) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:0 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006875964463688433 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006875964463688433 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:349.3828125 - response_length/max:723.0 - response_length/min:147.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:349.3828125 - response_length_non_aborted/max:723.0 - response_length_non_aborted/min:147.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.40625 - prompt_length/max:709.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014569610357284546 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3159475959837437) - timing_s/agent_loop/generate_sequences/max:np.float64(6.282181400805712) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5772640933064395) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.282181400805712) - timing_s/agent_loop/slowest/too
(TaskRunner pid=2001144) Training Progress: 26%|██▌ | 26/100 [18:20<40:27, 32.81s/it]
(TaskRunner pid=2001144) step:27 - global_seqlen/min:17657 - global_seqlen/max:22007 - global_seqlen/minmax_diff:4350 - global_seqlen/balanced_min:19736 - global_seqlen/balanced_max:19741 - global_seqlen/mean:19739.75 - actor/entropy:0.32098695635795593 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.352716326713562 - training/rollout_probs_diff_mean:0.006316934246569872 - training/rollout_probs_diff_std:0.014347612857818604 - training/rollout_actor_probs_pearson_corr:0.9983111619949341 - rollout_corr/rollout_is_mean:0.9999595284461975 - rollout_corr/rollout_is_ratio_fraction_high:1.0875475709326565e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.350190283730626e-05 - rollout_corr/rollout_is_max:3.377265214920044 - rollout_corr/rollout_is_min:0.39215895533561707 - rollout_corr/rollout_is_std:0.04489400237798691 - rollout_corr/rollout_is_eff_sample_size:0.9979884637659365 - rollout_corr/rollout_is_seq_mean:0.9999179244041443 - rollout_corr/rollout_is_seq_std:0.0026965278666466475 - rollout_corr/rollout_is_seq_max:1.0078588724136353 - rollout_corr/rollout_is_seq_min:0.9907032251358032 - rollout_corr/rollout_is_seq_max_deviation:0.009296774864196777 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.398104117717594) - rollout_corr/training_log_ppl:np.float64(0.33107622820534743) - rollout_corr/kl:np.float64(0.001073108376374421) - rollout_corr/k3_kl:np.float64(0.00107604429138064) - rollout_corr/rollout_ppl:np.float64(1.3965696305967867) - rollout_corr/rollout_log_ppl:np.float64(0.3300031187827699) - rollout_corr/log_ppl_diff:np.float64(0.001073109422577545) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021245690004434437) - rollout_corr/log_ppl_diff_max:np.float64(0.009940654039382935) - rollout_corr/log_ppl_diff_min:np.float64(-0.006391346454620361) - rollout_corr/ppl_ratio:np.float64(1.0010769127402455) - rollout_corr/chi2_token:np.float64(0.0021732328459620476) - rollout_corr/chi2_seq:np.float64(0.5239306807052344) - actor/pg_loss:np.float64(0.0003134759608656168) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008846624223224353) - actor/ppo_kl:np.float64(-7.025213841149736e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4928107485175133) - perf/mfu/actor:np.float64(0.057878429104460366) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(101.87171936035156) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002317835809662938 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002317835809662938 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:359.1796875 - response_length/max:831.0 - response_length/min:148.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:359.1796875 - response_length_non_aborted/max:831.0 - response_length_non_aborted/min:148.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:257.6875 - prompt_length/max:380.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013938359916210175 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7921749632805586) - timing_s/agent_loop/generate_sequences/max:np.float64(6.5331399366259575) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8684371362251113) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.5331399366259575) - timing_s/agen
(TaskRunner pid=2001144) Training Progress: 27%|██▋ | 27/100 [18:45<37:00, 30.42s/it]
(TaskRunner pid=2001144) step:28 - global_seqlen/min:17598 - global_seqlen/max:21183 - global_seqlen/minmax_diff:3585 - global_seqlen/balanced_min:19861 - global_seqlen/balanced_max:19868 - global_seqlen/mean:19865.25 - actor/entropy:0.31646955013275146 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35351768136024475 - training/rollout_probs_diff_mean:0.006314622238278389 - training/rollout_probs_diff_std:0.014190751127898693 - training/rollout_actor_probs_pearson_corr:0.9983447194099426 - rollout_corr/rollout_is_mean:0.9999757409095764 - rollout_corr/rollout_is_ratio_fraction_high:3.3637566957622766e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1212521712877788e-05 - rollout_corr/rollout_is_max:2.7362897396087646 - rollout_corr/rollout_is_min:0.4983958601951599 - rollout_corr/rollout_is_std:0.04502525180578232 - rollout_corr/rollout_is_eff_sample_size:0.9979767096161316 - rollout_corr/rollout_is_seq_mean:0.9999407529830933 - rollout_corr/rollout_is_seq_std:0.0028120612259954214 - rollout_corr/rollout_is_seq_max:1.0084820985794067 - rollout_corr/rollout_is_seq_min:0.9904807806015015 - rollout_corr/rollout_is_seq_max_deviation:0.009519219398498535 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3901182138361037) - rollout_corr/training_log_ppl:np.float64(0.3261186132731382) - rollout_corr/kl:np.float64(0.0011101681932430552) - rollout_corr/k3_kl:np.float64(0.0011281819336090848) - rollout_corr/rollout_ppl:np.float64(1.388537334278226) - rollout_corr/rollout_log_ppl:np.float64(0.32500844306196086) - rollout_corr/log_ppl_diff:np.float64(0.0011101702111773193) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022507039248012006) - rollout_corr/log_ppl_diff_max:np.float64(0.011187613010406494) - rollout_corr/log_ppl_diff_min:np.float64(-0.007259994745254517) - rollout_corr/ppl_ratio:np.float64(1.0011146296747029) - rollout_corr/chi2_token:np.float64(0.002260454697534442) - rollout_corr/chi2_seq:np.float64(1.3759961093310267) - actor/pg_loss:np.float64(8.262076880782843e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007168550359892834) - actor/ppo_kl:np.float64(-1.0054963227368319e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5459058359265327) - perf/mfu/actor:np.float64(0.0575105816113208) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(101.81379318237305) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008926569484174252 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008926569484174252 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:348.3828125 - response_length/max:742.0 - response_length/min:130.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:348.3828125 - response_length_non_aborted/max:742.0 - response_length_non_aborted/min:130.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.40625 - prompt_length/max:443.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014387629926204681 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4666844885796309) - timing_s/agent_loop/generate_sequences/max:np.float64(6.381468288600445) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.750248634140007) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.381468288600445) - timing_s
(TaskRunner pid=2001144) Training Progress: 28%|██▊ | 28/100 [19:09<34:26, 28.70s/it]
(TaskRunner pid=2001144) step:29 - global_seqlen/min:17501 - global_seqlen/max:25695 - global_seqlen/minmax_diff:8194 - global_seqlen/balanced_min:20842 - global_seqlen/balanced_max:20857 - global_seqlen/mean:20852.875 - actor/entropy:0.3055855333805084 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.353514701128006 - training/rollout_probs_diff_mean:0.0059598712250590324 - training/rollout_probs_diff_std:0.013857239857316017 - training/rollout_actor_probs_pearson_corr:0.9984294772148132 - rollout_corr/rollout_is_mean:1.0001864433288574 - rollout_corr/rollout_is_ratio_fraction_high:1.0419380487292074e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.1677521949168295e-05 - rollout_corr/rollout_is_max:2.1047282218933105 - rollout_corr/rollout_is_min:0.2578449249267578 - rollout_corr/rollout_is_std:0.04363497346639633 - rollout_corr/rollout_is_eff_sample_size:0.9981003201352026 - rollout_corr/rollout_is_seq_mean:1.0002881288528442 - rollout_corr/rollout_is_seq_std:0.0026735435239970684 - rollout_corr/rollout_is_seq_max:1.0167275667190552 - rollout_corr/rollout_is_seq_min:0.9913028478622437 - rollout_corr/rollout_is_seq_max_deviation:0.016727566719055176 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3838739884085953) - rollout_corr/training_log_ppl:np.float64(0.32128043269040063) - rollout_corr/kl:np.float64(0.001136318282488613) - rollout_corr/k3_kl:np.float64(0.0011952122654292907) - rollout_corr/rollout_ppl:np.float64(1.3822658765129745) - rollout_corr/rollout_log_ppl:np.float64(0.32014411230920814) - rollout_corr/log_ppl_diff:np.float64(0.0011363203811924905) - rollout_corr/log_ppl_abs_diff:np.float64(0.00250161238363944) - rollout_corr/log_ppl_diff_max:np.float64(0.012654438614845276) - rollout_corr/log_ppl_diff_min:np.float64(-0.007744014263153076) - rollout_corr/ppl_ratio:np.float64(1.0011417360510677) - rollout_corr/chi2_token:np.float64(0.002458808245137334) - rollout_corr/chi2_seq:np.float64(1.3620945131406188) - actor/pg_loss:np.float64(-0.0004294891841709614) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014547093599048821) - actor/ppo_kl:np.float64(0.0004023276947764387) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5281674265861511) - perf/mfu/actor:np.float64(0.060255372543370184) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(101.82099151611328) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0016423547640442848 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0016423547640442848 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:374.90234375 - response_length/max:1100.0 - response_length/min:128.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:374.90234375 - response_length_non_aborted/max:1100.0 - response_length_non_aborted/min:128.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.75 - prompt_length/max:492.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013095326721668243 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5605962127447128) - timing_s/agent_loop/generate_sequences/max:np.float64(8.123685423284769) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9714328672125703) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.123685423284769) - timing_
(TaskRunner pid=2001144) Training Progress: 29%|██▉ | 29/100 [19:36<33:14, 28.09s/it]
(TaskRunner pid=2001144) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30}
(TaskRunner pid=2001144) validation generation end
(TaskRunner pid=2001144) [prompt] system
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2001144) <reasoning>
(TaskRunner pid=2001144) ...
(TaskRunner pid=2001144) </reasoning>
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) ...
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) A
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) user
(TaskRunner pid=2001144) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2001144) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2001144) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2001144) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2001144) Please reason step by step.
(TaskRunner pid=2001144) assistant
(TaskRunner pid=2001144) <think>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) </think>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) [response] <reasoning>
(TaskRunner pid=2001144) The question asks which of the given molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". This product contains a complex structure with a nitro group (–NO2) and a substituted benzene ring with a chlorine atom and a trifluoromethyl group (–C(F)(F)F).
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) To determine the correct reactant, we need to analyze the structure of the product and compare it with the structures of the given options. The correct reactant should be a precursor that, when reacted, would form the product through a chemical reaction such as nucleophilic substitution, elimination, or condensation.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Option A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2001144) This structure contains a carbonyl group (C=O), an amide group (–CONH–), and a substituted benzene ring with a chlorine atom and a trifluoromethyl group. It closely resembles the product, but the placement of the nitro group and the amide group may not match the exact structure of the product.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Option B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2001144) This structure contains a chlorine atom and a substituted benzene ring with a chlorine atom, but it lacks the trifluoromethyl group and the nitro group that are present in the product. It also has a different arrangement of the nitrogen and carbon atoms, which may not align with the product's structure.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Option C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2001144) This structure contains a carbonyl group, an amide group, and a substituted benzene ring with a chlorine atom and a trifluoromethyl group. It also has a complex arrangement of the nitrogen and carbon atoms that may align with the product's structure. However, the exact placement of the nitro group and the amide group may not match the product.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Option D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2001144) This structure contains a carbonyl group, an amide group, and a trifluoromethyl group. However, it lacks the chlorine atom and the nitro group that are present in the product. It also has a different arrangement of the carbon atoms, which may not align with the product's structure.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Based on the analysis, Option A most closely resembles the product in terms of the presence of the chlorine atom, the trifluoromethyl group, and the amide group. The structure of Option A also matches the placement of the nitro group and the amide group in the product, making it the most likely correct reactant.
(TaskRunner pid=2001144) </reasoning>
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) A
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144) [ground_truth] A
(TaskRunner pid=2001144) [score] 1.0
(TaskRunner pid=2001144) [acc] 1.0
(TaskRunner pid=2001144) [pred] A
(TaskRunner pid=2001144) [incorrect_format] 1
(TaskRunner pid=2001144) [feedback]
(TaskRunner pid=2001144) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30
(WorkerDict pid=2001534) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20/actor
(TaskRunner pid=2001144) step:30 - global_seqlen/min:16733 - global_seqlen/max:23886 - global_seqlen/minmax_diff:7153 - global_seqlen/balanced_min:20246 - global_seqlen/balanced_max:20254 - global_seqlen/mean:20251.375 - actor/entropy:0.3016992211341858 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28880876302719116 - training/rollout_probs_diff_mean:0.006295952945947647 - training/rollout_probs_diff_std:0.014442361891269684 - training/rollout_actor_probs_pearson_corr:0.9982203245162964 - rollout_corr/rollout_is_mean:1.000058889389038 - rollout_corr/rollout_is_ratio_fraction_high:1.2071025594195817e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.4142051188391633e-05 - rollout_corr/rollout_is_max:2.5121476650238037 - rollout_corr/rollout_is_min:0.4851633310317993 - rollout_corr/rollout_is_std:0.04639057442545891 - rollout_corr/rollout_is_eff_sample_size:0.9978527736167245 - rollout_corr/rollout_is_seq_mean:1.0000797510147095 - rollout_corr/rollout_is_seq_std:0.0028501409105956554 - rollout_corr/rollout_is_seq_max:1.008534550666809 - rollout_corr/rollout_is_seq_min:0.9904745221138 - rollout_corr/rollout_is_seq_max_deviation:0.009525477886199951 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3812061371281743) - rollout_corr/training_log_ppl:np.float64(0.31866738124517724) - rollout_corr/kl:np.float64(0.0013764771011217647) - rollout_corr/k3_kl:np.float64(0.0012658665879143882) - rollout_corr/rollout_ppl:np.float64(1.3792695426382124) - rollout_corr/rollout_log_ppl:np.float64(0.3172909033601172) - rollout_corr/log_ppl_diff:np.float64(0.0013764778850600123) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024694554158486426) - rollout_corr/log_ppl_diff_max:np.float64(0.015178769826889038) - rollout_corr/log_ppl_diff_min:np.float64(-0.007453501224517822) - rollout_corr/ppl_ratio:np.float64(1.001381719717756) - rollout_corr/chi2_token:np.float64(0.002375947078689933) - rollout_corr/chi2_seq:np.float64(0.464690319262445) - actor/pg_loss:np.float64(-5.9206970036029816e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000814051581073727) - actor/ppo_kl:np.float64(0.0003232452908470407) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.542563185095787) - perf/mfu/actor:np.float64(0.05944302278824524) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(101.93528366088867) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.5526785714285715) - val-aux/sciknoweval/reward/std@16:np.float64(0.23017279671601099) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.647257142857143) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.18881560887294935) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.45902857142857134) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.20154151370415746) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5529761904761905) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2311364484256804) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7199238095238096) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.13703223183047653) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.37916666666666665) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.15573522404318324) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.5671761904761905) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.18103233978923747) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7743285714285715) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0913062339667138) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.3164952380952381) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.11233055469968115) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.5736714285714286) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.12973088161319943) - val-aux/sciknoweval/reward/best@16/m
(TaskRunner pid=2001144) Training Progress: 30%|███ | 30/100 [22:12<1:17:38, 66.55s/it]
(TaskRunner pid=2001144) step:31 - global_seqlen/min:17714 - global_seqlen/max:24828 - global_seqlen/minmax_diff:7114 - global_seqlen/balanced_min:21250 - global_seqlen/balanced_max:21268 - global_seqlen/mean:21257.5 - actor/entropy:0.29266250133514404 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9358408451080322 - training/rollout_probs_diff_mean:0.006022932007908821 - training/rollout_probs_diff_std:0.015107459388673306 - training/rollout_actor_probs_pearson_corr:0.9981313347816467 - rollout_corr/rollout_is_mean:0.9999860525131226 - rollout_corr/rollout_is_ratio_fraction_high:8.128429180942476e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012192643771413714 - rollout_corr/rollout_is_max:2.934584856033325 - rollout_corr/rollout_is_min:0.001762050436809659 - rollout_corr/rollout_is_std:0.045394401997327805 - rollout_corr/rollout_is_eff_sample_size:0.9979434670549656 - rollout_corr/rollout_is_seq_mean:0.9999060034751892 - rollout_corr/rollout_is_seq_std:0.00254035578109324 - rollout_corr/rollout_is_seq_max:1.0100243091583252 - rollout_corr/rollout_is_seq_min:0.9903190732002258 - rollout_corr/rollout_is_seq_max_deviation:0.010024309158325195 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3719704658724368) - rollout_corr/training_log_ppl:np.float64(0.31175580773560796) - rollout_corr/kl:np.float64(0.0015448000047655341) - rollout_corr/k3_kl:np.float64(0.001254782076784977) - rollout_corr/rollout_ppl:np.float64(1.369816878810525) - rollout_corr/rollout_log_ppl:np.float64(0.310211006886675) - rollout_corr/log_ppl_diff:np.float64(0.0015448008489329368) - rollout_corr/log_ppl_abs_diff:np.float64(0.002411051915260032) - rollout_corr/log_ppl_diff_max:np.float64(0.016888737678527832) - rollout_corr/log_ppl_diff_min:np.float64(-0.006264418363571167) - rollout_corr/ppl_ratio:np.float64(1.0015502274036407) - rollout_corr/chi2_token:np.float64(0.0018021143041551113) - rollout_corr/chi2_seq:np.float64(1.5113360479008406) - actor/pg_loss:np.float64(-0.00010793737601488829) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007984668686731311) - actor/ppo_kl:np.float64(0.00030401401364699154) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4822354167699814) - perf/mfu/actor:np.float64(0.06236819814399034) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(104.04623031616211) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004728459753096104 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.004728459753096104 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:384.453125 - response_length/max:860.0 - response_length/min:124.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:384.453125 - response_length_non_aborted/max:860.0 - response_length_non_aborted/min:124.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.84375 - prompt_length/max:635.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.4140185713768e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5119797736406326) - timing_s/agent_loop/generate_sequences/max:np.float64(7.011765647679567) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.103028764759074) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.011765647679567) - timing_s/agent_loo
(TaskRunner pid=2001144) Training Progress: 31%|███ | 31/100 [22:38<1:02:22, 54.23s/it]
(TaskRunner pid=2001144) step:32 - global_seqlen/min:16795 - global_seqlen/max:22401 - global_seqlen/minmax_diff:5606 - global_seqlen/balanced_min:19287 - global_seqlen/balanced_max:19295 - global_seqlen/mean:19292.75 - actor/entropy:0.30621060729026794 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2896888256072998 - training/rollout_probs_diff_mean:0.006212353706359863 - training/rollout_probs_diff_std:0.014408687129616737 - training/rollout_actor_probs_pearson_corr:0.9982616901397705 - rollout_corr/rollout_is_mean:1.0001611709594727 - rollout_corr/rollout_is_ratio_fraction_high:1.1487122719699983e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.446136906859465e-05 - rollout_corr/rollout_is_max:2.504072666168213 - rollout_corr/rollout_is_min:0.3192140460014343 - rollout_corr/rollout_is_std:0.044921875 - rollout_corr/rollout_is_eff_sample_size:0.997986801546125 - rollout_corr/rollout_is_seq_mean:1.0000799894332886 - rollout_corr/rollout_is_seq_std:0.002760638715699315 - rollout_corr/rollout_is_seq_max:1.0099968910217285 - rollout_corr/rollout_is_seq_min:0.9901965260505676 - rollout_corr/rollout_is_seq_max_deviation:0.009996891021728516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3891568020917475) - rollout_corr/training_log_ppl:np.float64(0.3237839702051133) - rollout_corr/kl:np.float64(0.0013850933052239611) - rollout_corr/k3_kl:np.float64(0.0012262050348681441) - rollout_corr/rollout_ppl:np.float64(1.3871236578561366) - rollout_corr/rollout_log_ppl:np.float64(0.32239887581090443) - rollout_corr/log_ppl_diff:np.float64(0.0013850943942088634) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025525445526000112) - rollout_corr/log_ppl_diff_max:np.float64(0.009468793869018555) - rollout_corr/log_ppl_diff_min:np.float64(-0.007260739803314209) - rollout_corr/ppl_ratio:np.float64(1.0013904080260545) - rollout_corr/chi2_token:np.float64(0.0021125597413629293) - rollout_corr/chi2_seq:np.float64(1.0202762610279024) - actor/pg_loss:np.float64(0.000287185306660831) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010464300344210642) - actor/ppo_kl:np.float64(0.0003855656113355699) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5716015249490738) - perf/mfu/actor:np.float64(0.05562412475208506) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(103.71525573730469) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:0 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0008658418664708734 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0008658418664708734 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:340.0546875 - response_length/max:963.0 - response_length/min:117.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:340.0546875 - response_length_non_aborted/max:963.0 - response_length_non_aborted/min:117.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.84375 - prompt_length/max:404.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001262817531824112 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4239984285086393) - timing_s/agent_loop/generate_sequences/max:np.float64(7.210344027727842) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.63726398122526) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.210344027727842) - timing_s/agent_loop
(TaskRunner pid=2001144) Training Progress: 32%|███▏ | 32/100 [23:04<51:48, 45.71s/it]
(TaskRunner pid=2001144) step:33 - global_seqlen/min:17872 - global_seqlen/max:23695 - global_seqlen/minmax_diff:5823 - global_seqlen/balanced_min:20391 - global_seqlen/balanced_max:20396 - global_seqlen/mean:20394.375 - actor/entropy:0.31763318181037903 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.33988261222839355 - training/rollout_probs_diff_mean:0.006426535081118345 - training/rollout_probs_diff_std:0.014441907405853271 - training/rollout_actor_probs_pearson_corr:0.9983048439025879 - rollout_corr/rollout_is_mean:0.9998688697814941 - rollout_corr/rollout_is_ratio_fraction_high:2.2738900952390395e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1369450476195198e-05 - rollout_corr/rollout_is_max:2.273996591567993 - rollout_corr/rollout_is_min:0.37990230321884155 - rollout_corr/rollout_is_std:0.04559354856610298 - rollout_corr/rollout_is_eff_sample_size:0.9979249471600874 - rollout_corr/rollout_is_seq_mean:0.9998652338981628 - rollout_corr/rollout_is_seq_std:0.0026055655907839537 - rollout_corr/rollout_is_seq_max:1.0073285102844238 - rollout_corr/rollout_is_seq_min:0.9927304983139038 - rollout_corr/rollout_is_seq_max_deviation:0.007328510284423828 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3831165824085474) - rollout_corr/training_log_ppl:np.float64(0.32152386428788304) - rollout_corr/kl:np.float64(0.0011739729767654694) - rollout_corr/k3_kl:np.float64(0.001127043632322966) - rollout_corr/rollout_ppl:np.float64(1.381461517419666) - rollout_corr/rollout_log_ppl:np.float64(0.3203498890798073) - rollout_corr/log_ppl_diff:np.float64(0.001173975208075717) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022589289292227477) - rollout_corr/log_ppl_diff_max:np.float64(0.011483490467071533) - rollout_corr/log_ppl_diff_min:np.float64(-0.005533456802368164) - rollout_corr/ppl_ratio:np.float64(1.0011782080400735) - rollout_corr/chi2_token:np.float64(0.0021792524494230747) - rollout_corr/chi2_seq:np.float64(1.3192662147339433) - actor/pg_loss:np.float64(-0.00013960874639451504) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007804996953382215) - actor/ppo_kl:np.float64(-3.183357045544177e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.49189724028110504) - perf/mfu/actor:np.float64(0.059522535117244174) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.62992143630981) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005180205684155226 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005180205684155226 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:343.57421875 - response_length/max:781.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:343.57421875 - response_length_non_aborted/max:781.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.75 - prompt_length/max:474.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010042823851108551 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9423268754035234) - timing_s/agent_loop/generate_sequences/max:np.float64(6.265337560325861) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.716529831646767) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.265337560325861) - timin
(TaskRunner pid=2001144) Training Progress: 33%|███▎ | 33/100 [23:28<43:57, 39.36s/it]
(TaskRunner pid=2001144) step:34 - global_seqlen/min:14820 - global_seqlen/max:23905 - global_seqlen/minmax_diff:9085 - global_seqlen/balanced_min:20242 - global_seqlen/balanced_max:20245 - global_seqlen/mean:20244.125 - actor/entropy:0.3040616512298584 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45596012473106384 - training/rollout_probs_diff_mean:0.0062696561217308044 - training/rollout_probs_diff_std:0.014528535306453705 - training/rollout_actor_probs_pearson_corr:0.9982279539108276 - rollout_corr/rollout_is_mean:0.99993497133255 - rollout_corr/rollout_is_ratio_fraction_high:2.3056083591654897e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.069629257079214e-05 - rollout_corr/rollout_is_max:2.423980474472046 - rollout_corr/rollout_is_min:0.10168886929750443 - rollout_corr/rollout_is_std:0.04620131477713585 - rollout_corr/rollout_is_eff_sample_size:0.9978697477052325 - rollout_corr/rollout_is_seq_mean:0.9999666213989258 - rollout_corr/rollout_is_seq_std:0.00307915429584682 - rollout_corr/rollout_is_seq_max:1.0090153217315674 - rollout_corr/rollout_is_seq_min:0.9888045191764832 - rollout_corr/rollout_is_seq_max_deviation:0.011195480823516846 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3847207999788225) - rollout_corr/training_log_ppl:np.float64(0.3209572141058743) - rollout_corr/kl:np.float64(0.0013557246338891105) - rollout_corr/k3_kl:np.float64(0.0014090067750203161) - rollout_corr/rollout_ppl:np.float64(1.3828391721472144) - rollout_corr/rollout_log_ppl:np.float64(0.31960149001679383) - rollout_corr/log_ppl_diff:np.float64(0.0013557240890804678) - rollout_corr/log_ppl_abs_diff:np.float64(0.00264196444186382) - rollout_corr/log_ppl_diff_max:np.float64(0.03128330409526825) - rollout_corr/log_ppl_diff_min:np.float64(-0.007604241371154785) - rollout_corr/ppl_ratio:np.float64(1.0013634769711643) - rollout_corr/chi2_token:np.float64(0.0027722581289708614) - rollout_corr/chi2_seq:np.float64(2.2011893370654434) - actor/pg_loss:np.float64(0.00014042237307876348) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011157185435877182) - actor/ppo_kl:np.float64(0.00017038123593238197) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.623115174472332) - perf/mfu/actor:np.float64(0.05921451405552741) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.53786849975586) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:0 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005212116055190563 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005212116055190563 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:338.84765625 - response_length/max:803.0 - response_length/min:130.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:338.84765625 - response_length_non_aborted/max:803.0 - response_length_non_aborted/min:130.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.78125 - prompt_length/max:621.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.873611032962799e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.532330960035324) - timing_s/agent_loop/generate_sequences/max:np.float64(6.295476105064154) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.596290582929214) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.295476105064154) - timing_s/ag
(TaskRunner pid=2001144) Training Progress: 34%|███▍ | 34/100 [23:53<38:24, 34.92s/it]
(TaskRunner pid=2001144) step:35 - global_seqlen/min:15468 - global_seqlen/max:23627 - global_seqlen/minmax_diff:8159 - global_seqlen/balanced_min:18998 - global_seqlen/balanced_max:19005 - global_seqlen/mean:19003.125 - actor/entropy:0.33289381861686707 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23161432147026062 - training/rollout_probs_diff_mean:0.006583846639841795 - training/rollout_probs_diff_std:0.014489260502159595 - training/rollout_actor_probs_pearson_corr:0.9983018040657043 - rollout_corr/rollout_is_mean:1.0002074241638184 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.712457692017779e-05 - rollout_corr/rollout_is_max:1.8174363374710083 - rollout_corr/rollout_is_min:0.13879214227199554 - rollout_corr/rollout_is_std:0.04523788020014763 - rollout_corr/rollout_is_eff_sample_size:0.9979585446564787 - rollout_corr/rollout_is_seq_mean:1.000243067741394 - rollout_corr/rollout_is_seq_std:0.0029132540803402662 - rollout_corr/rollout_is_seq_max:1.0096945762634277 - rollout_corr/rollout_is_seq_min:0.9907513856887817 - rollout_corr/rollout_is_seq_max_deviation:0.009694576263427734 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4022721350193024) - rollout_corr/training_log_ppl:np.float64(0.335028117231559) - rollout_corr/kl:np.float64(0.001010199507428311) - rollout_corr/k3_kl:np.float64(0.0011698875011916243) - rollout_corr/rollout_ppl:np.float64(1.4008438815362751) - rollout_corr/rollout_log_ppl:np.float64(0.33401791501091793) - rollout_corr/log_ppl_diff:np.float64(0.0010102022206410766) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025172769092023373) - rollout_corr/log_ppl_diff_max:np.float64(0.010893464088439941) - rollout_corr/log_ppl_diff_min:np.float64(-0.00689244270324707) - rollout_corr/ppl_ratio:np.float64(1.001015585847199) - rollout_corr/chi2_token:np.float64(0.002676799427717924) - rollout_corr/chi2_seq:np.float64(1.0432001661974937) - actor/pg_loss:np.float64(0.0002795668551698327) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005276161855363171) - actor/ppo_kl:np.float64(0.000165828399765644) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.38394179195165634) - perf/mfu/actor:np.float64(0.054842311367362354) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.18587493896484) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:0 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001243673381395638 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001243673381395638 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:315.66015625 - response_length/max:989.0 - response_length/min:109.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:315.66015625 - response_length_non_aborted/max:989.0 - response_length_non_aborted/min:109.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.1875 - prompt_length/max:514.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.397573113441467e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2251826245337725) - timing_s/agent_loop/generate_sequences/max:np.float64(7.012961965054274) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.320388871623436) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.012961965054274) - timing_s/agent_loop/slowest/to
(TaskRunner pid=2001144) Training Progress: 35%|███▌ | 35/100 [24:18<34:48, 32.13s/it]
(TaskRunner pid=2001144) step:36 - global_seqlen/min:16777 - global_seqlen/max:23416 - global_seqlen/minmax_diff:6639 - global_seqlen/balanced_min:19749 - global_seqlen/balanced_max:19756 - global_seqlen/mean:19754.25 - actor/entropy:0.32015159726142883 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7186453938484192 - training/rollout_probs_diff_mean:0.006522524170577526 - training/rollout_probs_diff_std:0.015260634943842888 - training/rollout_actor_probs_pearson_corr:0.998120903968811 - rollout_corr/rollout_is_mean:0.9997749924659729 - rollout_corr/rollout_is_ratio_fraction_high:2.302396751474589e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010360785381635651 - rollout_corr/rollout_is_max:2.1660430431365967 - rollout_corr/rollout_is_min:0.015045300126075745 - rollout_corr/rollout_is_std:0.04637836664915085 - rollout_corr/rollout_is_eff_sample_size:0.9978528923146391 - rollout_corr/rollout_is_seq_mean:0.9998394250869751 - rollout_corr/rollout_is_seq_std:0.0028048930689692497 - rollout_corr/rollout_is_seq_max:1.0107539892196655 - rollout_corr/rollout_is_seq_min:0.9907167553901672 - rollout_corr/rollout_is_seq_max_deviation:0.010753989219665527 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4081328534521163) - rollout_corr/training_log_ppl:np.float64(0.33694178896257654) - rollout_corr/kl:np.float64(0.0013650812686842873) - rollout_corr/k3_kl:np.float64(0.0011794150487958177) - rollout_corr/rollout_ppl:np.float64(1.4062057179398835) - rollout_corr/rollout_log_ppl:np.float64(0.33557670534355566) - rollout_corr/log_ppl_diff:np.float64(0.0013650836190208793) - rollout_corr/log_ppl_abs_diff:np.float64(0.00262318248860538) - rollout_corr/log_ppl_diff_max:np.float64(0.010683506727218628) - rollout_corr/log_ppl_diff_min:np.float64(-0.009649574756622314) - rollout_corr/ppl_ratio:np.float64(1.0013708726037294) - rollout_corr/chi2_token:np.float64(0.0018969529774039984) - rollout_corr/chi2_seq:np.float64(2.313177646836266) - actor/pg_loss:np.float64(1.6184523701667786e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009666064806879149) - actor/ppo_kl:np.float64(6.0696681973393396e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.573500245809555) - perf/mfu/actor:np.float64(0.05745518180810477) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.1072769165039) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:0 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008118826895952225 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008118826895952225 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:339.3203125 - response_length/max:857.0 - response_length/min:121.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:339.3203125 - response_length_non_aborted/max:857.0 - response_length_non_aborted/min:121.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.0 - prompt_length/max:704.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011991336941719055 - timing_s/agent_loop/generate_sequences/min:np.float64(1.26837108284235) - timing_s/agent_loop/generate_sequences/max:np.float64(6.509035259485245) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.593755562978913) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.509035259485245) - timing_s/agent_l
(TaskRunner pid=2001144) Training Progress: 36%|███▌ | 36/100 [24:43<31:59, 29.98s/it]
(TaskRunner pid=2001144) step:37 - global_seqlen/min:17579 - global_seqlen/max:23430 - global_seqlen/minmax_diff:5851 - global_seqlen/balanced_min:20428 - global_seqlen/balanced_max:20430 - global_seqlen/mean:20429.25 - actor/entropy:0.323385626077652 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2419598400592804 - training/rollout_probs_diff_mean:0.006533097941428423 - training/rollout_probs_diff_std:0.014553725719451904 - training/rollout_actor_probs_pearson_corr:0.9983060956001282 - rollout_corr/rollout_is_mean:0.9997928142547607 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9741640090942383 - rollout_corr/rollout_is_min:0.510465145111084 - rollout_corr/rollout_is_std:0.04599248990416527 - rollout_corr/rollout_is_eff_sample_size:0.9978882655513207 - rollout_corr/rollout_is_seq_mean:0.9996730089187622 - rollout_corr/rollout_is_seq_std:0.0025787404738366604 - rollout_corr/rollout_is_seq_max:1.0064846277236938 - rollout_corr/rollout_is_seq_min:0.9929295182228088 - rollout_corr/rollout_is_seq_max_deviation:0.007070481777191162 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4111237754113972) - rollout_corr/training_log_ppl:np.float64(0.34012489904125687) - rollout_corr/kl:np.float64(0.0017189475028942525) - rollout_corr/k3_kl:np.float64(0.00139507363201119) - rollout_corr/rollout_ppl:np.float64(1.4085904457606375) - rollout_corr/rollout_log_ppl:np.float64(0.3384059520321898) - rollout_corr/log_ppl_diff:np.float64(0.0017189470090670511) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027223139331908897) - rollout_corr/log_ppl_diff_max:np.float64(0.01680201292037964) - rollout_corr/log_ppl_diff_min:np.float64(-0.007197782397270203) - rollout_corr/ppl_ratio:np.float64(1.0017253935802728) - rollout_corr/chi2_token:np.float64(0.002061627572402358) - rollout_corr/chi2_seq:np.float64(3.732431426178664) - actor/pg_loss:np.float64(0.0003973828861489892) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014174879970596521) - actor/ppo_kl:np.float64(0.0003027047177930342) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5962310880422592) - perf/mfu/actor:np.float64(0.05952064011776796) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.058349609375) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:0 - critic/score/mean:0.51953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.000493330298922956 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.000493330298922956 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:325.6328125 - response_length/max:682.0 - response_length/min:135.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:325.6328125 - response_length_non_aborted/max:682.0 - response_length_non_aborted/min:135.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.78125 - prompt_length/max:1009.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.968846380710602e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6425558254122734) - timing_s/agent_loop/generate_sequences/max:np.float64(5.776010008528829) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5510547384037636) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.776010008528829) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0)
(TaskRunner pid=2001144) Training Progress: 37%|███▋ | 37/100 [25:07<29:36, 28.19s/it]
(TaskRunner pid=2001144) step:38 - global_seqlen/min:16576 - global_seqlen/max:22749 - global_seqlen/minmax_diff:6173 - global_seqlen/balanced_min:19218 - global_seqlen/balanced_max:19221 - global_seqlen/mean:19220.375 - actor/entropy:0.3162509799003601 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30576980113983154 - training/rollout_probs_diff_mean:0.006216809619218111 - training/rollout_probs_diff_std:0.014158033765852451 - training/rollout_actor_probs_pearson_corr:0.9983813762664795 - rollout_corr/rollout_is_mean:0.9998518824577332 - rollout_corr/rollout_is_ratio_fraction_high:1.1574744348763488e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.787371992482804e-05 - rollout_corr/rollout_is_max:2.142380475997925 - rollout_corr/rollout_is_min:0.3732762932777405 - rollout_corr/rollout_is_std:0.04509535804390907 - rollout_corr/rollout_is_eff_sample_size:0.9979700609267244 - rollout_corr/rollout_is_seq_mean:0.9999082088470459 - rollout_corr/rollout_is_seq_std:0.0026349870022386312 - rollout_corr/rollout_is_seq_max:1.0095975399017334 - rollout_corr/rollout_is_seq_min:0.9931058883666992 - rollout_corr/rollout_is_seq_max_deviation:0.009597539901733398 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3985088728368282) - rollout_corr/training_log_ppl:np.float64(0.33193886291701347) - rollout_corr/kl:np.float64(0.001627307975095249) - rollout_corr/k3_kl:np.float64(0.0014793925809044595) - rollout_corr/rollout_ppl:np.float64(1.3961667022667825) - rollout_corr/rollout_log_ppl:np.float64(0.3303115534072276) - rollout_corr/log_ppl_diff:np.float64(0.0016273095097858459) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029907657008152455) - rollout_corr/log_ppl_diff_max:np.float64(0.020298629999160767) - rollout_corr/log_ppl_diff_min:np.float64(-0.009222477674484253) - rollout_corr/ppl_ratio:np.float64(1.0016354664694518) - rollout_corr/chi2_token:np.float64(0.0028146093245595694) - rollout_corr/chi2_seq:np.float64(1.3283501688856632) - actor/pg_loss:np.float64(-0.00011650926899164915) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016769834787737636) - actor/ppo_kl:np.float64(0.0005056045590450253) - actor/pg_clipfrac_lower:np.float64(5.417822649178561e-06) - actor/grad_norm:np.float64(0.5272819548845291) - perf/mfu/actor:np.float64(0.0566838209250089) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.00714111328125) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:0 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001954684965312481 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001954684965312481 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:337.48046875 - response_length/max:764.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:337.48046875 - response_length_non_aborted/max:764.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.15625 - prompt_length/max:500.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001522395759820938 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6667475122958422) - timing_s/agent_loop/generate_sequences/max:np.float64(6.286012953147292) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.727812623757927) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.286012953
(TaskRunner pid=2001144) Training Progress: 38%|███▊ | 38/100 [25:32<28:00, 27.11s/it]
(TaskRunner pid=2001144) step:39 - global_seqlen/min:17058 - global_seqlen/max:23042 - global_seqlen/minmax_diff:5984 - global_seqlen/balanced_min:19504 - global_seqlen/balanced_max:19507 - global_seqlen/mean:19505.625 - actor/entropy:0.35913288593292236 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37773531675338745 - training/rollout_probs_diff_mean:0.006785455625504255 - training/rollout_probs_diff_std:0.0144418403506279 - training/rollout_actor_probs_pearson_corr:0.9984265565872192 - rollout_corr/rollout_is_mean:0.999972403049469 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.548704110784456e-05 - rollout_corr/rollout_is_max:1.9570597410202026 - rollout_corr/rollout_is_min:0.36442822217941284 - rollout_corr/rollout_is_std:0.046883899718523026 - rollout_corr/rollout_is_eff_sample_size:0.9978066022699049 - rollout_corr/rollout_is_seq_mean:1.0000252723693848 - rollout_corr/rollout_is_seq_std:0.0027740709483623505 - rollout_corr/rollout_is_seq_max:1.0088070631027222 - rollout_corr/rollout_is_seq_min:0.9929769039154053 - rollout_corr/rollout_is_seq_max_deviation:0.008807063102722168 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4513516179285944) - rollout_corr/training_log_ppl:np.float64(0.36863006779458374) - rollout_corr/kl:np.float64(0.0014226692903083205) - rollout_corr/k3_kl:np.float64(0.0013480821364737494) - rollout_corr/rollout_ppl:np.float64(1.4492747029289603) - rollout_corr/rollout_log_ppl:np.float64(0.36720739799784496) - rollout_corr/log_ppl_diff:np.float64(0.0014226697967387736) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028523848741315305) - rollout_corr/log_ppl_diff_max:np.float64(0.012905359268188477) - rollout_corr/log_ppl_diff_min:np.float64(-0.007768481969833374) - rollout_corr/ppl_ratio:np.float64(1.0014290269464254) - rollout_corr/chi2_token:np.float64(0.0026027336716651917) - rollout_corr/chi2_seq:np.float64(0.9936506301164627) - actor/pg_loss:np.float64(4.365795757621527e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010918598350144748) - actor/ppo_kl:np.float64(0.0003035598902911829) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.514805905520916) - perf/mfu/actor:np.float64(0.0572909617646197) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.0003890991211) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:0 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0025333906523883343 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0025333906523883343 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:327.26953125 - response_length/max:732.0 - response_length/min:139.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:327.26953125 - response_length_non_aborted/max:732.0 - response_length_non_aborted/min:139.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:282.28125 - prompt_length/max:452.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.573755621910095e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7248130720108747) - timing_s/agent_loop/generate_sequences/max:np.float64(6.088032918050885) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.640068974804308) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.088032918050885) - timing_s/agent_loop/slowes
(TaskRunner pid=2001144) Training Progress: 39%|███▉ | 39/100 [25:57<26:45, 26.32s/it]
(TaskRunner pid=2001144) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40}
(TaskRunner pid=2001144) validation generation end
(TaskRunner pid=2001144) [prompt] system
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2001144) <reasoning>
(TaskRunner pid=2001144) ...
(TaskRunner pid=2001144) </reasoning>
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) ...
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) A
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) user
(TaskRunner pid=2001144) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2001144) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2001144) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2001144) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2001144) Please reason step by step.
(TaskRunner pid=2001144) assistant
(TaskRunner pid=2001144) <think>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) </think>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) [response] <reasoning>
(TaskRunner pid=2001144) The question asks which of the given molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". This product appears to be a complex organic molecule with a nitrogen-containing amine group and a substituted aromatic ring.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) To identify the correct reactant, we must analyze the structure of the product and compare it with the options provided. The key part of the product is the "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" fragment, which includes a nitrogen atom connected to a chain that leads to a substituted aromatic ring with a Cl atom and a trifluoromethyl group (C(F)(F)F).
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Looking at the options:
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) - **Option A**: Contains a structure similar to the product, with a substituted aromatic ring and a nitrogen-containing amine group. It appears to be a plausible reactant for the synthesis of the product.
(TaskRunner pid=2001144) - **Option B**: Contains a substituted aromatic ring with a Cl atom but lacks the specific nitrogen-containing amine group seen in the product.
(TaskRunner pid=2001144) - **Option C**: Contains a substituted aromatic ring and a nitrogen-containing group, but the structure does not match the specific substitution pattern of the product.
(TaskRunner pid=2001144) - **Option D**: Contains a substituted aromatic ring and a nitrogen-containing group, but the structure is not consistent with the product’s specific substitution pattern.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Option A is the only one that closely matches the structure of the product, suggesting it is the correct reactant used in its synthesis.
(TaskRunner pid=2001144) </reasoning>
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) A
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144) [ground_truth] A
(TaskRunner pid=2001144) [score] 1.0
(TaskRunner pid=2001144) [acc] 1.0
(TaskRunner pid=2001144) [pred] A
(TaskRunner pid=2001144) [incorrect_format] 1
(TaskRunner pid=2001144) [feedback]
(TaskRunner pid=2001144) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40
(WorkerDict pid=2001534) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30/actor
(TaskRunner pid=2001144) step:40 - global_seqlen/min:18357 - global_seqlen/max:25913 - global_seqlen/minmax_diff:7556 - global_seqlen/balanced_min:21444 - global_seqlen/balanced_max:21452 - global_seqlen/mean:21449.75 - actor/entropy:0.36691588163375854 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7543222904205322 - training/rollout_probs_diff_mean:0.006701694335788488 - training/rollout_probs_diff_std:0.01469995453953743 - training/rollout_actor_probs_pearson_corr:0.998392641544342 - rollout_corr/rollout_is_mean:1.0000933408737183 - rollout_corr/rollout_is_ratio_fraction_high:5.4614964028587565e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.830693306867033e-05 - rollout_corr/rollout_is_max:7.368729591369629 - rollout_corr/rollout_is_min:0.2279360592365265 - rollout_corr/rollout_is_std:0.047546662390232086 - rollout_corr/rollout_is_eff_sample_size:0.9977445328740013 - rollout_corr/rollout_is_seq_mean:1.0001224279403687 - rollout_corr/rollout_is_seq_std:0.002747190184891224 - rollout_corr/rollout_is_seq_max:1.0098285675048828 - rollout_corr/rollout_is_seq_min:0.9924802184104919 - rollout_corr/rollout_is_seq_max_deviation:0.009828567504882812 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4537925370968878) - rollout_corr/training_log_ppl:np.float64(0.3690156069933437) - rollout_corr/kl:np.float64(0.0011667901635803446) - rollout_corr/k3_kl:np.float64(0.0012600065753076706) - rollout_corr/rollout_ppl:np.float64(1.4520994019694626) - rollout_corr/rollout_log_ppl:np.float64(0.3678488138830289) - rollout_corr/log_ppl_diff:np.float64(0.0011667931103147566) - rollout_corr/log_ppl_abs_diff:np.float64(0.002250000194180757) - rollout_corr/log_ppl_diff_max:np.float64(0.010493844747543335) - rollout_corr/log_ppl_diff_min:np.float64(-0.006001293659210205) - rollout_corr/ppl_ratio:np.float64(1.0011710203252733) - rollout_corr/chi2_token:np.float64(0.00277118943631649) - rollout_corr/chi2_seq:np.float64(0.6102668882813305) - actor/pg_loss:np.float64(-0.00010784633923321962) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007779959523759317) - actor/ppo_kl:np.float64(0.00011245050796127032) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3933354467153549) - perf/mfu/actor:np.float64(0.06287996526335489) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.12603759765625) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.5904761904761905) - val-aux/sciknoweval/reward/std@16:np.float64(0.19795952330643343) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6660952380952381) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15030138901877363) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5141476190476191) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.18168743820124414) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5898476190476191) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.19833727084411176) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7213047619047619) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.10709817732551635) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4436809523809524) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.15650683760274148) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6028428571428572) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.14765493559833423) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7628285714285715) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07630921986167764) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.3807238095238095) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.12851789522357487) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6069904761904762) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09995018361763987) - val-aux/sciknoweval/reward/best@1
(TaskRunner pid=2001144) Training Progress: 40%|████ | 40/100 [28:29<1:04:04, 64.07s/it]
(TaskRunner pid=2001144) step:41 - global_seqlen/min:15969 - global_seqlen/max:24730 - global_seqlen/minmax_diff:8761 - global_seqlen/balanced_min:19293 - global_seqlen/balanced_max:19297 - global_seqlen/mean:19295.5 - actor/entropy:0.3754251301288605 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34931766986846924 - training/rollout_probs_diff_mean:0.006441387813538313 - training/rollout_probs_diff_std:0.013664642348885536 - training/rollout_actor_probs_pearson_corr:0.9986243844032288 - rollout_corr/rollout_is_mean:0.9999328851699829 - rollout_corr/rollout_is_ratio_fraction_high:1.1520206498971675e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.4560620406409726e-05 - rollout_corr/rollout_is_max:2.010493040084839 - rollout_corr/rollout_is_min:0.3993358314037323 - rollout_corr/rollout_is_std:0.0444013886153698 - rollout_corr/rollout_is_eff_sample_size:0.9980321583867411 - rollout_corr/rollout_is_seq_mean:0.9998860359191895 - rollout_corr/rollout_is_seq_std:0.002443247474730015 - rollout_corr/rollout_is_seq_max:1.0063079595565796 - rollout_corr/rollout_is_seq_min:0.9928776621818542 - rollout_corr/rollout_is_seq_max_deviation:0.007122337818145752 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4713093480095267) - rollout_corr/training_log_ppl:np.float64(0.38211210755980574) - rollout_corr/kl:np.float64(0.0013293944941876745) - rollout_corr/k3_kl:np.float64(0.0011158423294546083) - rollout_corr/rollout_ppl:np.float64(1.4693495454266667) - rollout_corr/rollout_log_ppl:np.float64(0.3807827102718875) - rollout_corr/log_ppl_diff:np.float64(0.0013293972879182547) - rollout_corr/log_ppl_abs_diff:np.float64(0.002292817604029551) - rollout_corr/log_ppl_diff_max:np.float64(0.009773343801498413) - rollout_corr/log_ppl_diff_min:np.float64(-0.005606532096862793) - rollout_corr/ppl_ratio:np.float64(1.0013339414726943) - rollout_corr/chi2_token:np.float64(0.0018368919845670462) - rollout_corr/chi2_seq:np.float64(0.563911790959537) - actor/pg_loss:np.float64(0.00022271834313869476) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008147563394231838) - actor/ppo_kl:np.float64(0.0001877853810796637) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4912325441837311) - perf/mfu/actor:np.float64(0.05671217241513034) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(104.39080047607422) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:0 - critic/score/mean:0.51171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0012629026314243674 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0012629026314243674 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:339.078125 - response_length/max:810.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:339.078125 - response_length_non_aborted/max:810.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.90625 - prompt_length/max:631.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011256709694862366 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2559579368680716) - timing_s/agent_loop/generate_sequences/max:np.float64(6.301052588969469) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.528378706148942) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.301052588969469) - timing_s/ag
(TaskRunner pid=2001144) Training Progress: 41%|████ | 41/100 [28:53<51:18, 52.18s/it]
(TaskRunner pid=2001144) step:42 - global_seqlen/min:16465 - global_seqlen/max:21715 - global_seqlen/minmax_diff:5250 - global_seqlen/balanced_min:18849 - global_seqlen/balanced_max:18852 - global_seqlen/mean:18851.25 - actor/entropy:0.38540732860565186 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2250097692012787 - training/rollout_probs_diff_mean:0.00656507071107626 - training/rollout_probs_diff_std:0.013662499375641346 - training/rollout_actor_probs_pearson_corr:0.9986229538917542 - rollout_corr/rollout_is_mean:0.9999583959579468 - rollout_corr/rollout_is_ratio_fraction_high:2.3497896108892746e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1748948054446373e-05 - rollout_corr/rollout_is_max:3.1482057571411133 - rollout_corr/rollout_is_min:0.39518943428993225 - rollout_corr/rollout_is_std:0.04514027386903763 - rollout_corr/rollout_is_eff_sample_size:0.9979661429905217 - rollout_corr/rollout_is_seq_mean:0.9999608993530273 - rollout_corr/rollout_is_seq_std:0.002602730877697468 - rollout_corr/rollout_is_seq_max:1.0108071565628052 - rollout_corr/rollout_is_seq_min:0.9921960234642029 - rollout_corr/rollout_is_seq_max_deviation:0.010807156562805176 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4853256773203611) - rollout_corr/training_log_ppl:np.float64(0.3918430757476017) - rollout_corr/kl:np.float64(0.001233867842810099) - rollout_corr/k3_kl:np.float64(0.0011519862129034664) - rollout_corr/rollout_ppl:np.float64(1.4834744110703468) - rollout_corr/rollout_log_ppl:np.float64(0.3906092082615942) - rollout_corr/log_ppl_diff:np.float64(0.0012338674860075116) - rollout_corr/log_ppl_abs_diff:np.float64(0.002302582608535886) - rollout_corr/log_ppl_diff_max:np.float64(0.009481191635131836) - rollout_corr/log_ppl_diff_min:np.float64(-0.00770801305770874) - rollout_corr/ppl_ratio:np.float64(1.0012381442356855) - rollout_corr/chi2_token:np.float64(0.0022277936805039644) - rollout_corr/chi2_seq:np.float64(0.4249819794204086) - actor/pg_loss:np.float64(5.767587572336197e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008674365590195521) - actor/ppo_kl:np.float64(0.00015044993621304847) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4961194396018982) - perf/mfu/actor:np.float64(0.054950129113595265) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(104.4871826171875) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:0 - critic/score/mean:0.55078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006257783621549606 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006257783621549606 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:332.4765625 - response_length/max:680.0 - response_length/min:168.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:332.4765625 - response_length_non_aborted/max:680.0 - response_length_non_aborted/min:168.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:256.625 - prompt_length/max:557.0 - prompt_length/min:175.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012256577610969543 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6677542012184858) - timing_s/agent_loop/generate_sequences/max:np.float64(5.630872735753655) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4855631124301) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.630872735753655) - timing_s/agent_l
(TaskRunner pid=2001144) Training Progress: 42%|████▏ | 42/100 [29:18<42:27, 43.93s/it]
(TaskRunner pid=2001144) step:43 - global_seqlen/min:17959 - global_seqlen/max:21835 - global_seqlen/minmax_diff:3876 - global_seqlen/balanced_min:20578 - global_seqlen/balanced_max:20581 - global_seqlen/mean:20579.0 - actor/entropy:0.37470173835754395 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4484623968601227 - training/rollout_probs_diff_mean:0.006568420212715864 - training/rollout_probs_diff_std:0.014049583114683628 - training/rollout_actor_probs_pearson_corr:0.9985491037368774 - rollout_corr/rollout_is_mean:1.0000258684158325 - rollout_corr/rollout_is_ratio_fraction_high:3.4451080864528194e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.890216172905639e-05 - rollout_corr/rollout_is_max:2.314483404159546 - rollout_corr/rollout_is_min:0.26832666993141174 - rollout_corr/rollout_is_std:0.0463159941136837 - rollout_corr/rollout_is_eff_sample_size:0.9978595394429283 - rollout_corr/rollout_is_seq_mean:1.0000109672546387 - rollout_corr/rollout_is_seq_std:0.002857532352209091 - rollout_corr/rollout_is_seq_max:1.0081138610839844 - rollout_corr/rollout_is_seq_min:0.9908040761947632 - rollout_corr/rollout_is_seq_max_deviation:0.009195923805236816 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4757732609286904) - rollout_corr/training_log_ppl:np.float64(0.3853875285713002) - rollout_corr/kl:np.float64(0.001096273211718568) - rollout_corr/k3_kl:np.float64(0.0011729911917655045) - rollout_corr/rollout_ppl:np.float64(1.474128465168178) - rollout_corr/rollout_log_ppl:np.float64(0.384291255439166) - rollout_corr/log_ppl_diff:np.float64(0.001096273132134229) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023764665820635855) - rollout_corr/log_ppl_diff_max:np.float64(0.010235875844955444) - rollout_corr/log_ppl_diff_min:np.float64(-0.007035762071609497) - rollout_corr/ppl_ratio:np.float64(1.0011009911540896) - rollout_corr/chi2_token:np.float64(0.0025233884807676077) - rollout_corr/chi2_seq:np.float64(1.3428373110946268) - actor/pg_loss:np.float64(2.866866998374462e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012606361269718036) - actor/ppo_kl:np.float64(5.878310900264694e-06) - actor/pg_clipfrac_lower:np.float64(7.526493391196709e-06) - actor/grad_norm:np.float64(0.565574087202549) - perf/mfu/actor:np.float64(0.06034293853825906) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.6466293334961) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0036647336091846228 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0036647336091846228 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:340.15625 - response_length/max:807.0 - response_length/min:176.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:340.15625 - response_length_non_aborted/max:807.0 - response_length_non_aborted/min:176.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.9375 - prompt_length/max:813.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013952329754829407 - timing_s/agent_loop/generate_sequences/min:np.float64(2.077566647902131) - timing_s/agent_loop/generate_sequences/max:np.float64(6.608034310862422) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7474852346786065) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.608034310862422) - timing_s
(TaskRunner pid=2001144) Training Progress: 43%|████▎ | 43/100 [29:43<36:19, 38.23s/it]
(TaskRunner pid=2001144) step:44 - global_seqlen/min:17634 - global_seqlen/max:22949 - global_seqlen/minmax_diff:5315 - global_seqlen/balanced_min:20660 - global_seqlen/balanced_max:20665 - global_seqlen/mean:20662.375 - actor/entropy:0.3575077950954437 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45896613597869873 - training/rollout_probs_diff_mean:0.006433025002479553 - training/rollout_probs_diff_std:0.014396881684660912 - training/rollout_actor_probs_pearson_corr:0.9984731078147888 - rollout_corr/rollout_is_mean:1.0000427961349487 - rollout_corr/rollout_is_ratio_fraction_high:2.3381147912004963e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.183401951100677e-05 - rollout_corr/rollout_is_max:2.6709539890289307 - rollout_corr/rollout_is_min:0.3465282917022705 - rollout_corr/rollout_is_std:0.04617937654256821 - rollout_corr/rollout_is_eff_sample_size:0.9978722404521742 - rollout_corr/rollout_is_seq_mean:1.0000354051589966 - rollout_corr/rollout_is_seq_std:0.002842468209564686 - rollout_corr/rollout_is_seq_max:1.009746789932251 - rollout_corr/rollout_is_seq_min:0.9906759858131409 - rollout_corr/rollout_is_seq_max_deviation:0.009746789932250977 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4621909004636109) - rollout_corr/training_log_ppl:np.float64(0.3740108349593356) - rollout_corr/kl:np.float64(0.001342323942843393) - rollout_corr/k3_kl:np.float64(0.0012771108174547408) - rollout_corr/rollout_ppl:np.float64(1.4602555502206087) - rollout_corr/rollout_log_ppl:np.float64(0.37266850771266036) - rollout_corr/log_ppl_diff:np.float64(0.0013423272466752678) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024590483226347715) - rollout_corr/log_ppl_diff_max:np.float64(0.018486380577087402) - rollout_corr/log_ppl_diff_min:np.float64(-0.0063066184520721436) - rollout_corr/ppl_ratio:np.float64(1.0013479779008776) - rollout_corr/chi2_token:np.float64(0.0031542330980300903) - rollout_corr/chi2_seq:np.float64(2.244738094508648) - actor/pg_loss:np.float64(-1.5221070498228073e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009275444904233154) - actor/ppo_kl:np.float64(0.00027905690063789734) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.43255293369293213) - perf/mfu/actor:np.float64(0.06051050539956501) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.56446838378906) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:0 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001116449828259647 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001116449828259647 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:334.13671875 - response_length/max:715.0 - response_length/min:113.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:334.13671875 - response_length_non_aborted/max:715.0 - response_length_non_aborted/min:113.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:311.5625 - prompt_length/max:701.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.749457240104675e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.400841573253274) - timing_s/agent_loop/generate_sequences/max:np.float64(5.953770931810141) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7217183018801734) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.953770931810141) - timing_
(TaskRunner pid=2001144) Training Progress: 44%|████▍ | 44/100 [30:07<31:45, 34.02s/it]
(TaskRunner pid=2001144) step:45 - global_seqlen/min:16640 - global_seqlen/max:26689 - global_seqlen/minmax_diff:10049 - global_seqlen/balanced_min:20052 - global_seqlen/balanced_max:20053 - global_seqlen/mean:20052.25 - actor/entropy:0.38703450560569763 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6695032119750977 - training/rollout_probs_diff_mean:0.006644253619015217 - training/rollout_probs_diff_std:0.013965962454676628 - training/rollout_actor_probs_pearson_corr:0.9986119866371155 - rollout_corr/rollout_is_mean:1.000144362449646 - rollout_corr/rollout_is_ratio_fraction_high:2.5044453650480136e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.5044453650480136e-05 - rollout_corr/rollout_is_max:2.1456515789031982 - rollout_corr/rollout_is_min:0.21412421762943268 - rollout_corr/rollout_is_std:0.04527870938181877 - rollout_corr/rollout_is_eff_sample_size:0.9979546268106989 - rollout_corr/rollout_is_seq_mean:1.0002480745315552 - rollout_corr/rollout_is_seq_std:0.0025209328159689903 - rollout_corr/rollout_is_seq_max:1.0071067810058594 - rollout_corr/rollout_is_seq_min:0.9886420369148254 - rollout_corr/rollout_is_seq_max_deviation:0.01135796308517456 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5001712143421173) - rollout_corr/training_log_ppl:np.float64(0.4011685152654536) - rollout_corr/kl:np.float64(0.0009597091718021034) - rollout_corr/k3_kl:np.float64(0.0012612349983101012) - rollout_corr/rollout_ppl:np.float64(1.4987679654732347) - rollout_corr/rollout_log_ppl:np.float64(0.40020880443626083) - rollout_corr/log_ppl_diff:np.float64(0.0009597108291927725) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022613337787333876) - rollout_corr/log_ppl_diff_max:np.float64(0.010978728532791138) - rollout_corr/log_ppl_diff_min:np.float64(-0.00660020112991333) - rollout_corr/ppl_ratio:np.float64(1.0009640278294683) - rollout_corr/chi2_token:np.float64(0.003196456702426076) - rollout_corr/chi2_seq:np.float64(0.65230951923877) - actor/pg_loss:np.float64(0.0002657952718436718) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008488199023304333) - actor/ppo_kl:np.float64(0.0001463246287869424) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.45392607897520065) - perf/mfu/actor:np.float64(0.05854939903477489) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.7224555015564) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:0 - critic/score/mean:0.51171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00018000701675191522 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00018000701675191522 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:311.9453125 - response_length/max:729.0 - response_length/min:128.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:311.9453125 - response_length_non_aborted/max:729.0 - response_length_non_aborted/min:128.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.6875 - prompt_length/max:845.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.671195685863495e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.561294125393033) - timing_s/agent_loop/generate_sequences/max:np.float64(5.7592792473733425) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3728211936031585) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.7592792473733425) - timing_s
(TaskRunner pid=2001144) Training Progress: 45%|████▌ | 45/100 [30:31<28:28, 31.06s/it]
(TaskRunner pid=2001144) step:46 - global_seqlen/min:18150 - global_seqlen/max:24244 - global_seqlen/minmax_diff:6094 - global_seqlen/balanced_min:20401 - global_seqlen/balanced_max:20410 - global_seqlen/mean:20408.25 - actor/entropy:0.3904961347579956 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3519294857978821 - training/rollout_probs_diff_mean:0.006156236864626408 - training/rollout_probs_diff_std:0.013327605091035366 - training/rollout_actor_probs_pearson_corr:0.9987360239028931 - rollout_corr/rollout_is_mean:0.9999489188194275 - rollout_corr/rollout_is_ratio_fraction_high:3.232131712138653e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.3772780895233154 - rollout_corr/rollout_is_min:0.52516770362854 - rollout_corr/rollout_is_std:0.04307681322097778 - rollout_corr/rollout_is_eff_sample_size:0.9981475875802697 - rollout_corr/rollout_is_seq_mean:0.9998537302017212 - rollout_corr/rollout_is_seq_std:0.0025020919274538755 - rollout_corr/rollout_is_seq_max:1.0079485177993774 - rollout_corr/rollout_is_seq_min:0.9921724200248718 - rollout_corr/rollout_is_seq_max_deviation:0.007948517799377441 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5059236348606646) - rollout_corr/training_log_ppl:np.float64(0.4035984173824545) - rollout_corr/kl:np.float64(0.0010965432525402719) - rollout_corr/k3_kl:np.float64(0.001111666099291142) - rollout_corr/rollout_ppl:np.float64(1.504194260109216) - rollout_corr/rollout_log_ppl:np.float64(0.40250186767661944) - rollout_corr/log_ppl_diff:np.float64(0.0010965497058350593) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022097354230936617) - rollout_corr/log_ppl_diff_max:np.float64(0.011031657457351685) - rollout_corr/log_ppl_diff_min:np.float64(-0.008189737796783447) - rollout_corr/ppl_ratio:np.float64(1.0011010256130248) - rollout_corr/chi2_token:np.float64(0.0022728601470589638) - rollout_corr/chi2_seq:np.float64(0.7749577998183668) - actor/pg_loss:np.float64(8.76100966706872e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009620690407246002) - actor/ppo_kl:np.float64(-3.421309387618976e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5237475335597992) - perf/mfu/actor:np.float64(0.06036774860917735) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.52131652832031) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:0 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003886638442054391 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.003886638442054391 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:362.5703125 - response_length/max:1105.0 - response_length/min:152.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:362.5703125 - response_length_non_aborted/max:1105.0 - response_length_non_aborted/min:152.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:275.1875 - prompt_length/max:595.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.081560134887695e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8321135453879833) - timing_s/agent_loop/generate_sequences/max:np.float64(7.767448142170906) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8267360397512675) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.767448142170906) - timing_s/agent_loop/slowest/tool_cal
(TaskRunner pid=2001144) Training Progress: 46%|████▌ | 46/100 [30:57<26:35, 29.54s/it]
(TaskRunner pid=2001144) step:47 - global_seqlen/min:16656 - global_seqlen/max:23984 - global_seqlen/minmax_diff:7328 - global_seqlen/balanced_min:20970 - global_seqlen/balanced_max:20974 - global_seqlen/mean:20972.5 - actor/entropy:0.36923038959503174 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3894999623298645 - training/rollout_probs_diff_mean:0.006069595459848642 - training/rollout_probs_diff_std:0.013451742939651012 - training/rollout_actor_probs_pearson_corr:0.9986440539360046 - rollout_corr/rollout_is_mean:1.0000097751617432 - rollout_corr/rollout_is_ratio_fraction_high:3.236245902371593e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.1574973288807087e-05 - rollout_corr/rollout_is_max:2.383199691772461 - rollout_corr/rollout_is_min:0.45241281390190125 - rollout_corr/rollout_is_std:0.0431487038731575 - rollout_corr/rollout_is_eff_sample_size:0.9981416492131459 - rollout_corr/rollout_is_seq_mean:1.0000717639923096 - rollout_corr/rollout_is_seq_std:0.0024798517115414143 - rollout_corr/rollout_is_seq_max:1.0086157321929932 - rollout_corr/rollout_is_seq_min:0.9915279150009155 - rollout_corr/rollout_is_seq_max_deviation:0.008615732192993164 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4720067828893661) - rollout_corr/training_log_ppl:np.float64(0.3812735331885051) - rollout_corr/kl:np.float64(0.0009184170521407964) - rollout_corr/k3_kl:np.float64(0.001180113003556471) - rollout_corr/rollout_ppl:np.float64(1.470697249751538) - rollout_corr/rollout_log_ppl:np.float64(0.3803551164455712) - rollout_corr/log_ppl_diff:np.float64(0.0009184167429339141) - rollout_corr/log_ppl_abs_diff:np.float64(0.002264810638735071) - rollout_corr/log_ppl_diff_max:np.float64(0.009846776723861694) - rollout_corr/log_ppl_diff_min:np.float64(-0.00702279806137085) - rollout_corr/ppl_ratio:np.float64(1.0009226277470589) - rollout_corr/chi2_token:np.float64(0.002947910688817501) - rollout_corr/chi2_seq:np.float64(2.6617242116481066) - actor/pg_loss:np.float64(-0.0001755931880325079) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001428587483587762) - actor/ppo_kl:np.float64(2.0990544379984044e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5336242765188217) - perf/mfu/actor:np.float64(0.059977005395156865) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.58353567123413) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:8.495145448250696e-05 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:8.495145448250696e-05 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:362.109375 - response_length/max:967.0 - response_length/min:153.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:362.109375 - response_length_non_aborted/max:967.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.28125 - prompt_length/max:950.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010139308869838715 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8158737570047379) - timing_s/agent_loop/generate_sequences/max:np.float64(7.440636260434985) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8879454990528757) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.440636260434985) - timing_s/agent_loop/
(TaskRunner pid=2001144) Training Progress: 47%|████▋ | 47/100 [31:23<25:10, 28.49s/it]
(TaskRunner pid=2001144) step:48 - global_seqlen/min:17464 - global_seqlen/max:21338 - global_seqlen/minmax_diff:3874 - global_seqlen/balanced_min:19266 - global_seqlen/balanced_max:19276 - global_seqlen/mean:19272.875 - actor/entropy:0.40570738911628723 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3797455430030823 - training/rollout_probs_diff_mean:0.006236532237380743 - training/rollout_probs_diff_std:0.013078873977065086 - training/rollout_actor_probs_pearson_corr:0.9987834692001343 - rollout_corr/rollout_is_mean:0.999739408493042 - rollout_corr/rollout_is_ratio_fraction_high:1.160644842457259e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.4819346183212474e-05 - rollout_corr/rollout_is_max:2.3591055870056152 - rollout_corr/rollout_is_min:0.32449871301651 - rollout_corr/rollout_is_std:0.042700886726379395 - rollout_corr/rollout_is_eff_sample_size:0.9981789433302014 - rollout_corr/rollout_is_seq_mean:0.9996945858001709 - rollout_corr/rollout_is_seq_std:0.0025166745763272047 - rollout_corr/rollout_is_seq_max:1.0091334581375122 - rollout_corr/rollout_is_seq_min:0.9931390881538391 - rollout_corr/rollout_is_seq_max_deviation:0.009133458137512207 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5287531116046011) - rollout_corr/training_log_ppl:np.float64(0.4187461094115861) - rollout_corr/kl:np.float64(0.001219485045751867) - rollout_corr/k3_kl:np.float64(0.001174924948941225) - rollout_corr/rollout_ppl:np.float64(1.5268241628073156) - rollout_corr/rollout_log_ppl:np.float64(0.4175266235251911) - rollout_corr/log_ppl_diff:np.float64(0.0012194858863949776) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023000857327133417) - rollout_corr/log_ppl_diff_max:np.float64(0.008737266063690186) - rollout_corr/log_ppl_diff_min:np.float64(-0.007675737142562866) - rollout_corr/ppl_ratio:np.float64(1.0012238121125847) - rollout_corr/chi2_token:np.float64(0.0022212446201592684) - rollout_corr/chi2_seq:np.float64(0.5891820995602757) - actor/pg_loss:np.float64(7.152056787163019e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006652325555478455) - actor/ppo_kl:np.float64(-2.3931782806130286e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.44942254573106766) - perf/mfu/actor:np.float64(0.05618234392414996) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.56016159057617) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:0 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0059279934503138065 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0059279934503138065 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:336.55859375 - response_length/max:954.0 - response_length/min:171.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:336.55859375 - response_length_non_aborted/max:954.0 - response_length_non_aborted/min:171.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.71875 - prompt_length/max:563.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.973130464553833e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.061694385483861) - timing_s/agent_loop/generate_sequences/max:np.float64(7.1013307347893715) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7526944153141812) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.1013307347893715) - timin
(TaskRunner pid=2001144) Training Progress: 48%|████▊ | 48/100 [31:48<23:50, 27.50s/it]
(TaskRunner pid=2001144) step:49 - global_seqlen/min:18871 - global_seqlen/max:23201 - global_seqlen/minmax_diff:4330 - global_seqlen/balanced_min:21014 - global_seqlen/balanced_max:21017 - global_seqlen/mean:21016.0 - actor/entropy:0.40420278906822205 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3535176217556 - training/rollout_probs_diff_mean:0.006269898731261492 - training/rollout_probs_diff_std:0.013149457983672619 - training/rollout_actor_probs_pearson_corr:0.998780369758606 - rollout_corr/rollout_is_mean:0.999972403049469 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.23499251762405e-05 - rollout_corr/rollout_is_max:1.9171195030212402 - rollout_corr/rollout_is_min:0.22341948747634888 - rollout_corr/rollout_is_std:0.043610379099845886 - rollout_corr/rollout_is_eff_sample_size:0.9981016264617208 - rollout_corr/rollout_is_seq_mean:0.9999768137931824 - rollout_corr/rollout_is_seq_std:0.002442432101815939 - rollout_corr/rollout_is_seq_max:1.0092902183532715 - rollout_corr/rollout_is_seq_min:0.9924293160438538 - rollout_corr/rollout_is_seq_max_deviation:0.009290218353271484 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5228071282617748) - rollout_corr/training_log_ppl:np.float64(0.4144144859164953) - rollout_corr/kl:np.float64(0.0011527932316646172) - rollout_corr/k3_kl:np.float64(0.001146854107219042) - rollout_corr/rollout_ppl:np.float64(1.5210088770836592) - rollout_corr/rollout_log_ppl:np.float64(0.41326169128296897) - rollout_corr/log_ppl_diff:np.float64(0.001152794633526355) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023170363274402916) - rollout_corr/log_ppl_diff_max:np.float64(0.013294965028762817) - rollout_corr/log_ppl_diff_min:np.float64(-0.005968630313873291) - rollout_corr/ppl_ratio:np.float64(1.0011574917007238) - rollout_corr/chi2_token:np.float64(0.002203058684244752) - rollout_corr/chi2_seq:np.float64(0.9327588398009539) - actor/pg_loss:np.float64(-7.828709203749895e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006469293557529454) - actor/ppo_kl:np.float64(0.00012664338392553987) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.40669070184230804) - perf/mfu/actor:np.float64(0.06113867543598209) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.5828971862793) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:0 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005816417280584574 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005816417280584574 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:377.9375 - response_length/max:888.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:377.9375 - response_length_non_aborted/max:888.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.8125 - prompt_length/max:418.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.9501115679740906e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9344606157392263) - timing_s/agent_loop/generate_sequences/max:np.float64(7.022324288263917) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.149138148823113) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.022324288263917) - timing_s/agent_loop/slowest/tool_calls:np.float6
(TaskRunner pid=2001144) Training Progress: 49%|████▉ | 49/100 [32:14<22:51, 26.89s/it]
(TaskRunner pid=2001144) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50}
(TaskRunner pid=2001144) validation generation end
(TaskRunner pid=2001144) [prompt] system
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2001144) <reasoning>
(TaskRunner pid=2001144) ...
(TaskRunner pid=2001144) </reasoning>
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) ...
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) A
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) user
(TaskRunner pid=2001144) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2001144) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2001144) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2001144) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2001144) Please reason step by step.
(TaskRunner pid=2001144) assistant
(TaskRunner pid=2001144) <think>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) </think>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) [response] <reasoning>
(TaskRunner pid=2001144) The question asks which of the given molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". The product has a complex structure, and the task is to identify which of the four options is the correct reactant that would be involved in its synthesis.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Looking at the product, we can see that it contains a nitrogen atom connected to a carbon chain, and there is a "Cc2ccc(Cl)cc2C(F)(F)F" fragment, which suggests that the molecule includes a chloro-substituted aromatic ring and a trifluoromethyl group (C(F)(F)F).
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Now, let's analyze each option:
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Option A: "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"
(TaskRunner pid=2001144) This molecule includes a nitro group (C(=O)N), and the "Cc2ccc(Cl)cc2C(F)(F)F" fragment is present, which matches the fragment in the product. This suggests that this molecule could be a reactant in the synthesis of the product, as it contains the same key fragment.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Option B: "O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1"
(TaskRunner pid=2001144) This molecule includes a chloro group (C(Cl)) and a nitrogen connected to a carbon chain, but it does not contain the "Cc2ccc(Cl)cc2C(F)(F)F" fragment that is present in the product. Therefore, it is less likely to be the correct reactant.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Option C: "O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1"
(TaskRunner pid=2001144) This molecule has a more complex structure, including a trifluoromethyl group (C(F)(F)F) and a chloro-substituted aromatic ring, but the fragment "Cc2ccc(Cl)cc2C(F)(F)F" is not clearly present in the same way as in the product. It may be a possible reactant, but it is not as directly matching as Option A.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Option D: "O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F"
(TaskRunner pid=2001144) This molecule includes a trifluoromethyl group (C(F)(F)F), but it does not contain the "Cc2ccc(Cl)cc2" fragment that is present in the product. Therefore, it is not a match for the product.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Based on the comparison, Option A is the only one that contains the exact fragment "Cc2ccc(Cl)cc2C(F)(F)F" that is present in the product. This makes it the most likely correct reactant used in the synthesis of the product.
(TaskRunner pid=2001144) </reasoning>
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) A
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144) [ground_truth] A
(TaskRunner pid=2001144) [score] 1.0
(TaskRunner pid=2001144) [acc] 1.0
(TaskRunner pid=2001144) [pred] A
(TaskRunner pid=2001144) [incorrect_format] 1
(TaskRunner pid=2001144) [feedback]
(TaskRunner pid=2001144) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50
(WorkerDict pid=2001534) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40/actor
(TaskRunner pid=2001144) step:50 - global_seqlen/min:17778 - global_seqlen/max:28773 - global_seqlen/minmax_diff:10995 - global_seqlen/balanced_min:20669 - global_seqlen/balanced_max:20679 - global_seqlen/mean:20676.0 - actor/entropy:0.35181713104248047 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35835734009742737 - training/rollout_probs_diff_mean:0.006033692043274641 - training/rollout_probs_diff_std:0.013511843979358673 - training/rollout_actor_probs_pearson_corr:0.998656690120697 - rollout_corr/rollout_is_mean:1.000030279159546 - rollout_corr/rollout_is_ratio_fraction_high:1.066188997356221e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.198566992068663e-05 - rollout_corr/rollout_is_max:2.5543415546417236 - rollout_corr/rollout_is_min:0.4723665118217468 - rollout_corr/rollout_is_std:0.04342272877693176 - rollout_corr/rollout_is_eff_sample_size:0.9981182527343043 - rollout_corr/rollout_is_seq_mean:0.9999439716339111 - rollout_corr/rollout_is_seq_std:0.0025750212371349335 - rollout_corr/rollout_is_seq_max:1.0065737962722778 - rollout_corr/rollout_is_seq_min:0.9921970963478088 - rollout_corr/rollout_is_seq_max_deviation:0.007802903652191162 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4640723415650427) - rollout_corr/training_log_ppl:np.float64(0.37527703819796443) - rollout_corr/kl:np.float64(0.0012600775986335577) - rollout_corr/k3_kl:np.float64(0.0010788011197746528) - rollout_corr/rollout_ppl:np.float64(1.4621774558909237) - rollout_corr/rollout_log_ppl:np.float64(0.3740169608499855) - rollout_corr/log_ppl_diff:np.float64(0.0012600773479789495) - rollout_corr/log_ppl_abs_diff:np.float64(0.002310194307938218) - rollout_corr/log_ppl_diff_max:np.float64(0.01322859525680542) - rollout_corr/log_ppl_diff_min:np.float64(-0.005867868661880493) - rollout_corr/ppl_ratio:np.float64(1.0012646485120058) - rollout_corr/chi2_token:np.float64(0.0018092594109475613) - rollout_corr/chi2_seq:np.float64(0.8130898117087781) - actor/pg_loss:np.float64(-0.00020641821902245283) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010971540564241877) - actor/ppo_kl:np.float64(0.00022531882519416513) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5574217513203621) - perf/mfu/actor:np.float64(0.059797037035428754) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(102.56632232666016) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6318452380952381) - val-aux/sciknoweval/reward/std@16:np.float64(0.1557241230202734) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6962095238095238) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.12022533316099064) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5664904761904762) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14289551623537924) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6317761904761905) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.15515082964535465) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7428714285714285) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.07879244315583399) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.509104761904762) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11394243835423033) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6439714285714286) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1261712258962943) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7713857142857142) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.04578167060583907) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4646333333333333) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08017289098785915) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6503333333333333) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09438389149615142) - val-aux/sciknoweval/reward/best@
(TaskRunner pid=2001144) Training Progress: 50%|█████ | 50/100 [34:48<54:18, 65.17s/it]
(TaskRunner pid=2001144) step:51 - global_seqlen/min:19554 - global_seqlen/max:25588 - global_seqlen/minmax_diff:6034 - global_seqlen/balanced_min:22683 - global_seqlen/balanced_max:22687 - global_seqlen/mean:22685.5 - actor/entropy:0.39613524079322815 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26470232009887695 - training/rollout_probs_diff_mean:0.006312207784503698 - training/rollout_probs_diff_std:0.013187206350266933 - training/rollout_actor_probs_pearson_corr:0.9987953901290894 - rollout_corr/rollout_is_mean:0.9999272227287292 - rollout_corr/rollout_is_ratio_fraction_high:1.9709093976416625e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.912728011026047e-05 - rollout_corr/rollout_is_max:3.281769037246704 - rollout_corr/rollout_is_min:0.24726668000221252 - rollout_corr/rollout_is_std:0.044743724167346954 - rollout_corr/rollout_is_eff_sample_size:0.9980015242571268 - rollout_corr/rollout_is_seq_mean:0.999950647354126 - rollout_corr/rollout_is_seq_std:0.002747724298387766 - rollout_corr/rollout_is_seq_max:1.0084319114685059 - rollout_corr/rollout_is_seq_min:0.9901689887046814 - rollout_corr/rollout_is_seq_max_deviation:0.009831011295318604 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5107489707879722) - rollout_corr/training_log_ppl:np.float64(0.4079059749492444) - rollout_corr/kl:np.float64(0.0013574793727624268) - rollout_corr/k3_kl:np.float64(0.0012360485586100367) - rollout_corr/rollout_ppl:np.float64(1.508709030225873) - rollout_corr/rollout_log_ppl:np.float64(0.4065484952297993) - rollout_corr/log_ppl_diff:np.float64(0.0013574797194451094) - rollout_corr/log_ppl_abs_diff:np.float64(0.002422065823338926) - rollout_corr/log_ppl_diff_max:np.float64(0.00890931487083435) - rollout_corr/log_ppl_diff_min:np.float64(-0.009129047393798828) - rollout_corr/ppl_ratio:np.float64(1.0013624818529934) - rollout_corr/chi2_token:np.float64(0.0022446452639997005) - rollout_corr/chi2_seq:np.float64(2.3331969084683806) - actor/pg_loss:np.float64(-0.00018547044601291418) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008677035398250155) - actor/ppo_kl:np.float64(0.00026371053127505206) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5032129362225533) - perf/mfu/actor:np.float64(0.06645185189294886) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(104.37567520141602) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:0 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003904864192008972 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003904864192008972 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:396.390625 - response_length/max:961.0 - response_length/min:160.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:396.390625 - response_length_non_aborted/max:961.0 - response_length_non_aborted/min:160.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.53125 - prompt_length/max:608.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000105295330286026 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9396796356886625) - timing_s/agent_loop/generate_sequences/max:np.float64(7.769244348630309) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.257267202920048) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.769244348630309) - timing_s/agent_lo
(TaskRunner pid=2001144) Training Progress: 51%|█████ | 51/100 [35:14<43:38, 53.45s/it]
(TaskRunner pid=2001144) step:52 - global_seqlen/min:17382 - global_seqlen/max:26126 - global_seqlen/minmax_diff:8744 - global_seqlen/balanced_min:21386 - global_seqlen/balanced_max:21389 - global_seqlen/mean:21388.25 - actor/entropy:0.38237500190734863 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016907572746277 - training/rollout_probs_diff_mean:0.005891134031116962 - training/rollout_probs_diff_std:0.012641138397157192 - training/rollout_actor_probs_pearson_corr:0.9988550543785095 - rollout_corr/rollout_is_mean:0.9999381899833679 - rollout_corr/rollout_is_ratio_fraction_high:9.633725312596653e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.890117684728466e-05 - rollout_corr/rollout_is_max:3.098963737487793 - rollout_corr/rollout_is_min:0.36265504360198975 - rollout_corr/rollout_is_std:0.04137157276272774 - rollout_corr/rollout_is_eff_sample_size:0.998291080008873 - rollout_corr/rollout_is_seq_mean:0.9999604821205139 - rollout_corr/rollout_is_seq_std:0.0022436766885221004 - rollout_corr/rollout_is_seq_max:1.0077764987945557 - rollout_corr/rollout_is_seq_min:0.9923790097236633 - rollout_corr/rollout_is_seq_max_deviation:0.007776498794555664 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4827068336308002) - rollout_corr/training_log_ppl:np.float64(0.38936525472672656) - rollout_corr/kl:np.float64(0.0008875675469823818) - rollout_corr/k3_kl:np.float64(0.0010023709598385722) - rollout_corr/rollout_ppl:np.float64(1.4813770963810384) - rollout_corr/rollout_log_ppl:np.float64(0.38847768801497295) - rollout_corr/log_ppl_diff:np.float64(0.0008875667117536068) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018904238240793347) - rollout_corr/log_ppl_diff_max:np.float64(0.009192556142807007) - rollout_corr/log_ppl_diff_min:np.float64(-0.005056023597717285) - rollout_corr/ppl_ratio:np.float64(1.0008907769806683) - rollout_corr/chi2_token:np.float64(0.0022238660603761673) - rollout_corr/chi2_seq:np.float64(0.5789201743900776) - actor/pg_loss:np.float64(-2.1226005628705025e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005477525810420047) - actor/ppo_kl:np.float64(-3.959797676955645e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.43142857402563095) - perf/mfu/actor:np.float64(0.06288350474055739) - perf/max_memory_allocated_gb:np.float64(123.40249061584473) - perf/max_memory_reserved_gb:np.float64(129.16796875) - perf/cpu_memory_used_gb:np.float64(103.7133903503418) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:0 - critic/score/mean:0.55078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001442650449462235 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001442650449462235 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:405.4765625 - response_length/max:841.0 - response_length/min:183.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:405.4765625 - response_length_non_aborted/max:841.0 - response_length_non_aborted/min:183.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.90625 - prompt_length/max:453.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.047985076904297e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0897927284240723) - timing_s/agent_loop/generate_sequences/max:np.float64(6.885364580899477) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.258958232465375) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.885364580899477) - timing_s
(TaskRunner pid=2001144) Training Progress: 52%|█████▏ | 52/100 [35:40<36:05, 45.11s/it]
(TaskRunner pid=2001144) step:53 - global_seqlen/min:22092 - global_seqlen/max:31278 - global_seqlen/minmax_diff:9186 - global_seqlen/balanced_min:23544 - global_seqlen/balanced_max:30689 - global_seqlen/mean:24439.625 - actor/entropy:0.3605777323246002 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553979158401489 - training/rollout_probs_diff_mean:0.005666907411068678 - training/rollout_probs_diff_std:0.01265560183674097 - training/rollout_actor_probs_pearson_corr:0.9988373517990112 - rollout_corr/rollout_is_mean:1.0000286102294922 - rollout_corr/rollout_is_ratio_fraction_high:2.503066207282245e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.3374217309756204e-05 - rollout_corr/rollout_is_max:2.554331064224243 - rollout_corr/rollout_is_min:0.28212618827819824 - rollout_corr/rollout_is_std:0.04188132658600807 - rollout_corr/rollout_is_eff_sample_size:0.9982491445941432 - rollout_corr/rollout_is_seq_mean:1.0000684261322021 - rollout_corr/rollout_is_seq_std:0.002272321842610836 - rollout_corr/rollout_is_seq_max:1.0080233812332153 - rollout_corr/rollout_is_seq_min:0.9929831624031067 - rollout_corr/rollout_is_seq_max_deviation:0.008023381233215332 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4900991539470851) - rollout_corr/training_log_ppl:np.float64(0.3941606451626285) - rollout_corr/kl:np.float64(0.0010211967264037725) - rollout_corr/k3_kl:np.float64(0.001036700705981275) - rollout_corr/rollout_ppl:np.float64(1.488562170881778) - rollout_corr/rollout_log_ppl:np.float64(0.39313944731839) - rollout_corr/log_ppl_diff:np.float64(0.0010211978442384861) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021293721874826588) - rollout_corr/log_ppl_diff_max:np.float64(0.008637219667434692) - rollout_corr/log_ppl_diff_min:np.float64(-0.006850838661193848) - rollout_corr/ppl_ratio:np.float64(1.0010247104801238) - rollout_corr/chi2_token:np.float64(0.002140681492164731) - rollout_corr/chi2_seq:np.float64(5.13545296783559) - actor/pg_loss:np.float64(0.00032719061709940434) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00047709801265227725) - actor/ppo_kl:np.float64(0.00011368624683427697) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3619671016931534) - perf/mfu/actor:np.float64(0.06669451336596828) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(102.79273986816406) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:0 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0071462541818618774 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0071462541818618774 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:468.17578125 - response_length/max:8192.0 - response_length/min:193.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:468.17578125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:193.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:295.5625 - prompt_length/max:532.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010414794087409973 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2887335177510977) - timing_s/agent_loop/generate_sequences/max:np.float64(49.66548676043749) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.864663976142765) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.66548676043749
(TaskRunner pid=2001144) Training Progress: 53%|█████▎ | 53/100 [36:49<40:59, 52.32s/it]
(TaskRunner pid=2001144) step:54 - global_seqlen/min:18238 - global_seqlen/max:24819 - global_seqlen/minmax_diff:6581 - global_seqlen/balanced_min:21517 - global_seqlen/balanced_max:21639 - global_seqlen/mean:21563.0 - actor/entropy:0.3850070536136627 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29233109951019287 - training/rollout_probs_diff_mean:0.0059627084992825985 - training/rollout_probs_diff_std:0.012883388437330723 - training/rollout_actor_probs_pearson_corr:0.99881511926651 - rollout_corr/rollout_is_mean:1.000229001045227 - rollout_corr/rollout_is_ratio_fraction_high:9.69142547546653e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.9566259384155273 - rollout_corr/rollout_is_min:0.5040711760520935 - rollout_corr/rollout_is_std:0.04181580990552902 - rollout_corr/rollout_is_eff_sample_size:0.998255203034057 - rollout_corr/rollout_is_seq_mean:1.0002682209014893 - rollout_corr/rollout_is_seq_std:0.002388204215094447 - rollout_corr/rollout_is_seq_max:1.0080527067184448 - rollout_corr/rollout_is_seq_min:0.9936806559562683 - rollout_corr/rollout_is_seq_max_deviation:0.008052706718444824 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.49410800030455) - rollout_corr/training_log_ppl:np.float64(0.3970826826116536) - rollout_corr/kl:np.float64(0.0007368279453441318) - rollout_corr/k3_kl:np.float64(0.0010290787217854813) - rollout_corr/rollout_ppl:np.float64(1.4929802035912871) - rollout_corr/rollout_log_ppl:np.float64(0.396345855348045) - rollout_corr/log_ppl_diff:np.float64(0.0007368272636085749) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020569806220009923) - rollout_corr/log_ppl_diff_max:np.float64(0.008177697658538818) - rollout_corr/log_ppl_diff_min:np.float64(-0.006545931100845337) - rollout_corr/ppl_ratio:np.float64(1.0007402715273201) - rollout_corr/chi2_token:np.float64(0.0026401157956570387) - rollout_corr/chi2_seq:np.float64(1.610291046788916) - actor/pg_loss:np.float64(-6.428128108382225e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008901855480871745) - actor/ppo_kl:np.float64(8.491149665701414e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4226038455963135) - perf/mfu/actor:np.float64(0.06335077013388332) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(102.71623992919922) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:0 - critic/score/mean:0.5234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00518248975276947 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00518248975276947 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:403.0625 - response_length/max:1321.0 - response_length/min:145.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:403.0625 - response_length_non_aborted/max:1321.0 - response_length_non_aborted/min:145.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.78125 - prompt_length/max:597.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.928186535835266e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7907565478235483) - timing_s/agent_loop/generate_sequences/max:np.float64(9.254914930090308) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.344925412297016) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.254914930090308) - timing_s/agent_loop/slowest/tool_calls:np.floa
(TaskRunner pid=2001144) Training Progress: 54%|█████▍ | 54/100 [37:17<34:25, 44.90s/it]
(TaskRunner pid=2001144) step:55 - global_seqlen/min:20041 - global_seqlen/max:28273 - global_seqlen/minmax_diff:8232 - global_seqlen/balanced_min:24768 - global_seqlen/balanced_max:25416 - global_seqlen/mean:24862.125 - actor/entropy:0.36040911078453064 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7417161464691162 - training/rollout_probs_diff_mean:0.005705657880753279 - training/rollout_probs_diff_std:0.013135991059243679 - training/rollout_actor_probs_pearson_corr:0.9987334609031677 - rollout_corr/rollout_is_mean:0.9998733401298523 - rollout_corr/rollout_is_ratio_fraction_high:4.13411071349401e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.61457670503296e-05 - rollout_corr/rollout_is_max:2.667907953262329 - rollout_corr/rollout_is_min:0.09278460592031479 - rollout_corr/rollout_is_std:0.041808679699897766 - rollout_corr/rollout_is_eff_sample_size:0.9982546090661082 - rollout_corr/rollout_is_seq_mean:0.9998046159744263 - rollout_corr/rollout_is_seq_std:0.0022476217709481716 - rollout_corr/rollout_is_seq_max:1.0061683654785156 - rollout_corr/rollout_is_seq_min:0.9922005534172058 - rollout_corr/rollout_is_seq_max_deviation:0.0077994465827941895 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4778216108679771) - rollout_corr/training_log_ppl:np.float64(0.38318014328251593) - rollout_corr/kl:np.float64(0.0011985727838625948) - rollout_corr/k3_kl:np.float64(0.0011402029837199734) - rollout_corr/rollout_ppl:np.float64(1.4760038666427135) - rollout_corr/rollout_log_ppl:np.float64(0.3819815694587305) - rollout_corr/log_ppl_diff:np.float64(0.0011985738237854093) - rollout_corr/log_ppl_abs_diff:np.float64(0.002102934435242787) - rollout_corr/log_ppl_diff_max:np.float64(0.009179949760437012) - rollout_corr/log_ppl_diff_min:np.float64(-0.0046655237674713135) - rollout_corr/ppl_ratio:np.float64(1.0012026506010443) - rollout_corr/chi2_token:np.float64(0.0022482206113636494) - rollout_corr/chi2_seq:np.float64(0.9269298564177006) - actor/pg_loss:np.float64(0.00013449718244373798) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006665278847322043) - actor/ppo_kl:np.float64(4.007426944596659e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3848492503166199) - perf/mfu/actor:np.float64(0.07180921542469111) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(102.64776992797852) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:0 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005354706663638353 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005354706663638353 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:472.44140625 - response_length/max:2272.0 - response_length/min:206.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:472.44140625 - response_length_non_aborted/max:2272.0 - response_length_non_aborted/min:206.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:304.5 - prompt_length/max:625.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.4050474166870117e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.549405161291361) - timing_s/agent_loop/generate_sequences/max:np.float64(14.281918419525027) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.027420592334238) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.281918419525027) - timing_s/age
(TaskRunner pid=2001144) Training Progress: 55%|█████▌ | 55/100 [37:50<30:59, 41.31s/it]
(TaskRunner pid=2001144) step:56 - global_seqlen/min:17969 - global_seqlen/max:23824 - global_seqlen/minmax_diff:5855 - global_seqlen/balanced_min:21287 - global_seqlen/balanced_max:21306 - global_seqlen/mean:21296.875 - actor/entropy:0.42455294728279114 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37057238817214966 - training/rollout_probs_diff_mean:0.00616619735956192 - training/rollout_probs_diff_std:0.012770228087902069 - training/rollout_actor_probs_pearson_corr:0.9988499283790588 - rollout_corr/rollout_is_mean:1.000146746635437 - rollout_corr/rollout_is_ratio_fraction_high:2.9269720471347682e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.9269720471347682e-05 - rollout_corr/rollout_is_max:2.4842162132263184 - rollout_corr/rollout_is_min:0.44295981526374817 - rollout_corr/rollout_is_std:0.04254356026649475 - rollout_corr/rollout_is_eff_sample_size:0.9981937904899303 - rollout_corr/rollout_is_seq_mean:1.0002650022506714 - rollout_corr/rollout_is_seq_std:0.002604559762403369 - rollout_corr/rollout_is_seq_max:1.011404037475586 - rollout_corr/rollout_is_seq_min:0.991585373878479 - rollout_corr/rollout_is_seq_max_deviation:0.011404037475585938 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5576387261971831) - rollout_corr/training_log_ppl:np.float64(0.4380289943073876) - rollout_corr/kl:np.float64(0.0007650309142608336) - rollout_corr/k3_kl:np.float64(0.0010639836057180219) - rollout_corr/rollout_ppl:np.float64(1.5564593458548188) - rollout_corr/rollout_log_ppl:np.float64(0.4372639606008306) - rollout_corr/log_ppl_diff:np.float64(0.0007650337065570056) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020009888685308397) - rollout_corr/log_ppl_diff_max:np.float64(0.009718269109725952) - rollout_corr/log_ppl_diff_min:np.float64(-0.006958872079849243) - rollout_corr/ppl_ratio:np.float64(1.0007684922311455) - rollout_corr/chi2_token:np.float64(0.002849150914698839) - rollout_corr/chi2_seq:np.float64(1.624248061561957) - actor/pg_loss:np.float64(-0.00010959338396787643) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006753645138815045) - actor/ppo_kl:np.float64(7.183530904342206e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.40421901270747185) - perf/mfu/actor:np.float64(0.06279996364935547) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(102.66665649414062) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:0 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009195570601150393 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009195570601150393 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:400.37109375 - response_length/max:1213.0 - response_length/min:204.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:400.37109375 - response_length_non_aborted/max:1213.0 - response_length_non_aborted/min:204.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.15625 - prompt_length/max:458.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010105036199092865 - timing_s/agent_loop/generate_sequences/min:np.float64(2.439522124826908) - timing_s/agent_loop/generate_sequences/max:np.float64(8.786577094346285) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.378782643907471) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.786577094346285) - ti
(TaskRunner pid=2001144) Training Progress: 56%|█████▌ | 56/100 [38:17<27:14, 37.15s/it]
(TaskRunner pid=2001144) step:57 - global_seqlen/min:19694 - global_seqlen/max:26143 - global_seqlen/minmax_diff:6449 - global_seqlen/balanced_min:22625 - global_seqlen/balanced_max:22634 - global_seqlen/mean:22630.875 - actor/entropy:0.39015623927116394 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8246057629585266 - training/rollout_probs_diff_mean:0.006058368366211653 - training/rollout_probs_diff_std:0.013476626016199589 - training/rollout_actor_probs_pearson_corr:0.9987455606460571 - rollout_corr/rollout_is_mean:0.9998747706413269 - rollout_corr/rollout_is_ratio_fraction_high:3.716401261044666e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.361903019249439e-05 - rollout_corr/rollout_is_max:6.6603217124938965 - rollout_corr/rollout_is_min:0.010470828041434288 - rollout_corr/rollout_is_std:0.04309618100523949 - rollout_corr/rollout_is_eff_sample_size:0.9981455685275196 - rollout_corr/rollout_is_seq_mean:0.999874472618103 - rollout_corr/rollout_is_seq_std:0.002415696857497096 - rollout_corr/rollout_is_seq_max:1.01177179813385 - rollout_corr/rollout_is_seq_min:0.9918592572212219 - rollout_corr/rollout_is_seq_max_deviation:0.011771798133850098 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.505086008924991) - rollout_corr/training_log_ppl:np.float64(0.4036481650546193) - rollout_corr/kl:np.float64(0.0011303338132364615) - rollout_corr/k3_kl:np.float64(0.0010789905898036523) - rollout_corr/rollout_ppl:np.float64(1.503377104178071) - rollout_corr/rollout_log_ppl:np.float64(0.40251782949781045) - rollout_corr/log_ppl_diff:np.float64(0.0011303355568088591) - rollout_corr/log_ppl_abs_diff:np.float64(0.002040403720457107) - rollout_corr/log_ppl_diff_max:np.float64(0.010023295879364014) - rollout_corr/log_ppl_diff_min:np.float64(-0.009803056716918945) - rollout_corr/ppl_ratio:np.float64(1.0011339930351824) - rollout_corr/chi2_token:np.float64(0.002207841956987977) - rollout_corr/chi2_seq:np.float64(1.1339338412508368) - actor/pg_loss:np.float64(0.0001734982943162322) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006692003239550104) - actor/ppo_kl:np.float64(-2.9361545024642055e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.38425227999687195) - perf/mfu/actor:np.float64(0.06520391840079295) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(102.64802932739258) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:0 - critic/score/mean:0.578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0017548382747918367 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0017548382747918367 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:420.43359375 - response_length/max:977.0 - response_length/min:211.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:420.43359375 - response_length_non_aborted/max:977.0 - response_length_non_aborted/min:211.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.78125 - prompt_length/max:660.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.601929008960724e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.5279028844088316) - timing_s/agent_loop/generate_sequences/max:np.float64(7.7026083413511515) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.56056218050071) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.7026083413511515) - timing_s/
(TaskRunner pid=2001144) Training Progress: 57%|█████▋ | 57/100 [38:43<24:13, 33.81s/it]
(TaskRunner pid=2001144) step:58 - global_seqlen/min:21560 - global_seqlen/max:27464 - global_seqlen/minmax_diff:5904 - global_seqlen/balanced_min:23493 - global_seqlen/balanced_max:23656 - global_seqlen/mean:23515.375 - actor/entropy:0.4219380021095276 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35338371992111206 - training/rollout_probs_diff_mean:0.005739074200391769 - training/rollout_probs_diff_std:0.012013072147965431 - training/rollout_actor_probs_pearson_corr:0.9990105628967285 - rollout_corr/rollout_is_mean:1.000062108039856 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.5586135961930268e-05 - rollout_corr/rollout_is_max:1.8115310668945312 - rollout_corr/rollout_is_min:0.24191930890083313 - rollout_corr/rollout_is_std:0.04003310576081276 - rollout_corr/rollout_is_eff_sample_size:0.9984001523437732 - rollout_corr/rollout_is_seq_mean:1.0000706911087036 - rollout_corr/rollout_is_seq_std:0.0024040325079113245 - rollout_corr/rollout_is_seq_max:1.0109398365020752 - rollout_corr/rollout_is_seq_min:0.9924777150154114 - rollout_corr/rollout_is_seq_max_deviation:0.010939836502075195 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5533099682070315) - rollout_corr/training_log_ppl:np.float64(0.4348131374572404) - rollout_corr/kl:np.float64(0.0008075449165616533) - rollout_corr/k3_kl:np.float64(0.0009305526820071464) - rollout_corr/rollout_ppl:np.float64(1.5520000508986413) - rollout_corr/rollout_log_ppl:np.float64(0.4340055894572288) - rollout_corr/log_ppl_diff:np.float64(0.0008075480000115931) - rollout_corr/log_ppl_abs_diff:np.float64(0.002058708167169243) - rollout_corr/log_ppl_diff_max:np.float64(0.011946380138397217) - rollout_corr/log_ppl_diff_min:np.float64(-0.005818605422973633) - rollout_corr/ppl_ratio:np.float64(1.0008111805655062) - rollout_corr/chi2_token:np.float64(0.002111998852342367) - rollout_corr/chi2_seq:np.float64(1.8191634574905038) - actor/pg_loss:np.float64(5.404348485171795e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004776264283918863) - actor/ppo_kl:np.float64(2.556468981396165e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3422366492450237) - perf/mfu/actor:np.float64(0.06855371924366756) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(102.77903366088867) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:0 - critic/score/mean:0.6015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:9.488192154094577e-05 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:9.488192154094577e-05 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:458.01171875 - response_length/max:1421.0 - response_length/min:168.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:458.01171875 - response_length_non_aborted/max:1421.0 - response_length_non_aborted/min:168.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.84375 - prompt_length/max:598.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015367567539215088 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9693270549178123) - timing_s/agent_loop/generate_sequences/max:np.float64(9.921870244666934) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.85390070592257) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.921870244666934) - timing_s/agent_loop/slowe
(TaskRunner pid=2001144) Training Progress: 58%|█████▊ | 58/100 [39:11<22:29, 32.13s/it]
(TaskRunner pid=2001144) step:59 - global_seqlen/min:17760 - global_seqlen/max:24464 - global_seqlen/minmax_diff:6704 - global_seqlen/balanced_min:21572 - global_seqlen/balanced_max:21593 - global_seqlen/mean:21585.375 - actor/entropy:0.43182381987571716 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35146763920783997 - training/rollout_probs_diff_mean:0.005994895938783884 - training/rollout_probs_diff_std:0.012256143614649773 - training/rollout_actor_probs_pearson_corr:0.9989531636238098 - rollout_corr/rollout_is_mean:1.000177264213562 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9243536371504888e-05 - rollout_corr/rollout_is_max:1.8311649560928345 - rollout_corr/rollout_is_min:0.3545463979244232 - rollout_corr/rollout_is_std:0.040910858660936356 - rollout_corr/rollout_is_eff_sample_size:0.9983296922170798 - rollout_corr/rollout_is_seq_mean:1.0001717805862427 - rollout_corr/rollout_is_seq_std:0.0021411844063550234 - rollout_corr/rollout_is_seq_max:1.00700843334198 - rollout_corr/rollout_is_seq_min:0.9944503307342529 - rollout_corr/rollout_is_seq_max_deviation:0.0070084333419799805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5644833943806589) - rollout_corr/training_log_ppl:np.float64(0.44385944720124826) - rollout_corr/kl:np.float64(0.0008257440472836386) - rollout_corr/k3_kl:np.float64(0.0010107478150302995) - rollout_corr/rollout_ppl:np.float64(1.5631401697173715) - rollout_corr/rollout_log_ppl:np.float64(0.4430336999357678) - rollout_corr/log_ppl_diff:np.float64(0.0008257472654804587) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020738329039886594) - rollout_corr/log_ppl_diff_max:np.float64(0.012843847274780273) - rollout_corr/log_ppl_diff_min:np.float64(-0.007352948188781738) - rollout_corr/ppl_ratio:np.float64(1.0008293797727674) - rollout_corr/chi2_token:np.float64(0.0023556456435471773) - rollout_corr/chi2_seq:np.float64(2.9180437861941755) - actor/pg_loss:np.float64(-0.00019804073963314295) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00047333618658740306) - actor/ppo_kl:np.float64(0.0001295031242043443) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.39384642243385315) - perf/mfu/actor:np.float64(0.06261116409778117) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(102.64363098144531) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:0 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004357458092272282 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004357458092272282 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:405.98046875 - response_length/max:1377.0 - response_length/min:189.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:405.98046875 - response_length_non_aborted/max:1377.0 - response_length_non_aborted/min:189.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.5625 - prompt_length/max:615.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.935505032539368e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2591890152543783) - timing_s/agent_loop/generate_sequences/max:np.float64(9.403489911928773) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.364368350521545) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.403489911928773) - timing_s/agent_loop/s
(TaskRunner pid=2001144) Training Progress: 59%|█████▉ | 59/100 [39:39<21:02, 30.80s/it]
(TaskRunner pid=2001144) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60}
(TaskRunner pid=2001144) validation generation end
(TaskRunner pid=2001144) [prompt] system
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2001144) <reasoning>
(TaskRunner pid=2001144) ...
(TaskRunner pid=2001144) </reasoning>
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) ...
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) A
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) user
(TaskRunner pid=2001144) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2001144) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2001144) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2001144) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2001144) Please reason step by step.
(TaskRunner pid=2001144) assistant
(TaskRunner pid=2001144) <think>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) </think>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) [response] <reasoning>
(TaskRunner pid=2001144) The question asks which of the given options is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". This implies that the correct answer is the one that matches the structure of the product or is a direct precursor to it.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Let's analyze the product structure:
(TaskRunner pid=2001144) "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"
(TaskRunner pid=2001144) This is a complex organic molecule with a nitrogen-containing ring system, and it contains a substituted benzene ring (with a Cl and a trifluoromethyl group, C(F)(F)F).
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Now, let's examine the options:
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) **Option A:**
(TaskRunner pid=2001144) O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2001144) This option has a very similar structure to the product. It contains the same key substituents: a Cl on a benzene ring and a trifluoromethyl group (C(F)(F)F). It also includes a nitrogen atom connected to a carbon chain, which matches the structure of the product. The only difference is the presence of an oxygen double bond (C=O) in the structure, but that may be part of a functional group that is not critical to the synthesis of the product. This option appears to be the most similar to the product.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) **Option B:**
(TaskRunner pid=2001144) O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2001144) This option has a Cl on the benzene ring, but the rest of the structure is different. It has a different nitrogen-containing chain and does not match the product's structure closely.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) **Option C:**
(TaskRunner pid=2001144) O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2001144) This option includes a more complex substituent with an oxygen and a longer chain. While it includes a Cl and a trifluoromethyl group, the overall structure is more complicated and does not match the product's simpler structure.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) **Option D:**
(TaskRunner pid=2001144) O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2001144) This option has a different structure, with a double bond and a different substituent pattern. It does not match the product's structure as closely as Option A.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Based on the similarity in structure and the presence of the key substituents (Cl and C(F)(F)F), **Option A** is the most likely correct reactant used in the synthesis of the product.
(TaskRunner pid=2001144) </reasoning>
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) A
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144) [ground_truth] A
(TaskRunner pid=2001144) [score] 1.0
(TaskRunner pid=2001144) [acc] 1.0
(TaskRunner pid=2001144) [pred] A
(TaskRunner pid=2001144) [incorrect_format] 1
(TaskRunner pid=2001144) [feedback]
(TaskRunner pid=2001144) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60
(WorkerDict pid=2001534) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50/actor
(TaskRunner pid=2001144) step:60 - global_seqlen/min:17842 - global_seqlen/max:25210 - global_seqlen/minmax_diff:7368 - global_seqlen/balanced_min:21970 - global_seqlen/balanced_max:21982 - global_seqlen/mean:21979.375 - actor/entropy:0.4403538703918457 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.41013577580451965 - training/rollout_probs_diff_mean:0.006110155954957008 - training/rollout_probs_diff_std:0.012367784976959229 - training/rollout_actor_probs_pearson_corr:0.9989744424819946 - rollout_corr/rollout_is_mean:1.0003244876861572 - rollout_corr/rollout_is_ratio_fraction_high:2.7840935217682272e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.280312042392325e-06 - rollout_corr/rollout_is_max:2.604189157485962 - rollout_corr/rollout_is_min:0.03039470501244068 - rollout_corr/rollout_is_std:0.04170019179582596 - rollout_corr/rollout_is_eff_sample_size:0.9982651818012834 - rollout_corr/rollout_is_seq_mean:1.000322699546814 - rollout_corr/rollout_is_seq_std:0.002230000915005803 - rollout_corr/rollout_is_seq_max:1.0089749097824097 - rollout_corr/rollout_is_seq_min:0.9945498704910278 - rollout_corr/rollout_is_seq_max_deviation:0.008974909782409668 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5822291895747185) - rollout_corr/training_log_ppl:np.float64(0.45394952106289566) - rollout_corr/kl:np.float64(0.0008267760611460062) - rollout_corr/k3_kl:np.float64(0.001037105767295543) - rollout_corr/rollout_ppl:np.float64(1.580900760833174) - rollout_corr/rollout_log_ppl:np.float64(0.4531227457919158) - rollout_corr/log_ppl_diff:np.float64(0.0008267752709798515) - rollout_corr/log_ppl_abs_diff:np.float64(0.002010064839851111) - rollout_corr/log_ppl_diff_max:np.float64(0.008430540561676025) - rollout_corr/log_ppl_diff_min:np.float64(-0.006060987710952759) - rollout_corr/ppl_ratio:np.float64(1.0008301613852382) - rollout_corr/chi2_token:np.float64(0.0024820046965032816) - rollout_corr/chi2_seq:np.float64(1.2146960981190205) - actor/pg_loss:np.float64(-0.00010784179903566837) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005563689874179545) - actor/ppo_kl:np.float64(0.0002278029360631706) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4723544269800186) - perf/mfu/actor:np.float64(0.06387134510710415) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(103.39831161499023) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6482142857142857) - val-aux/sciknoweval/reward/std@16:np.float64(0.17550310418233606) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7200380952380953) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13716182872628702) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5768857142857143) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1602966897230937) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.647809523809524) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.17526730896119697) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7711142857142855) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09262764045726349) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5128190476190476) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12952890739984624) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.662547619047619) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.14027669807753104) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8054761904761903) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.059423992554637685) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.46027142857142855) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09520898740910534) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6721761904761904) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1034627186238184) - val-aux/sciknoweval/reward/best@
(TaskRunner pid=2001144) Training Progress: 60%|██████ | 60/100 [42:21<46:42, 70.07s/it]
(TaskRunner pid=2001144) step:61 - global_seqlen/min:17215 - global_seqlen/max:24908 - global_seqlen/minmax_diff:7693 - global_seqlen/balanced_min:21739 - global_seqlen/balanced_max:21744 - global_seqlen/mean:21742.375 - actor/entropy:0.41738754510879517 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2932341992855072 - training/rollout_probs_diff_mean:0.005850965157151222 - training/rollout_probs_diff_std:0.012236741371452808 - training/rollout_actor_probs_pearson_corr:0.9989736080169678 - rollout_corr/rollout_is_mean:0.9999605417251587 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.8074378860765137e-05 - rollout_corr/rollout_is_max:1.9552534818649292 - rollout_corr/rollout_is_min:0.17450448870658875 - rollout_corr/rollout_is_std:0.04126771166920662 - rollout_corr/rollout_is_eff_sample_size:0.998299752644773 - rollout_corr/rollout_is_seq_mean:0.9999628067016602 - rollout_corr/rollout_is_seq_std:0.0023588729090988636 - rollout_corr/rollout_is_seq_max:1.0081961154937744 - rollout_corr/rollout_is_seq_min:0.9935832023620605 - rollout_corr/rollout_is_seq_max_deviation:0.008196115493774414 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5585635351017118) - rollout_corr/training_log_ppl:np.float64(0.4367933518951759) - rollout_corr/kl:np.float64(0.0012103183988756427) - rollout_corr/k3_kl:np.float64(0.001033652987757705) - rollout_corr/rollout_ppl:np.float64(1.5566195617429912) - rollout_corr/rollout_log_ppl:np.float64(0.43558303377358243) - rollout_corr/log_ppl_diff:np.float64(0.0012103181215934455) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022993262973614037) - rollout_corr/log_ppl_diff_max:np.float64(0.009897172451019287) - rollout_corr/log_ppl_diff_min:np.float64(-0.007474124431610107) - rollout_corr/ppl_ratio:np.float64(1.00121472007595) - rollout_corr/chi2_token:np.float64(0.0016955803148448467) - rollout_corr/chi2_seq:np.float64(2.497442876221612) - actor/pg_loss:np.float64(-1.17946183308959e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000651234391170874) - actor/ppo_kl:np.float64(0.00026272342059030507) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3500925600528717) - perf/mfu/actor:np.float64(0.06256922497689921) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(106.35950088500977) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003430922981351614 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003430922981351614 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:417.41796875 - response_length/max:870.0 - response_length/min:181.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:417.41796875 - response_length_non_aborted/max:870.0 - response_length_non_aborted/min:181.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.03125 - prompt_length/max:462.0 - prompt_length/min:177.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001322925090789795 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7889548633247614) - timing_s/agent_loop/generate_sequences/max:np.float64(7.4360432624816895) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.45666817464371) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.4360432624816895) - timing_s/agent_loop/slowest/t
(TaskRunner pid=2001144) Training Progress: 61%|██████ | 61/100 [42:47<36:56, 56.82s/it]
(TaskRunner pid=2001144) step:62 - global_seqlen/min:20011 - global_seqlen/max:25671 - global_seqlen/minmax_diff:5660 - global_seqlen/balanced_min:22766 - global_seqlen/balanced_max:22772 - global_seqlen/mean:22769.375 - actor/entropy:0.41837045550346375 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34792429208755493 - training/rollout_probs_diff_mean:0.00607671495527029 - training/rollout_probs_diff_std:0.012674469500780106 - training/rollout_actor_probs_pearson_corr:0.9988909959793091 - rollout_corr/rollout_is_mean:0.9998384118080139 - rollout_corr/rollout_is_ratio_fraction_high:1.8948903743876144e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.789780748775229e-05 - rollout_corr/rollout_is_max:2.3115429878234863 - rollout_corr/rollout_is_min:0.2821033298969269 - rollout_corr/rollout_is_std:0.042287807911634445 - rollout_corr/rollout_is_eff_sample_size:0.9982144584716122 - rollout_corr/rollout_is_seq_mean:0.9998161792755127 - rollout_corr/rollout_is_seq_std:0.002178584923967719 - rollout_corr/rollout_is_seq_max:1.0047497749328613 - rollout_corr/rollout_is_seq_min:0.9922394752502441 - rollout_corr/rollout_is_seq_max_deviation:0.007760524749755859 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5434541776776314) - rollout_corr/training_log_ppl:np.float64(0.4284537000930868) - rollout_corr/kl:np.float64(0.0014265859828768157) - rollout_corr/k3_kl:np.float64(0.0012022101592492618) - rollout_corr/rollout_ppl:np.float64(1.5411579930223525) - rollout_corr/rollout_log_ppl:np.float64(0.42702711204765365) - rollout_corr/log_ppl_diff:np.float64(0.0014265880454331636) - rollout_corr/log_ppl_abs_diff:np.float64(0.002385147847235203) - rollout_corr/log_ppl_diff_max:np.float64(0.008987247943878174) - rollout_corr/log_ppl_diff_min:np.float64(-0.007447272539138794) - rollout_corr/ppl_ratio:np.float64(1.0014316337183118) - rollout_corr/chi2_token:np.float64(0.0019206618890166283) - rollout_corr/chi2_seq:np.float64(1.6949006277136505) - actor/pg_loss:np.float64(0.00029082188848406076) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014474346698989393) - actor/ppo_kl:np.float64(0.0003121974393627269) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4932637959718704) - perf/mfu/actor:np.float64(0.06602033667516095) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(105.81099700927734) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:1 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00048556565889157355 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00048556565889157355 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:412.29296875 - response_length/max:1078.0 - response_length/min:169.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:412.29296875 - response_length_non_aborted/max:1078.0 - response_length_non_aborted/min:169.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.25 - prompt_length/max:662.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.993836283683777e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.992330465465784) - timing_s/agent_loop/generate_sequences/max:np.float64(8.521201653406024) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.473851929833472) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.521201653406024) - timi
(TaskRunner pid=2001144) Training Progress: 62%|██████▏ | 62/100 [43:14<30:18, 47.85s/it]
(TaskRunner pid=2001144) step:63 - global_seqlen/min:18608 - global_seqlen/max:27831 - global_seqlen/minmax_diff:9223 - global_seqlen/balanced_min:22760 - global_seqlen/balanced_max:22955 - global_seqlen/mean:22806.25 - actor/entropy:0.4153381586074829 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5485354661941528 - training/rollout_probs_diff_mean:0.006234325002878904 - training/rollout_probs_diff_std:0.013178987428545952 - training/rollout_actor_probs_pearson_corr:0.9988259673118591 - rollout_corr/rollout_is_mean:0.999946117401123 - rollout_corr/rollout_is_ratio_fraction_high:3.663741881609894e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.159354522125795e-05 - rollout_corr/rollout_is_max:3.8425495624542236 - rollout_corr/rollout_is_min:0.23784394562244415 - rollout_corr/rollout_is_std:0.04415774345397949 - rollout_corr/rollout_is_eff_sample_size:0.9980536508947626 - rollout_corr/rollout_is_seq_mean:0.9999497532844543 - rollout_corr/rollout_is_seq_std:0.002399944933131337 - rollout_corr/rollout_is_seq_max:1.006082534790039 - rollout_corr/rollout_is_seq_min:0.9925247430801392 - rollout_corr/rollout_is_seq_max_deviation:0.00747525691986084 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5512802684679627) - rollout_corr/training_log_ppl:np.float64(0.43343031173571944) - rollout_corr/kl:np.float64(0.0011172183292629256) - rollout_corr/k3_kl:np.float64(0.0011449105093106482) - rollout_corr/rollout_ppl:np.float64(1.5495519577525556) - rollout_corr/rollout_log_ppl:np.float64(0.4323130922857672) - rollout_corr/log_ppl_diff:np.float64(0.0011172194499522448) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020810606656596065) - rollout_corr/log_ppl_diff_max:np.float64(0.008500427007675171) - rollout_corr/log_ppl_diff_min:np.float64(-0.005176037549972534) - rollout_corr/ppl_ratio:np.float64(1.001120786415413) - rollout_corr/chi2_token:np.float64(0.0024358213413506746) - rollout_corr/chi2_seq:np.float64(0.7318861426319927) - actor/pg_loss:np.float64(-2.7049100026488304e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009216382943577628) - actor/ppo_kl:np.float64(3.8369405875204876e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4949973076581955) - perf/mfu/actor:np.float64(0.06557468969580606) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(105.15737533569336) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:1 - critic/score/mean:0.51953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0029687758069485426 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0029687758069485426 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:426.4765625 - response_length/max:1606.0 - response_length/min:191.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:426.4765625 - response_length_non_aborted/max:1606.0 - response_length_non_aborted/min:191.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.21875 - prompt_length/max:532.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010391324758529663 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3714834824204445) - timing_s/agent_loop/generate_sequences/max:np.float64(10.988159496337175) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.584561837160436) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.988159496337175) - timin
(TaskRunner pid=2001144) Training Progress: 63%|██████▎ | 63/100 [43:43<26:05, 42.32s/it]
(TaskRunner pid=2001144) step:64 - global_seqlen/min:17762 - global_seqlen/max:26400 - global_seqlen/minmax_diff:8638 - global_seqlen/balanced_min:23854 - global_seqlen/balanced_max:23866 - global_seqlen/mean:23862.625 - actor/entropy:0.41882699728012085 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27078545093536377 - training/rollout_probs_diff_mean:0.005884089507162571 - training/rollout_probs_diff_std:0.012289062142372131 - training/rollout_actor_probs_pearson_corr:0.9989817142486572 - rollout_corr/rollout_is_mean:0.9997639656066895 - rollout_corr/rollout_is_ratio_fraction_high:1.7247772120754234e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.623886060377117e-06 - rollout_corr/rollout_is_max:2.404822826385498 - rollout_corr/rollout_is_min:0.3133420944213867 - rollout_corr/rollout_is_std:0.04151180386543274 - rollout_corr/rollout_is_eff_sample_size:0.998278962331624 - rollout_corr/rollout_is_seq_mean:0.9996981024742126 - rollout_corr/rollout_is_seq_std:0.0020679025910794735 - rollout_corr/rollout_is_seq_max:1.0065325498580933 - rollout_corr/rollout_is_seq_min:0.9942275285720825 - rollout_corr/rollout_is_seq_max_deviation:0.006532549858093262 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5537331653758883) - rollout_corr/training_log_ppl:np.float64(0.43447394081158563) - rollout_corr/kl:np.float64(0.001242162329816665) - rollout_corr/k3_kl:np.float64(0.0009837684399371938) - rollout_corr/rollout_ppl:np.float64(1.5517677855677903) - rollout_corr/rollout_log_ppl:np.float64(0.4332317790831439) - rollout_corr/log_ppl_diff:np.float64(0.0012421617284417152) - rollout_corr/log_ppl_abs_diff:np.float64(0.002061901264823973) - rollout_corr/log_ppl_diff_max:np.float64(0.008058756589889526) - rollout_corr/log_ppl_diff_min:np.float64(-0.004633873701095581) - rollout_corr/ppl_ratio:np.float64(1.0012455550022423) - rollout_corr/chi2_token:np.float64(0.0014252380933612585) - rollout_corr/chi2_seq:np.float64(2.18913138192147) - actor/pg_loss:np.float64(8.033821359276772e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005663788310812379) - actor/ppo_kl:np.float64(3.301178976489183e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3284470736980438) - perf/mfu/actor:np.float64(0.06997136391340386) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(103.40476608276367) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:1 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002546202391386032 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002546202391386032 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:452.95703125 - response_length/max:1042.0 - response_length/min:183.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:452.95703125 - response_length_non_aborted/max:1042.0 - response_length_non_aborted/min:183.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:292.75 - prompt_length/max:495.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010465271770954132 - timing_s/agent_loop/generate_sequences/min:np.float64(2.173621702939272) - timing_s/agent_loop/generate_sequences/max:np.float64(8.345015175640583) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.927353117287566) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.345015175640583) - timing_s/a
(TaskRunner pid=2001144) Training Progress: 64%|██████▍ | 64/100 [44:10<22:37, 37.70s/it]
(TaskRunner pid=2001144) step:65 - global_seqlen/min:18896 - global_seqlen/max:25575 - global_seqlen/minmax_diff:6679 - global_seqlen/balanced_min:23294 - global_seqlen/balanced_max:23301 - global_seqlen/mean:23299.0 - actor/entropy:0.4230119287967682 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29106032848358154 - training/rollout_probs_diff_mean:0.006009966600686312 - training/rollout_probs_diff_std:0.012522626668214798 - training/rollout_actor_probs_pearson_corr:0.9989484548568726 - rollout_corr/rollout_is_mean:0.9999626278877258 - rollout_corr/rollout_is_ratio_fraction_high:8.787964361545164e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.272778435028158e-05 - rollout_corr/rollout_is_max:2.695523500442505 - rollout_corr/rollout_is_min:0.27054542303085327 - rollout_corr/rollout_is_std:0.04202907532453537 - rollout_corr/rollout_is_eff_sample_size:0.9982364339886056 - rollout_corr/rollout_is_seq_mean:1.0000206232070923 - rollout_corr/rollout_is_seq_std:0.0021546559873968363 - rollout_corr/rollout_is_seq_max:1.0059337615966797 - rollout_corr/rollout_is_seq_min:0.9937772154808044 - rollout_corr/rollout_is_seq_max_deviation:0.006222784519195557 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.549645908176899) - rollout_corr/training_log_ppl:np.float64(0.4326260302623268) - rollout_corr/kl:np.float64(0.0009405948233940364) - rollout_corr/k3_kl:np.float64(0.0009876283462517677) - rollout_corr/rollout_ppl:np.float64(1.5481612221337855) - rollout_corr/rollout_log_ppl:np.float64(0.4316854359640274) - rollout_corr/log_ppl_diff:np.float64(0.000940594298299402) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017565569723956287) - rollout_corr/log_ppl_diff_max:np.float64(0.007013112306594849) - rollout_corr/log_ppl_diff_min:np.float64(-0.0055883824825286865) - rollout_corr/ppl_ratio:np.float64(1.0009432421065867) - rollout_corr/chi2_token:np.float64(0.002048040274530649) - rollout_corr/chi2_seq:np.float64(1.7270857498515397) - actor/pg_loss:np.float64(0.00020612275693565607) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002279625578012201) - actor/ppo_kl:np.float64(4.630882359357713e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.31356867775321007) - perf/mfu/actor:np.float64(0.06801990010768477) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(103.5004653930664) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:1 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0017081605037674308 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0017081605037674308 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:444.5 - response_length/max:935.0 - response_length/min:221.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:444.5 - response_length_non_aborted/max:935.0 - response_length_non_aborted/min:221.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.59375 - prompt_length/max:818.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.537806570529938e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6582496240735054) - timing_s/agent_loop/generate_sequences/max:np.float64(7.792725443840027) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.904458262935805) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.792725443840027) - timing_s/agent_loop/slo
(TaskRunner pid=2001144) Training Progress: 65%|██████▌ | 65/100 [44:36<19:58, 34.26s/it]
(TaskRunner pid=2001144) step:66 - global_seqlen/min:20285 - global_seqlen/max:27405 - global_seqlen/minmax_diff:7120 - global_seqlen/balanced_min:23992 - global_seqlen/balanced_max:24054 - global_seqlen/mean:24001.5 - actor/entropy:0.42116597294807434 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.780156135559082 - training/rollout_probs_diff_mean:0.005888564977794886 - training/rollout_probs_diff_std:0.012623178772628307 - training/rollout_actor_probs_pearson_corr:0.998918890953064 - rollout_corr/rollout_is_mean:1.0000916719436646 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.796479283366352e-05 - rollout_corr/rollout_is_max:1.8171378374099731 - rollout_corr/rollout_is_min:0.02513178624212742 - rollout_corr/rollout_is_std:0.04106064885854721 - rollout_corr/rollout_is_eff_sample_size:0.9983173359854819 - rollout_corr/rollout_is_seq_mean:1.000185489654541 - rollout_corr/rollout_is_seq_std:0.0020307281520217657 - rollout_corr/rollout_is_seq_max:1.0078269243240356 - rollout_corr/rollout_is_seq_min:0.9939817786216736 - rollout_corr/rollout_is_seq_max_deviation:0.007826924324035645 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5610650107264519) - rollout_corr/training_log_ppl:np.float64(0.43970605652430095) - rollout_corr/kl:np.float64(0.0009164250218702819) - rollout_corr/k3_kl:np.float64(0.0010040769424222162) - rollout_corr/rollout_ppl:np.float64(1.5596140241250396) - rollout_corr/rollout_log_ppl:np.float64(0.4387896290572826) - rollout_corr/log_ppl_diff:np.float64(0.0009164274670183659) - rollout_corr/log_ppl_abs_diff:np.float64(0.002002356108278036) - rollout_corr/log_ppl_diff_max:np.float64(0.008027374744415283) - rollout_corr/log_ppl_diff_min:np.float64(-0.0069466233253479) - rollout_corr/ppl_ratio:np.float64(1.0009195969905704) - rollout_corr/chi2_token:np.float64(0.002093494636937976) - rollout_corr/chi2_seq:np.float64(1.019367356551811) - actor/pg_loss:np.float64(8.354021701961756e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003971411197198904) - actor/ppo_kl:np.float64(0.00016613948628219077) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.26465531811118126) - perf/mfu/actor:np.float64(0.0697737026713732) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(103.52342081069946) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:1 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001576995593495667 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001576995593495667 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:459.796875 - response_length/max:1418.0 - response_length/min:230.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:459.796875 - response_length_non_aborted/max:1418.0 - response_length_non_aborted/min:230.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.25 - prompt_length/max:617.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.625591337680817e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.806927666068077) - timing_s/agent_loop/generate_sequences/max:np.float64(10.129601622000337) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.985797041066689) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.129601622000337) - timing_s/agent_loop/slowest/tool_ca
(TaskRunner pid=2001144) Training Progress: 66%|██████▌ | 66/100 [45:05<18:27, 32.59s/it]
(TaskRunner pid=2001144) step:67 - global_seqlen/min:19798 - global_seqlen/max:28177 - global_seqlen/minmax_diff:8379 - global_seqlen/balanced_min:24209 - global_seqlen/balanced_max:24224 - global_seqlen/mean:24221.625 - actor/entropy:0.4108027219772339 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22918221354484558 - training/rollout_probs_diff_mean:0.005777637008577585 - training/rollout_probs_diff_std:0.012154486030340195 - training/rollout_actor_probs_pearson_corr:0.998947262763977 - rollout_corr/rollout_is_mean:0.9999423027038574 - rollout_corr/rollout_is_ratio_fraction_high:8.399056241614744e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.399056241614744e-06 - rollout_corr/rollout_is_max:2.099741220474243 - rollout_corr/rollout_is_min:0.3923536539077759 - rollout_corr/rollout_is_std:0.04051116108894348 - rollout_corr/rollout_is_eff_sample_size:0.9983614158661788 - rollout_corr/rollout_is_seq_mean:0.9998986721038818 - rollout_corr/rollout_is_seq_std:0.0021807404700666666 - rollout_corr/rollout_is_seq_max:1.006320595741272 - rollout_corr/rollout_is_seq_min:0.9927462935447693 - rollout_corr/rollout_is_seq_max_deviation:0.007253706455230713 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5362624898552895) - rollout_corr/training_log_ppl:np.float64(0.42262632783968) - rollout_corr/kl:np.float64(0.0009958227703705802) - rollout_corr/k3_kl:np.float64(0.0009492553913332813) - rollout_corr/rollout_ppl:np.float64(1.5347287938930094) - rollout_corr/rollout_log_ppl:np.float64(0.42163050116505474) - rollout_corr/log_ppl_diff:np.float64(0.0009958266746252775) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018602811032906175) - rollout_corr/log_ppl_diff_max:np.float64(0.012781322002410889) - rollout_corr/log_ppl_diff_min:np.float64(-0.004694998264312744) - rollout_corr/ppl_ratio:np.float64(1.000998825300485) - rollout_corr/chi2_token:np.float64(0.0018126959912478924) - rollout_corr/chi2_seq:np.float64(1.4865628469269723) - actor/pg_loss:np.float64(5.797401536256075e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004973736013198504) - actor/ppo_kl:np.float64(2.980629317050898e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4306974336504936) - perf/mfu/actor:np.float64(0.07046976768490616) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(103.43095779418945) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:1 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001775350421667099 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001775350421667099 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:465.08203125 - response_length/max:1328.0 - response_length/min:218.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:465.08203125 - response_length_non_aborted/max:1328.0 - response_length_non_aborted/min:218.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.84375 - prompt_length/max:576.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001113060861825943 - timing_s/agent_loop/generate_sequences/min:np.float64(1.614652095362544) - timing_s/agent_loop/generate_sequences/max:np.float64(9.36545778810978) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.509383130192873) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.36545778810978) - timing_s/agent_loop/sl
(TaskRunner pid=2001144) Training Progress: 67%|██████▋ | 67/100 [45:33<17:09, 31.19s/it]
(TaskRunner pid=2001144) step:68 - global_seqlen/min:19448 - global_seqlen/max:26477 - global_seqlen/minmax_diff:7029 - global_seqlen/balanced_min:23041 - global_seqlen/balanced_max:23049 - global_seqlen/mean:23046.125 - actor/entropy:0.4589834213256836 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3341520130634308 - training/rollout_probs_diff_mean:0.0065439967438578606 - training/rollout_probs_diff_std:0.01288579124957323 - training/rollout_actor_probs_pearson_corr:0.9989064931869507 - rollout_corr/rollout_is_mean:1.0000734329223633 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.787427653558552e-05 - rollout_corr/rollout_is_max:1.6649922132492065 - rollout_corr/rollout_is_min:0.29720374941825867 - rollout_corr/rollout_is_std:0.04406991973519325 - rollout_corr/rollout_is_eff_sample_size:0.9980619631626657 - rollout_corr/rollout_is_seq_mean:1.0000429153442383 - rollout_corr/rollout_is_seq_std:0.0023389861453324556 - rollout_corr/rollout_is_seq_max:1.0063287019729614 - rollout_corr/rollout_is_seq_min:0.9928947687149048 - rollout_corr/rollout_is_seq_max_deviation:0.007105231285095215 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6037817248143256) - rollout_corr/training_log_ppl:np.float64(0.46730721392668784) - rollout_corr/kl:np.float64(0.001080117375325429) - rollout_corr/k3_kl:np.float64(0.0011220512478757882) - rollout_corr/rollout_ppl:np.float64(1.6019747834652662) - rollout_corr/rollout_log_ppl:np.float64(0.46622709502116777) - rollout_corr/log_ppl_diff:np.float64(0.0010801189055200666) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022775835532229394) - rollout_corr/log_ppl_diff_max:np.float64(0.01031041145324707) - rollout_corr/log_ppl_diff_min:np.float64(-0.007001012563705444) - rollout_corr/ppl_ratio:np.float64(1.0010844324715436) - rollout_corr/chi2_token:np.float64(0.002279511420056224) - rollout_corr/chi2_seq:np.float64(2.0179682401940227) - actor/pg_loss:np.float64(-0.00015560665633529425) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007488313690373616) - actor/ppo_kl:np.float64(0.00013123494263389546) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3338998481631279) - perf/mfu/actor:np.float64(0.06703447123385581) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(103.40663528442383) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:1 - critic/score/mean:0.40625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0011164912721142173 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0011164912721142173 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:404.97265625 - response_length/max:877.0 - response_length/min:145.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:404.97265625 - response_length_non_aborted/max:877.0 - response_length_non_aborted/min:145.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.21875 - prompt_length/max:704.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001519918441772461 - timing_s/agent_loop/generate_sequences/min:np.float64(1.768206236883998) - timing_s/agent_loop/generate_sequences/max:np.float64(7.227780120447278) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.498167933677905) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.227780120447278) - timing_s/agent_loop/slowest
(TaskRunner pid=2001144) Training Progress: 68%|██████▊ | 68/100 [45:59<15:50, 29.69s/it]
(TaskRunner pid=2001144) step:69 - global_seqlen/min:21048 - global_seqlen/max:30095 - global_seqlen/minmax_diff:9047 - global_seqlen/balanced_min:25084 - global_seqlen/balanced_max:31852 - global_seqlen/mean:25936.75 - actor/entropy:0.38279062509536743 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43752580881118774 - training/rollout_probs_diff_mean:0.005642194766551256 - training/rollout_probs_diff_std:0.012398446910083294 - training/rollout_actor_probs_pearson_corr:0.9989319443702698 - rollout_corr/rollout_is_mean:0.9999570250511169 - rollout_corr/rollout_is_ratio_fraction_high:7.801407264196314e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.46098526683636e-05 - rollout_corr/rollout_is_max:5.670766830444336 - rollout_corr/rollout_is_min:0.39600396156311035 - rollout_corr/rollout_is_std:0.040910858660936356 - rollout_corr/rollout_is_eff_sample_size:0.9983287417268669 - rollout_corr/rollout_is_seq_mean:0.9998832941055298 - rollout_corr/rollout_is_seq_std:0.002169084269553423 - rollout_corr/rollout_is_seq_max:1.0094548463821411 - rollout_corr/rollout_is_seq_min:0.992139458656311 - rollout_corr/rollout_is_seq_max_deviation:0.009454846382141113 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5398975536227226) - rollout_corr/training_log_ppl:np.float64(0.42568120550367894) - rollout_corr/kl:np.float64(0.0011571167556283513) - rollout_corr/k3_kl:np.float64(0.0010113259275499331) - rollout_corr/rollout_ppl:np.float64(1.5380508783273399) - rollout_corr/rollout_log_ppl:np.float64(0.42452408979261236) - rollout_corr/log_ppl_diff:np.float64(0.0011571157110665808) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020482658028413425) - rollout_corr/log_ppl_diff_max:np.float64(0.007831275463104248) - rollout_corr/log_ppl_diff_min:np.float64(-0.005546361207962036) - rollout_corr/ppl_ratio:np.float64(1.0011605005711317) - rollout_corr/chi2_token:np.float64(0.001685052178800106) - rollout_corr/chi2_seq:np.float64(0.8799947444349527) - actor/pg_loss:np.float64(0.00020205590408295393) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005960097582828894) - actor/ppo_kl:np.float64(8.243544034058914e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.36086802929639816) - perf/mfu/actor:np.float64(0.07015468983454687) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(103.41557693481445) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:1 - critic/score/mean:0.6015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0020761494524776936 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0020761494524776936 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:500.7109375 - response_length/max:8192.0 - response_length/min:209.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:500.7109375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:209.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:309.8125 - prompt_length/max:597.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.429105997085571e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4574673138558865) - timing_s/agent_loop/generate_sequences/max:np.float64(50.65919073112309) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.245039781228115) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.6591907311
(TaskRunner pid=2001144) Training Progress: 69%|██████▉ | 69/100 [47:09<21:36, 41.82s/it]
(TaskRunner pid=2001144) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70}
(TaskRunner pid=2001144) validation generation end
(TaskRunner pid=2001144) [prompt] system
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2001144) <reasoning>
(TaskRunner pid=2001144) ...
(TaskRunner pid=2001144) </reasoning>
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) ...
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) A
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) user
(TaskRunner pid=2001144) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2001144) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2001144) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2001144) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2001144) Please reason step by step.
(TaskRunner pid=2001144) assistant
(TaskRunner pid=2001144) <think>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) </think>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) [response] <reasoning>
(TaskRunner pid=2001144) The question asks which of the given options is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". This is a complex organic chemistry question involving the identification of a correct reactant that would lead to the formation of the given product through a synthesis reaction.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) To solve this, we need to analyze the structure of the product and compare it to the structures of the given options. The product has the following key features:
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) - It contains a nitrogen atom (N) that is bonded to a carbon chain (c1ccn...).
(TaskRunner pid=2001144) - The chain includes a substituent "Cc2ccc(Cl)cc2C(F)(F)F", which suggests a substituted aromatic ring with a chlorine atom and a trifluoromethyl group (C(F)(F)F).
(TaskRunner pid=2001144) - The overall structure appears to be a complex aromatic system with multiple substituents.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) We must determine which of the given options is the correct reactant that would lead to the formation of this product. The key is to look for a reactant that, when combined with other reagents, would produce the given product through a chemical reaction (such as a nucleophilic substitution, condensation, or other organic reaction).
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Option A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2001144) This structure appears to have a similar pattern to the product. It includes a substituted aromatic ring with a chlorine atom and a trifluoromethyl group, and it has a nitrogen atom that is part of a chain that could lead to the formation of the product. The structure also has a similar substitution pattern to the product.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Option B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2001144) This structure includes a chlorine atom but does not have the exact same substitution pattern as the product. It also has a different arrangement of atoms, suggesting it may not be the correct reactant.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Option C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2001144) This structure is more complex and includes additional substituents that are not present in the product. It has a different arrangement of atoms and may not be the correct reactant.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Option D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2001144) This structure includes a trifluoromethyl group (C(F)(F)F), which is present in the product, but the rest of the structure is different. It includes a different arrangement of atoms that may not lead to the product.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Based on the analysis, Option A has the most similar structure to the product, with the correct substitution pattern and the correct nitrogen-containing chain that would lead to the formation of the product. Therefore, Option A is the most likely correct reactant.
(TaskRunner pid=2001144) </reasoning>
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) A
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144) [ground_truth] A
(TaskRunner pid=2001144) [score] 1.0
(TaskRunner pid=2001144) [acc] 1.0
(TaskRunner pid=2001144) [pred] A
(TaskRunner pid=2001144) [incorrect_format] 1
(TaskRunner pid=2001144) [feedback]
(TaskRunner pid=2001144) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70
(WorkerDict pid=2001534) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60/actor
(TaskRunner pid=2001144) step:70 - global_seqlen/min:18104 - global_seqlen/max:26399 - global_seqlen/minmax_diff:8295 - global_seqlen/balanced_min:23391 - global_seqlen/balanced_max:23395 - global_seqlen/mean:23393.125 - actor/entropy:0.42802414298057556 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3654877245426178 - training/rollout_probs_diff_mean:0.00630423566326499 - training/rollout_probs_diff_std:0.012846443802118301 - training/rollout_actor_probs_pearson_corr:0.9988842606544495 - rollout_corr/rollout_is_mean:0.9998522400856018 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9838422536849976 - rollout_corr/rollout_is_min:0.5048040151596069 - rollout_corr/rollout_is_std:0.04270577058196068 - rollout_corr/rollout_is_eff_sample_size:0.9981790621057267 - rollout_corr/rollout_is_seq_mean:0.9998517632484436 - rollout_corr/rollout_is_seq_std:0.002017417922616005 - rollout_corr/rollout_is_seq_max:1.0052543878555298 - rollout_corr/rollout_is_seq_min:0.9946851134300232 - rollout_corr/rollout_is_seq_max_deviation:0.005314886569976807 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5495784217491746) - rollout_corr/training_log_ppl:np.float64(0.43369465495925397) - rollout_corr/kl:np.float64(0.0011770569458953162) - rollout_corr/k3_kl:np.float64(0.0010381349239878546) - rollout_corr/rollout_ppl:np.float64(1.5477001233957708) - rollout_corr/rollout_log_ppl:np.float64(0.432517594890669) - rollout_corr/log_ppl_diff:np.float64(0.0011770600685849786) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018837241223081946) - rollout_corr/log_ppl_diff_max:np.float64(0.009053826332092285) - rollout_corr/log_ppl_diff_min:np.float64(-0.004020482301712036) - rollout_corr/ppl_ratio:np.float64(1.001179899321869) - rollout_corr/chi2_token:np.float64(0.0017858666833490133) - rollout_corr/chi2_seq:np.float64(0.5765461842529476) - actor/pg_loss:np.float64(-4.352419637143612e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004967188915543375) - actor/ppo_kl:np.float64(9.898858171020208e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3318125456571579) - perf/mfu/actor:np.float64(0.06817271764642531) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(103.5211067199707) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.655952380952381) - val-aux/sciknoweval/reward/std@16:np.float64(0.15695247014761712) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7196761904761905) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.12247465419503888) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5915904761904762) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1436010885725026) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6552095238095238) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.15644144301978585) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7662190476190477) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.08164227177415044) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5342476190476191) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11521371656591234) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6674809523809524) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.12489757220027457) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7976809523809523) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.05089268879946206) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4876095238095239) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08463281214401285) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6733333333333333) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09193805280378953) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8167476190476191)
(TaskRunner pid=2001144) Training Progress: 70%|███████ | 70/100 [49:50<38:50, 77.68s/it]
(TaskRunner pid=2001144) step:71 - global_seqlen/min:23363 - global_seqlen/max:28746 - global_seqlen/minmax_diff:5383 - global_seqlen/balanced_min:25416 - global_seqlen/balanced_max:25690 - global_seqlen/mean:25459.0 - actor/entropy:0.41205909848213196 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4399867355823517 - training/rollout_probs_diff_mean:0.0059791067615151405 - training/rollout_probs_diff_std:0.012625901959836483 - training/rollout_actor_probs_pearson_corr:0.9988660216331482 - rollout_corr/rollout_is_mean:0.9998722672462463 - rollout_corr/rollout_is_ratio_fraction_high:2.3759741452522576e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.16796540573705e-05 - rollout_corr/rollout_is_max:2.2563412189483643 - rollout_corr/rollout_is_min:0.27345186471939087 - rollout_corr/rollout_is_std:0.04195384681224823 - rollout_corr/rollout_is_eff_sample_size:0.9982426110648863 - rollout_corr/rollout_is_seq_mean:0.9999057054519653 - rollout_corr/rollout_is_seq_std:0.00222775642760098 - rollout_corr/rollout_is_seq_max:1.0069891214370728 - rollout_corr/rollout_is_seq_min:0.9915140271186829 - rollout_corr/rollout_is_seq_max_deviation:0.008485972881317139 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5374808609485626) - rollout_corr/training_log_ppl:np.float64(0.4250078178301919) - rollout_corr/kl:np.float64(0.0010268305321957105) - rollout_corr/k3_kl:np.float64(0.0010821249153423196) - rollout_corr/rollout_ppl:np.float64(1.5358611489646137) - rollout_corr/rollout_log_ppl:np.float64(0.4239809854188934) - rollout_corr/log_ppl_diff:np.float64(0.0010268324112985283) - rollout_corr/log_ppl_abs_diff:np.float64(0.002006425493163988) - rollout_corr/log_ppl_diff_max:np.float64(0.008771628141403198) - rollout_corr/log_ppl_diff_min:np.float64(-0.008090883493423462) - rollout_corr/ppl_ratio:np.float64(1.0010302911978215) - rollout_corr/chi2_token:np.float64(0.0022360237780958414) - rollout_corr/chi2_seq:np.float64(1.4678814092185348) - actor/pg_loss:np.float64(-6.92977337166667e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005757826820627088) - actor/ppo_kl:np.float64(-1.249257383051372e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.39176852256059647) - perf/mfu/actor:np.float64(0.07347110174057103) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(107.28773880004883) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:1 - critic/score/mean:0.55078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004960835911333561 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004960835911333561 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:493.21875 - response_length/max:1859.0 - response_length/min:158.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:493.21875 - response_length_non_aborted/max:1859.0 - response_length_non_aborted/min:158.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.375 - prompt_length/max:666.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.5631622672080994e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.799711162224412) - timing_s/agent_loop/generate_sequences/max:np.float64(12.301090884953737) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.291331741893373) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.301090884953737) - timing_s/ag
(TaskRunner pid=2001144) Training Progress: 71%|███████ | 71/100 [50:21<30:44, 63.62s/it]
(TaskRunner pid=2001144) step:72 - global_seqlen/min:19579 - global_seqlen/max:27710 - global_seqlen/minmax_diff:8131 - global_seqlen/balanced_min:22748 - global_seqlen/balanced_max:22753 - global_seqlen/mean:22750.875 - actor/entropy:0.4111070930957794 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2310577630996704 - training/rollout_probs_diff_mean:0.00630597909912467 - training/rollout_probs_diff_std:0.012845057062804699 - training/rollout_actor_probs_pearson_corr:0.9988619685173035 - rollout_corr/rollout_is_mean:1.0000945329666138 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.605111826094799e-05 - rollout_corr/rollout_is_max:1.647740364074707 - rollout_corr/rollout_is_min:0.34073442220687866 - rollout_corr/rollout_is_std:0.043213579803705215 - rollout_corr/rollout_is_eff_sample_size:0.9981365422739324 - rollout_corr/rollout_is_seq_mean:1.0000654458999634 - rollout_corr/rollout_is_seq_std:0.0021228997502475977 - rollout_corr/rollout_is_seq_max:1.0063730478286743 - rollout_corr/rollout_is_seq_min:0.9926246404647827 - rollout_corr/rollout_is_seq_max_deviation:0.007375359535217285 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5242752027697861) - rollout_corr/training_log_ppl:np.float64(0.4166734802420251) - rollout_corr/kl:np.float64(0.0010505110755776315) - rollout_corr/k3_kl:np.float64(0.0010427682495901536) - rollout_corr/rollout_ppl:np.float64(1.5226748585700989) - rollout_corr/rollout_log_ppl:np.float64(0.41562296816846356) - rollout_corr/log_ppl_diff:np.float64(0.0010505120735615492) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020876229973509908) - rollout_corr/log_ppl_diff_max:np.float64(0.009741872549057007) - rollout_corr/log_ppl_diff_min:np.float64(-0.005568176507949829) - rollout_corr/ppl_ratio:np.float64(1.0010542022064328) - rollout_corr/chi2_token:np.float64(0.002088770968839526) - rollout_corr/chi2_seq:np.float64(1.6762113182339817) - actor/pg_loss:np.float64(-0.0001293893437832594) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005647189673254616) - actor/ppo_kl:np.float64(0.00015567482746803307) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3745649680495262) - perf/mfu/actor:np.float64(0.06614673674738844) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(106.29275608062744) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:1 - critic/score/mean:0.67578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.67578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0041008517146110535 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0041008517146110535 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:424.12109375 - response_length/max:894.0 - response_length/min:215.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:424.12109375 - response_length_non_aborted/max:894.0 - response_length_non_aborted/min:215.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.84375 - prompt_length/max:620.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010326690971851349 - timing_s/agent_loop/generate_sequences/min:np.float64(2.580171564593911) - timing_s/agent_loop/generate_sequences/max:np.float64(7.283241093158722) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.689614264600095) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.283241093158722) - timing_s/agent_loop/s
(TaskRunner pid=2001144) Training Progress: 72%|███████▏ | 72/100 [50:47<24:26, 52.36s/it]
(TaskRunner pid=2001144) step:73 - global_seqlen/min:17776 - global_seqlen/max:28136 - global_seqlen/minmax_diff:10360 - global_seqlen/balanced_min:23458 - global_seqlen/balanced_max:23463 - global_seqlen/mean:23461.375 - actor/entropy:0.4236791133880615 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5284976363182068 - training/rollout_probs_diff_mean:0.006063377019017935 - training/rollout_probs_diff_std:0.01252244133502245 - training/rollout_actor_probs_pearson_corr:0.9989043474197388 - rollout_corr/rollout_is_mean:1.0002351999282837 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.0368245613062754e-05 - rollout_corr/rollout_is_max:1.851340651512146 - rollout_corr/rollout_is_min:0.06283614784479141 - rollout_corr/rollout_is_std:0.04231035336852074 - rollout_corr/rollout_is_eff_sample_size:0.9982141021198494 - rollout_corr/rollout_is_seq_mean:1.0002738237380981 - rollout_corr/rollout_is_seq_std:0.002158844145014882 - rollout_corr/rollout_is_seq_max:1.0072818994522095 - rollout_corr/rollout_is_seq_min:0.9943947792053223 - rollout_corr/rollout_is_seq_max_deviation:0.007281899452209473 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5522524621337652) - rollout_corr/training_log_ppl:np.float64(0.43486891296925023) - rollout_corr/kl:np.float64(0.000694787611983827) - rollout_corr/k3_kl:np.float64(0.0009920408933794533) - rollout_corr/rollout_ppl:np.float64(1.5511141093447804) - rollout_corr/rollout_log_ppl:np.float64(0.434174123511184) - rollout_corr/log_ppl_diff:np.float64(0.000694789458066225) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018241442739963531) - rollout_corr/log_ppl_diff_max:np.float64(0.008612185716629028) - rollout_corr/log_ppl_diff_min:np.float64(-0.005452096462249756) - rollout_corr/ppl_ratio:np.float64(1.0006976823788136) - rollout_corr/chi2_token:np.float64(0.0025686074513942003) - rollout_corr/chi2_seq:np.float64(1.0989803588017821) - actor/pg_loss:np.float64(-5.969149060547352e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017682414363662247) - actor/ppo_kl:np.float64(2.0015023526553932e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.244550459086895) - perf/mfu/actor:np.float64(0.06775358442184806) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(105.96631622314453) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:1 - critic/score/mean:0.55078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009464878821745515 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009464878821745515 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:452.94921875 - response_length/max:890.0 - response_length/min:179.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:452.94921875 - response_length_non_aborted/max:890.0 - response_length_non_aborted/min:179.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.21875 - prompt_length/max:557.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.505210280418396e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1212684009224176) - timing_s/agent_loop/generate_sequences/max:np.float64(7.888029620051384) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.0011278946331) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.888029620051384) - timing_s/agent_loop/slowe
(TaskRunner pid=2001144) Training Progress: 73%|███████▎ | 73/100 [51:14<20:03, 44.59s/it]
(TaskRunner pid=2001144) step:74 - global_seqlen/min:19281 - global_seqlen/max:27401 - global_seqlen/minmax_diff:8120 - global_seqlen/balanced_min:23096 - global_seqlen/balanced_max:23101 - global_seqlen/mean:23099.0 - actor/entropy:0.43460193276405334 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24067646265029907 - training/rollout_probs_diff_mean:0.006116139702498913 - training/rollout_probs_diff_std:0.012313715182244778 - training/rollout_actor_probs_pearson_corr:0.9989598989486694 - rollout_corr/rollout_is_mean:0.999896228313446 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7955281734466553 - rollout_corr/rollout_is_min:0.5500364899635315 - rollout_corr/rollout_is_std:0.04148092120885849 - rollout_corr/rollout_is_eff_sample_size:0.9982819323233423 - rollout_corr/rollout_is_seq_mean:0.9999107122421265 - rollout_corr/rollout_is_seq_std:0.002067468361929059 - rollout_corr/rollout_is_seq_max:1.0066832304000854 - rollout_corr/rollout_is_seq_min:0.9935411214828491 - rollout_corr/rollout_is_seq_max_deviation:0.006683230400085449 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5679499474354088) - rollout_corr/training_log_ppl:np.float64(0.445471141661983) - rollout_corr/kl:np.float64(0.0009617310399985257) - rollout_corr/k3_kl:np.float64(0.0009378647765743153) - rollout_corr/rollout_ppl:np.float64(1.5664173467084765) - rollout_corr/rollout_log_ppl:np.float64(0.44450941152172163) - rollout_corr/log_ppl_diff:np.float64(0.0009617301402613521) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017801534850150347) - rollout_corr/log_ppl_diff_max:np.float64(0.007514506578445435) - rollout_corr/log_ppl_diff_min:np.float64(-0.004243016242980957) - rollout_corr/ppl_ratio:np.float64(1.0009645365644246) - rollout_corr/chi2_token:np.float64(0.0018200732301920652) - rollout_corr/chi2_seq:np.float64(0.6104079929646105) - actor/pg_loss:np.float64(-0.0001332067186012864) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00041085688872044557) - actor/ppo_kl:np.float64(-1.624915879716582e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3354508578777313) - perf/mfu/actor:np.float64(0.06757762827139512) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(105.5615005493164) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:1 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004418553784489632 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004418553784489632 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:448.4375 - response_length/max:1009.0 - response_length/min:193.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:448.4375 - response_length_non_aborted/max:1009.0 - response_length_non_aborted/min:193.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.40625 - prompt_length/max:665.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010411255061626434 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2857920173555613) - timing_s/agent_loop/generate_sequences/max:np.float64(8.085642512887716) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.898749945372401) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.085642512887716) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.
(TaskRunner pid=2001144) Training Progress: 74%|███████▍ | 74/100 [51:40<16:57, 39.12s/it]
(TaskRunner pid=2001144) step:75 - global_seqlen/min:19609 - global_seqlen/max:30276 - global_seqlen/minmax_diff:10667 - global_seqlen/balanced_min:25476 - global_seqlen/balanced_max:25483 - global_seqlen/mean:25480.375 - actor/entropy:0.3828934133052826 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7286418080329895 - training/rollout_probs_diff_mean:0.005869799293577671 - training/rollout_probs_diff_std:0.013156627304852009 - training/rollout_actor_probs_pearson_corr:0.9987536668777466 - rollout_corr/rollout_is_mean:1.0000144243240356 - rollout_corr/rollout_is_ratio_fraction_high:1.6180836610146798e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.85425116494298e-05 - rollout_corr/rollout_is_max:2.957324981689453 - rollout_corr/rollout_is_min:0.09351254999637604 - rollout_corr/rollout_is_std:0.04185142740607262 - rollout_corr/rollout_is_eff_sample_size:0.9982515204441683 - rollout_corr/rollout_is_seq_mean:0.9999915957450867 - rollout_corr/rollout_is_seq_std:0.0021712409798055887 - rollout_corr/rollout_is_seq_max:1.0068303346633911 - rollout_corr/rollout_is_seq_min:0.9898998141288757 - rollout_corr/rollout_is_seq_max_deviation:0.010100185871124268 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4937474867329001) - rollout_corr/training_log_ppl:np.float64(0.3955021771835163) - rollout_corr/kl:np.float64(0.0009730852594174166) - rollout_corr/k3_kl:np.float64(0.0010216976074843842) - rollout_corr/rollout_ppl:np.float64(1.49228285998106) - rollout_corr/rollout_log_ppl:np.float64(0.39452908863313496) - rollout_corr/log_ppl_diff:np.float64(0.0009730885503813624) - rollout_corr/log_ppl_abs_diff:np.float64(0.001967592630535364) - rollout_corr/log_ppl_diff_max:np.float64(0.009815573692321777) - rollout_corr/log_ppl_diff_min:np.float64(-0.005886584520339966) - rollout_corr/ppl_ratio:np.float64(1.0009764197748154) - rollout_corr/chi2_token:np.float64(0.002125905128195882) - rollout_corr/chi2_seq:np.float64(1.5159224581439048) - actor/pg_loss:np.float64(5.427328869700432e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002961760606012831) - actor/ppo_kl:np.float64(6.538084219442908e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.28953711688518524) - perf/mfu/actor:np.float64(0.07425570964686921) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(105.40571212768555) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:1 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00010618674423312768 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00010618674423312768 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:482.82421875 - response_length/max:940.0 - response_length/min:229.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:482.82421875 - response_length_non_aborted/max:940.0 - response_length_non_aborted/min:229.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.4375 - prompt_length/max:608.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016676262021064758 - timing_s/agent_loop/generate_sequences/min:np.float64(2.700253961607814) - timing_s/agent_loop/generate_sequences/max:np.float64(8.154316319152713) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.209555803849071) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.154316319152713) - timing_
(TaskRunner pid=2001144) Training Progress: 75%|███████▌ | 75/100 [52:07<14:45, 35.40s/it]
(TaskRunner pid=2001144) step:76 - global_seqlen/min:22094 - global_seqlen/max:25952 - global_seqlen/minmax_diff:3858 - global_seqlen/balanced_min:24545 - global_seqlen/balanced_max:24562 - global_seqlen/mean:24557.625 - actor/entropy:0.4123407006263733 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3002305328845978 - training/rollout_probs_diff_mean:0.005858258344233036 - training/rollout_probs_diff_std:0.012601342052221298 - training/rollout_actor_probs_pearson_corr:0.9988989233970642 - rollout_corr/rollout_is_mean:1.0000865459442139 - rollout_corr/rollout_is_ratio_fraction_high:8.108656402328052e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.432596738799475e-05 - rollout_corr/rollout_is_max:2.3734848499298096 - rollout_corr/rollout_is_min:0.23166567087173462 - rollout_corr/rollout_is_std:0.04177587851881981 - rollout_corr/rollout_is_eff_sample_size:0.9982580540900504 - rollout_corr/rollout_is_seq_mean:1.0000982284545898 - rollout_corr/rollout_is_seq_std:0.0021354020573198795 - rollout_corr/rollout_is_seq_max:1.0053281784057617 - rollout_corr/rollout_is_seq_min:0.993293285369873 - rollout_corr/rollout_is_seq_max_deviation:0.006706714630126953 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.544180324766785) - rollout_corr/training_log_ppl:np.float64(0.4273947725305334) - rollout_corr/kl:np.float64(0.001034639477527577) - rollout_corr/k3_kl:np.float64(0.0009760225999571048) - rollout_corr/rollout_ppl:np.float64(1.542579477187246) - rollout_corr/rollout_log_ppl:np.float64(0.42636013065930456) - rollout_corr/log_ppl_diff:np.float64(0.0010346418712288141) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019495849264785647) - rollout_corr/log_ppl_diff_max:np.float64(0.009195715188980103) - rollout_corr/log_ppl_diff_min:np.float64(-0.004431962966918945) - rollout_corr/ppl_ratio:np.float64(1.0010376404970884) - rollout_corr/chi2_token:np.float64(0.0018619457259774208) - rollout_corr/chi2_seq:np.float64(0.8393021856900305) - actor/pg_loss:np.float64(0.00011749705299735069) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002348042553421692) - actor/ppo_kl:np.float64(0.00020846147090480827) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2717743292450905) - perf/mfu/actor:np.float64(0.0718267903988465) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(105.42528915405273) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:1 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007677883841097355 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007677883841097355 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:481.73828125 - response_length/max:1312.0 - response_length/min:221.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:481.73828125 - response_length_non_aborted/max:1312.0 - response_length_non_aborted/min:221.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:285.6875 - prompt_length/max:526.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001557227224111557 - timing_s/agent_loop/generate_sequences/min:np.float64(2.653002245351672) - timing_s/agent_loop/generate_sequences/max:np.float64(9.793218195438385) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.188218843839422) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.793218195438385) - timing_s/agent_loop/slow
(TaskRunner pid=2001144) Training Progress: 76%|███████▌ | 76/100 [52:35<13:17, 33.21s/it]
(TaskRunner pid=2001144) step:77 - global_seqlen/min:20260 - global_seqlen/max:27131 - global_seqlen/minmax_diff:6871 - global_seqlen/balanced_min:23571 - global_seqlen/balanced_max:23576 - global_seqlen/mean:23574.25 - actor/entropy:0.3989763855934143 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967508316040039 - training/rollout_probs_diff_mean:0.006071285344660282 - training/rollout_probs_diff_std:0.012924023903906345 - training/rollout_actor_probs_pearson_corr:0.9988321661949158 - rollout_corr/rollout_is_mean:1.0001028776168823 - rollout_corr/rollout_is_ratio_fraction_high:9.30942678678548e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.65471348434221e-05 - rollout_corr/rollout_is_max:2.5484020709991455 - rollout_corr/rollout_is_min:0.42574936151504517 - rollout_corr/rollout_is_std:0.04270995780825615 - rollout_corr/rollout_is_eff_sample_size:0.9981796559837771 - rollout_corr/rollout_is_seq_mean:1.0000712871551514 - rollout_corr/rollout_is_seq_std:0.002302023349329829 - rollout_corr/rollout_is_seq_max:1.0060687065124512 - rollout_corr/rollout_is_seq_min:0.9926195740699768 - rollout_corr/rollout_is_seq_max_deviation:0.007380425930023193 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5235404535196722) - rollout_corr/training_log_ppl:np.float64(0.4153496529906988) - rollout_corr/kl:np.float64(0.0011430231546647462) - rollout_corr/k3_kl:np.float64(0.0009922952636998161) - rollout_corr/rollout_ppl:np.float64(1.5217806049622595) - rollout_corr/rollout_log_ppl:np.float64(0.4142066292697564) - rollout_corr/log_ppl_diff:np.float64(0.0011430237209424376) - rollout_corr/log_ppl_abs_diff:np.float64(0.002006533439271152) - rollout_corr/log_ppl_diff_max:np.float64(0.009556949138641357) - rollout_corr/log_ppl_diff_min:np.float64(-0.004853725433349609) - rollout_corr/ppl_ratio:np.float64(1.0011463174596429) - rollout_corr/chi2_token:np.float64(0.0016961859073489904) - rollout_corr/chi2_seq:np.float64(0.7233788173180073) - actor/pg_loss:np.float64(-5.829066503793001e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00026930487638310296) - actor/ppo_kl:np.float64(0.00028201621353751705) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2561480849981308) - perf/mfu/actor:np.float64(0.06982076645892168) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(105.42876434326172) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:1 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.000602785381488502 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.000602785381488502 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:419.6015625 - response_length/max:913.0 - response_length/min:206.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:419.6015625 - response_length_non_aborted/max:913.0 - response_length_non_aborted/min:206.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:317.09375 - prompt_length/max:716.0 - prompt_length/min:173.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014156103134155273 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4395600240677595) - timing_s/agent_loop/generate_sequences/max:np.float64(7.5798758417367935) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.47761472381535) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.5798758417367935) - timing_s/age
(TaskRunner pid=2001144) Training Progress: 77%|███████▋ | 77/100 [53:01<11:55, 31.10s/it]
(TaskRunner pid=2001144) step:78 - global_seqlen/min:20686 - global_seqlen/max:30044 - global_seqlen/minmax_diff:9358 - global_seqlen/balanced_min:24641 - global_seqlen/balanced_max:24655 - global_seqlen/mean:24648.0 - actor/entropy:0.41220811009407043 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8012882471084595 - training/rollout_probs_diff_mean:0.005713974125683308 - training/rollout_probs_diff_std:0.012194479815661907 - training/rollout_actor_probs_pearson_corr:0.9990101456642151 - rollout_corr/rollout_is_mean:1.0001120567321777 - rollout_corr/rollout_is_ratio_fraction_high:1.5921537851681933e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.9803846448194236e-05 - rollout_corr/rollout_is_max:4.229130268096924 - rollout_corr/rollout_is_min:0.1853894293308258 - rollout_corr/rollout_is_std:0.04114910215139389 - rollout_corr/rollout_is_eff_sample_size:0.9983099699160126 - rollout_corr/rollout_is_seq_mean:1.0001230239868164 - rollout_corr/rollout_is_seq_std:0.0019666876178234816 - rollout_corr/rollout_is_seq_max:1.0058064460754395 - rollout_corr/rollout_is_seq_min:0.9937610626220703 - rollout_corr/rollout_is_seq_max_deviation:0.0062389373779296875 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5428671552799642) - rollout_corr/training_log_ppl:np.float64(0.4284245790913701) - rollout_corr/kl:np.float64(0.0008091379673089705) - rollout_corr/k3_kl:np.float64(0.0009761353979911291) - rollout_corr/rollout_ppl:np.float64(1.541559505276382) - rollout_corr/rollout_log_ppl:np.float64(0.4276154397521168) - rollout_corr/log_ppl_diff:np.float64(0.0008091393392533064) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016639067325741053) - rollout_corr/log_ppl_diff_max:np.float64(0.007621109485626221) - rollout_corr/log_ppl_diff_min:np.float64(-0.0048393309116363525) - rollout_corr/ppl_ratio:np.float64(1.0008115584496409) - rollout_corr/chi2_token:np.float64(0.00240683532319963) - rollout_corr/chi2_seq:np.float64(0.8103806893341243) - actor/pg_loss:np.float64(-0.0002235656138509512) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005409726607012999) - actor/ppo_kl:np.float64(2.9511160425599314e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.30885864794254303) - perf/mfu/actor:np.float64(0.07228394537764868) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(103.64043807983398) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006118846125900745 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006118846125900745 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:490.6875 - response_length/max:1537.0 - response_length/min:215.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:490.6875 - response_length_non_aborted/max:1537.0 - response_length_non_aborted/min:215.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.5625 - prompt_length/max:449.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001364927738904953 - timing_s/agent_loop/generate_sequences/min:np.float64(2.5598331801593304) - timing_s/agent_loop/generate_sequences/max:np.float64(10.52706834860146) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.2438156525895465) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.52706834860146) - timing_s/agent_
(TaskRunner pid=2001144) Training Progress: 78%|███████▊ | 78/100 [53:30<11:09, 30.44s/it]
(TaskRunner pid=2001144) step:79 - global_seqlen/min:17622 - global_seqlen/max:26760 - global_seqlen/minmax_diff:9138 - global_seqlen/balanced_min:23141 - global_seqlen/balanced_max:23208 - global_seqlen/mean:23155.375 - actor/entropy:0.4080638289451599 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3705262839794159 - training/rollout_probs_diff_mean:0.005767693277448416 - training/rollout_probs_diff_std:0.0124347610399127 - training/rollout_actor_probs_pearson_corr:0.9989129900932312 - rollout_corr/rollout_is_mean:1.000084400177002 - rollout_corr/rollout_is_ratio_fraction_high:8.496826012560632e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.398730405024253e-05 - rollout_corr/rollout_is_max:2.1021571159362793 - rollout_corr/rollout_is_min:0.3637278974056244 - rollout_corr/rollout_is_std:0.040735650807619095 - rollout_corr/rollout_is_eff_sample_size:0.9983437121579525 - rollout_corr/rollout_is_seq_mean:1.000071406364441 - rollout_corr/rollout_is_seq_std:0.002001135842874646 - rollout_corr/rollout_is_seq_max:1.0059212446212769 - rollout_corr/rollout_is_seq_min:0.9941735863685608 - rollout_corr/rollout_is_seq_max_deviation:0.0059212446212768555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5331762479618192) - rollout_corr/training_log_ppl:np.float64(0.4227583435131237) - rollout_corr/kl:np.float64(0.0009363071688106572) - rollout_corr/k3_kl:np.float64(0.0009690592393098996) - rollout_corr/rollout_ppl:np.float64(1.5317258117720485) - rollout_corr/rollout_log_ppl:np.float64(0.4218220323091373) - rollout_corr/log_ppl_diff:np.float64(0.0009363112039864063) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018450788920745254) - rollout_corr/log_ppl_diff_max:np.float64(0.007291913032531738) - rollout_corr/log_ppl_diff_min:np.float64(-0.005144238471984863) - rollout_corr/ppl_ratio:np.float64(1.0009391147177666) - rollout_corr/chi2_token:np.float64(0.0020212128292769194) - rollout_corr/chi2_seq:np.float64(0.6943999424111098) - actor/pg_loss:np.float64(0.0001222133869305253) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007869038299759268) - actor/ppo_kl:np.float64(0.00012776638482847602) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.406066857278347) - perf/mfu/actor:np.float64(0.0675828366328014) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(103.5647087097168) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:1 - critic/score/mean:0.55078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009766039438545704 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009766039438545704 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:459.73046875 - response_length/max:1689.0 - response_length/min:222.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:459.73046875 - response_length_non_aborted/max:1689.0 - response_length_non_aborted/min:222.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.875 - prompt_length/max:766.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010993517935276031 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6599212754517794) - timing_s/agent_loop/generate_sequences/max:np.float64(11.448845503851771) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.871901929072919) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.448845503851771) - timing_s/
(TaskRunner pid=2001144) Training Progress: 79%|███████▉ | 79/100 [54:00<10:34, 30.24s/it]
(TaskRunner pid=2001144) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80}
(TaskRunner pid=2001144) validation generation end
(TaskRunner pid=2001144) [prompt] system
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2001144) <reasoning>
(TaskRunner pid=2001144) ...
(TaskRunner pid=2001144) </reasoning>
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) ...
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) A
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) user
(TaskRunner pid=2001144) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2001144) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2001144) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2001144) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2001144) Please reason step by step.
(TaskRunner pid=2001144) assistant
(TaskRunner pid=2001144) <think>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) </think>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) [response] <reasoning>
(TaskRunner pid=2001144) The question asks which of the given options is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". To determine this, we need to analyze the structure of the product and compare it with the structures of the given options to identify which one is a direct precursor or reactant in the synthesis of the product.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) The product is:
(TaskRunner pid=2001144) **Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1**
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) This is a complex organic molecule with a nitrogen atom connected to a chain that includes a substituted aromatic ring (with a Cl and a trifluoromethyl group, C(F)(F)F). The structure suggests that the molecule is derived from a reaction involving a substituted aromatic ring and a nitrogen-containing fragment.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Now let's analyze each option:
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) **Option A**:
(TaskRunner pid=2001144) O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) This structure is very similar to the product. It has a similar aromatic ring (c2ccccc2), a nitrogen (N1), and a chain that includes the same substituted aromatic ring (Cc2ccc(Cl)cc2C(F)(F)F). The only difference is the presence of a carbonyl group (O=C1) and a chain of carbon atoms (c2ccccc2). However, this appears to be a possible precursor to the product, as it has the same key substituents and the same nitrogen-containing chain.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) **Option B**:
(TaskRunner pid=2001144) O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) This structure has a Cl atom on the aromatic ring, but the rest of the structure is different from the product. It has a different chain and does not match the structure of the product, especially the part involving the C(F)(F)F group and the nitrogen-containing chain.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) **Option C**:
(TaskRunner pid=2001144) O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) This option has a more complex structure with a branching chain and additional substituents. The part involving the C(F)(F)F group is present, but the rest of the structure is not the same as the product. It is not a direct precursor to the product.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) **Option D**:
(TaskRunner pid=2001144) O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) This option has a different structure with a different chain and substituents. It does not match the structure of the product in the key parts, especially the nitrogen-containing chain and the specific arrangement of the aromatic ring.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Based on the analysis, **Option A** is the only one that matches the structure of the product closely, suggesting that it is the correct reactant used in the synthesis of the product.
(TaskRunner pid=2001144) </reasoning>
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) A
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144) [ground_truth] A
(TaskRunner pid=2001144) [score] 1.0
(TaskRunner pid=2001144) [acc] 1.0
(TaskRunner pid=2001144) [pred] A
(TaskRunner pid=2001144) [incorrect_format] 1
(TaskRunner pid=2001144) [feedback]
(TaskRunner pid=2001144) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80
(WorkerDict pid=2001534) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70/actor
(TaskRunner pid=2001144) step:80 - global_seqlen/min:19995 - global_seqlen/max:28879 - global_seqlen/minmax_diff:8884 - global_seqlen/balanced_min:24758 - global_seqlen/balanced_max:24762 - global_seqlen/mean:24760.625 - actor/entropy:0.41400912404060364 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9731078743934631 - training/rollout_probs_diff_mean:0.005834315437823534 - training/rollout_probs_diff_std:0.01270891074091196 - training/rollout_actor_probs_pearson_corr:0.9989573359489441 - rollout_corr/rollout_is_mean:0.9999210238456726 - rollout_corr/rollout_is_ratio_fraction_high:1.6980668078758754e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.396133615751751e-05 - rollout_corr/rollout_is_max:3.2528841495513916 - rollout_corr/rollout_is_min:0.007165858056396246 - rollout_corr/rollout_is_std:0.041938215494155884 - rollout_corr/rollout_is_eff_sample_size:0.9982441553459014 - rollout_corr/rollout_is_seq_mean:0.9999313950538635 - rollout_corr/rollout_is_seq_std:0.0019047128735110164 - rollout_corr/rollout_is_seq_max:1.0056259632110596 - rollout_corr/rollout_is_seq_min:0.9926962852478027 - rollout_corr/rollout_is_seq_max_deviation:0.007303714752197266 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5427662800066173) - rollout_corr/training_log_ppl:np.float64(0.42582694796146825) - rollout_corr/kl:np.float64(0.0008912713598034916) - rollout_corr/k3_kl:np.float64(0.0011206881810039704) - rollout_corr/rollout_ppl:np.float64(1.541378607507795) - rollout_corr/rollout_log_ppl:np.float64(0.42493567589554004) - rollout_corr/log_ppl_diff:np.float64(0.0008912720659282058) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017067085427697748) - rollout_corr/log_ppl_diff_max:np.float64(0.00971338152885437) - rollout_corr/log_ppl_diff_min:np.float64(-0.004646748304367065) - rollout_corr/ppl_ratio:np.float64(1.000893758609891) - rollout_corr/chi2_token:np.float64(0.0064836349338293076) - rollout_corr/chi2_seq:np.float64(2.9489730645436794) - actor/pg_loss:np.float64(-2.6987865567207336e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000421825099238049) - actor/ppo_kl:np.float64(-0.0001140698974140264) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5148773342370987) - perf/mfu/actor:np.float64(0.07220279762952235) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(103.62097120285034) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6657738095238095) - val-aux/sciknoweval/reward/std@16:np.float64(0.1547563414602757) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7263333333333334) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.11046545385260208) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6054190476190475) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14916365964646325) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6658142857142858) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.15443354240914273) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7655190476190477) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.07149819702971767) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.545947619047619) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.13215596549059974) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6820428571428571) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.11629399188161595) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.791947619047619) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.044963658187852404) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.49119047619047623) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10780068597712135) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6908095238095238) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07981707145179656) - val-aux/sciknoweval/reward
(TaskRunner pid=2001144) Training Progress: 80%|████████ | 80/100 [56:42<23:15, 69.79s/it]
(TaskRunner pid=2001144) step:81 - global_seqlen/min:17214 - global_seqlen/max:31331 - global_seqlen/minmax_diff:14117 - global_seqlen/balanced_min:24395 - global_seqlen/balanced_max:24406 - global_seqlen/mean:24401.75 - actor/entropy:0.411903977394104 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9516690969467163 - training/rollout_probs_diff_mean:0.0057448106817901134 - training/rollout_probs_diff_std:0.013151517137885094 - training/rollout_actor_probs_pearson_corr:0.9988468289375305 - rollout_corr/rollout_is_mean:1.000136137008667 - rollout_corr/rollout_is_ratio_fraction_high:8.588851414970122e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.588851778768003e-05 - rollout_corr/rollout_is_max:2.09285306930542 - rollout_corr/rollout_is_min:0.04745984077453613 - rollout_corr/rollout_is_std:0.04168875515460968 - rollout_corr/rollout_is_eff_sample_size:0.998265538189517 - rollout_corr/rollout_is_seq_mean:1.0001261234283447 - rollout_corr/rollout_is_seq_std:0.002119193784892559 - rollout_corr/rollout_is_seq_max:1.007901668548584 - rollout_corr/rollout_is_seq_min:0.9942180514335632 - rollout_corr/rollout_is_seq_max_deviation:0.007901668548583984 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.546320866793394) - rollout_corr/training_log_ppl:np.float64(0.4296358540887013) - rollout_corr/kl:np.float64(0.0010130558077667828) - rollout_corr/k3_kl:np.float64(0.0009829288060245744) - rollout_corr/rollout_ppl:np.float64(1.54475741321221) - rollout_corr/rollout_log_ppl:np.float64(0.4286227965494618) - rollout_corr/log_ppl_diff:np.float64(0.0010130575392395258) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019264344591647387) - rollout_corr/log_ppl_diff_max:np.float64(0.008218944072723389) - rollout_corr/log_ppl_diff_min:np.float64(-0.005005776882171631) - rollout_corr/ppl_ratio:np.float64(1.001016051042825) - rollout_corr/chi2_token:np.float64(0.0019054301083087921) - rollout_corr/chi2_seq:np.float64(1.4213474048301578) - actor/pg_loss:np.float64(-7.377227302640676e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005340900440842233) - actor/ppo_kl:np.float64(0.00020235588424455386) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6206544786691666) - perf/mfu/actor:np.float64(0.07075054245518711) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(106.29306030273438) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:1 - critic/score/mean:0.57421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.57421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.000458429945865646 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.000458429945865646 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:454.8046875 - response_length/max:1236.0 - response_length/min:202.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:454.8046875 - response_length_non_aborted/max:1236.0 - response_length_non_aborted/min:202.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:307.75 - prompt_length/max:735.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.716085433959961e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3671316374093294) - timing_s/agent_loop/generate_sequences/max:np.float64(9.199800789356232) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.861064994220214) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.199800789356232) - timing_s/agent_lo
(TaskRunner pid=2001144) Training Progress: 81%|████████ | 81/100 [57:10<18:06, 57.21s/it]
(TaskRunner pid=2001144) step:82 - global_seqlen/min:18387 - global_seqlen/max:29825 - global_seqlen/minmax_diff:11438 - global_seqlen/balanced_min:23682 - global_seqlen/balanced_max:23800 - global_seqlen/mean:23727.0 - actor/entropy:0.41564303636550903 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44002431631088257 - training/rollout_probs_diff_mean:0.00550758745521307 - training/rollout_probs_diff_std:0.011829081922769547 - training/rollout_actor_probs_pearson_corr:0.9990654587745667 - rollout_corr/rollout_is_mean:0.9998658895492554 - rollout_corr/rollout_is_ratio_fraction_high:8.216657079174183e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.6433314158348367e-05 - rollout_corr/rollout_is_max:2.5243916511535645 - rollout_corr/rollout_is_min:0.35559791326522827 - rollout_corr/rollout_is_std:0.0394209623336792 - rollout_corr/rollout_is_eff_sample_size:0.9984480423897838 - rollout_corr/rollout_is_seq_mean:0.9999570846557617 - rollout_corr/rollout_is_seq_std:0.0021330828312784433 - rollout_corr/rollout_is_seq_max:1.009461760520935 - rollout_corr/rollout_is_seq_min:0.9942116737365723 - rollout_corr/rollout_is_seq_max_deviation:0.009461760520935059 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5534576666541398) - rollout_corr/training_log_ppl:np.float64(0.43340867588995025) - rollout_corr/kl:np.float64(0.0009325072234531717) - rollout_corr/k3_kl:np.float64(0.0009051784624034553) - rollout_corr/rollout_ppl:np.float64(1.5519792907871306) - rollout_corr/rollout_log_ppl:np.float64(0.43247616826556623) - rollout_corr/log_ppl_diff:np.float64(0.0009325076243840158) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019120039069093764) - rollout_corr/log_ppl_diff_max:np.float64(0.006664097309112549) - rollout_corr/log_ppl_diff_min:np.float64(-0.006185948848724365) - rollout_corr/ppl_ratio:np.float64(1.000935405259952) - rollout_corr/chi2_token:np.float64(0.001768690999597311) - rollout_corr/chi2_seq:np.float64(1.1733228489756584) - actor/pg_loss:np.float64(-5.0893635489046574e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003691550273288158) - actor/ppo_kl:np.float64(6.810807569479493e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.34256511926651) - perf/mfu/actor:np.float64(0.06966690166491128) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(106.03290557861328) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:1 - critic/score/mean:0.5078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012129839742556214 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0012129839742556214 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:475.40625 - response_length/max:1725.0 - response_length/min:202.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:475.40625 - response_length_non_aborted/max:1725.0 - response_length_non_aborted/min:202.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.0625 - prompt_length/max:408.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010274350643157959 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4269375298172235) - timing_s/agent_loop/generate_sequences/max:np.float64(11.904329400509596) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.966502552924794) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.904329400509596) - timing_s/ag
(TaskRunner pid=2001144) Training Progress: 82%|████████▏ | 82/100 [57:40<14:45, 49.18s/it]
(TaskRunner pid=2001144) step:83 - global_seqlen/min:19366 - global_seqlen/max:31899 - global_seqlen/minmax_diff:12533 - global_seqlen/balanced_min:23151 - global_seqlen/balanced_max:29832 - global_seqlen/mean:23990.625 - actor/entropy:0.4063042998313904 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4490484297275543 - training/rollout_probs_diff_mean:0.005610758904367685 - training/rollout_probs_diff_std:0.012151612900197506 - training/rollout_actor_probs_pearson_corr:0.9989926815032959 - rollout_corr/rollout_is_mean:1.000045657157898 - rollout_corr/rollout_is_ratio_fraction_high:3.4660544770304114e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.599540675873868e-05 - rollout_corr/rollout_is_max:3.0771634578704834 - rollout_corr/rollout_is_min:0.3347010314464569 - rollout_corr/rollout_is_std:0.04088171198964119 - rollout_corr/rollout_is_eff_sample_size:0.9983315932029352 - rollout_corr/rollout_is_seq_mean:0.9999570846557617 - rollout_corr/rollout_is_seq_std:0.002269598888233304 - rollout_corr/rollout_is_seq_max:1.0054603815078735 - rollout_corr/rollout_is_seq_min:0.9929542541503906 - rollout_corr/rollout_is_seq_max_deviation:0.007045745849609375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.575851984322071) - rollout_corr/training_log_ppl:np.float64(0.44891759139864007) - rollout_corr/kl:np.float64(0.0011061883358536306) - rollout_corr/k3_kl:np.float64(0.0010375906503270471) - rollout_corr/rollout_ppl:np.float64(1.5740382866933942) - rollout_corr/rollout_log_ppl:np.float64(0.44781140143459197) - rollout_corr/log_ppl_diff:np.float64(0.0011061899640480988) - rollout_corr/log_ppl_abs_diff:np.float64(0.002166399201087188) - rollout_corr/log_ppl_diff_max:np.float64(0.007969856262207031) - rollout_corr/log_ppl_diff_min:np.float64(-0.005478501319885254) - rollout_corr/ppl_ratio:np.float64(1.0011101472191513) - rollout_corr/chi2_token:np.float64(0.0019627013243734837) - rollout_corr/chi2_seq:np.float64(1.2461105533875525) - actor/pg_loss:np.float64(1.152348704636097e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007317227059502329) - actor/ppo_kl:np.float64(0.00014919286657999464) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3714331202208996) - perf/mfu/actor:np.float64(0.06610067515193886) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(104.0905532836914) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:1 - critic/score/mean:0.46875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00023720809258520603 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00023720809258520603 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:450.80078125 - response_length/max:8192.0 - response_length/min:193.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:450.80078125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:193.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:298.90625 - prompt_length/max:563.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.326264262199402e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.328967448323965) - timing_s/agent_loop/generate_sequences/max:np.float64(50.681706136092544) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.644793929379375) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.6817061360925
(TaskRunner pid=2001144) Training Progress: 83%|████████▎ | 83/100 [58:50<15:43, 55.48s/it]
(TaskRunner pid=2001144) step:84 - global_seqlen/min:18257 - global_seqlen/max:26181 - global_seqlen/minmax_diff:7924 - global_seqlen/balanced_min:21324 - global_seqlen/balanced_max:21331 - global_seqlen/mean:21328.5 - actor/entropy:0.39475008845329285 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4097302258014679 - training/rollout_probs_diff_mean:0.005772678647190332 - training/rollout_probs_diff_std:0.012423577718436718 - training/rollout_actor_probs_pearson_corr:0.9989161491394043 - rollout_corr/rollout_is_mean:0.9998981952667236 - rollout_corr/rollout_is_ratio_fraction_high:1.8940110749099404e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.8940110749099404e-05 - rollout_corr/rollout_is_max:2.700134038925171 - rollout_corr/rollout_is_min:0.294344037771225 - rollout_corr/rollout_is_std:0.04123303294181824 - rollout_corr/rollout_is_eff_sample_size:0.9983022475405439 - rollout_corr/rollout_is_seq_mean:0.9998761415481567 - rollout_corr/rollout_is_seq_std:0.0024779951199889183 - rollout_corr/rollout_is_seq_max:1.0055348873138428 - rollout_corr/rollout_is_seq_min:0.9913337826728821 - rollout_corr/rollout_is_seq_max_deviation:0.00866621732711792 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.498114536050707) - rollout_corr/training_log_ppl:np.float64(0.4002968348795548) - rollout_corr/kl:np.float64(0.001111937762448889) - rollout_corr/k3_kl:np.float64(0.0010363085725657584) - rollout_corr/rollout_ppl:np.float64(1.496427730191499) - rollout_corr/rollout_log_ppl:np.float64(0.39918489579577) - rollout_corr/log_ppl_diff:np.float64(0.0011119390837848186) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022091150749474764) - rollout_corr/log_ppl_diff_max:np.float64(0.008591175079345703) - rollout_corr/log_ppl_diff_min:np.float64(-0.007465779781341553) - rollout_corr/ppl_ratio:np.float64(1.0011159556452185) - rollout_corr/chi2_token:np.float64(0.0019986939150840044) - rollout_corr/chi2_seq:np.float64(0.747325667180121) - actor/pg_loss:np.float64(-0.00014323822688311338) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005684766506419692) - actor/ppo_kl:np.float64(0.00010532644228078425) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.39897775650024414) - perf/mfu/actor:np.float64(0.06240462744178239) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(103.93211364746094) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:1 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005057009868323803 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005057009868323803 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:412.484375 - response_length/max:1074.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:412.484375 - response_length_non_aborted/max:1074.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.03125 - prompt_length/max:466.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013117864727973938 - timing_s/agent_loop/generate_sequences/min:np.float64(1.73036003485322) - timing_s/agent_loop/generate_sequences/max:np.float64(8.208134315907955) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.368914880717057) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.208134315907955) - timing_s/agent_l
(TaskRunner pid=2001144) Training Progress: 84%|████████▍ | 84/100 [59:17<12:29, 46.82s/it]
(TaskRunner pid=2001144) step:85 - global_seqlen/min:19786 - global_seqlen/max:29440 - global_seqlen/minmax_diff:9654 - global_seqlen/balanced_min:23939 - global_seqlen/balanced_max:23948 - global_seqlen/mean:23944.125 - actor/entropy:0.44731348752975464 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37975066900253296 - training/rollout_probs_diff_mean:0.005546091590076685 - training/rollout_probs_diff_std:0.011556282639503479 - training/rollout_actor_probs_pearson_corr:0.9991078972816467 - rollout_corr/rollout_is_mean:0.9999495148658752 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.6132176571991295e-05 - rollout_corr/rollout_is_max:1.753438115119934 - rollout_corr/rollout_is_min:0.42437079548835754 - rollout_corr/rollout_is_std:0.03872227296233177 - rollout_corr/rollout_is_eff_sample_size:0.9985025927285522 - rollout_corr/rollout_is_seq_mean:0.9999096989631653 - rollout_corr/rollout_is_seq_std:0.0019852789118885994 - rollout_corr/rollout_is_seq_max:1.006288766860962 - rollout_corr/rollout_is_seq_min:0.9948282837867737 - rollout_corr/rollout_is_seq_max_deviation:0.006288766860961914 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.579591211862862) - rollout_corr/training_log_ppl:np.float64(0.45078226953046396) - rollout_corr/kl:np.float64(0.0008846460639073683) - rollout_corr/k3_kl:np.float64(0.0008941572496041772) - rollout_corr/rollout_ppl:np.float64(1.5781497955322266) - rollout_corr/rollout_log_ppl:np.float64(0.4498976224567741) - rollout_corr/log_ppl_diff:np.float64(0.0008846470736898482) - rollout_corr/log_ppl_abs_diff:np.float64(0.001859337615314871) - rollout_corr/log_ppl_diff_max:np.float64(0.00898560881614685) - rollout_corr/log_ppl_diff_min:np.float64(-0.0047689080238342285) - rollout_corr/ppl_ratio:np.float64(1.0008874624036252) - rollout_corr/chi2_token:np.float64(0.0017990418709814548) - rollout_corr/chi2_seq:np.float64(1.3523820703849196) - actor/pg_loss:np.float64(-5.421251989901066e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022966621008890797) - actor/ppo_kl:np.float64(-3.478579903237744e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.30529405549168587) - perf/mfu/actor:np.float64(0.07045664168760558) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(103.71277618408203) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:1 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011766012758016586 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011766012758016586 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:448.44140625 - response_length/max:1209.0 - response_length/min:169.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:448.44140625 - response_length_non_aborted/max:1209.0 - response_length_non_aborted/min:169.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.8125 - prompt_length/max:706.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001437794417142868 - timing_s/agent_loop/generate_sequences/min:np.float64(2.024157227948308) - timing_s/agent_loop/generate_sequences/max:np.float64(8.984660109505057) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.6237742709126906) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.984660109505057) - timing_s/agent_loop/slowe
(TaskRunner pid=2001144) Training Progress: 85%|████████▌ | 85/100 [59:44<10:13, 40.92s/it]
(TaskRunner pid=2001144) step:86 - global_seqlen/min:18111 - global_seqlen/max:27878 - global_seqlen/minmax_diff:9767 - global_seqlen/balanced_min:21429 - global_seqlen/balanced_max:21434 - global_seqlen/mean:21431.875 - actor/entropy:0.4335083067417145 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3201632797718048 - training/rollout_probs_diff_mean:0.005873966030776501 - training/rollout_probs_diff_std:0.012315038591623306 - training/rollout_actor_probs_pearson_corr:0.9990165829658508 - rollout_corr/rollout_is_mean:1.000231385231018 - rollout_corr/rollout_is_ratio_fraction_high:9.955499081115704e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.9821996324462816e-05 - rollout_corr/rollout_is_max:2.2062063217163086 - rollout_corr/rollout_is_min:0.015417049638926983 - rollout_corr/rollout_is_std:0.04097054898738861 - rollout_corr/rollout_is_eff_sample_size:0.9983250585943869 - rollout_corr/rollout_is_seq_mean:1.0002015829086304 - rollout_corr/rollout_is_seq_std:0.0023343891371041536 - rollout_corr/rollout_is_seq_max:1.008996844291687 - rollout_corr/rollout_is_seq_min:0.9928853511810303 - rollout_corr/rollout_is_seq_max_deviation:0.008996844291687012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5678474153392017) - rollout_corr/training_log_ppl:np.float64(0.4431306371698156) - rollout_corr/kl:np.float64(0.000815108983079682) - rollout_corr/k3_kl:np.float64(0.0009139451163946433) - rollout_corr/rollout_ppl:np.float64(1.5665374086238444) - rollout_corr/rollout_log_ppl:np.float64(0.4423155255499296) - rollout_corr/log_ppl_diff:np.float64(0.0008151116198860109) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019687291351146996) - rollout_corr/log_ppl_diff_max:np.float64(0.007799625396728516) - rollout_corr/log_ppl_diff_min:np.float64(-0.005343198776245117) - rollout_corr/ppl_ratio:np.float64(1.0008181473240256) - rollout_corr/chi2_token:np.float64(0.0020186675246804953) - rollout_corr/chi2_seq:np.float64(0.9531310864258558) - actor/pg_loss:np.float64(-0.0002085904125124216) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005678822931258765) - actor/ppo_kl:np.float64(0.00014307813706082584) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3813612535595894) - perf/mfu/actor:np.float64(0.0630616739645436) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(103.77202987670898) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:1 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006554451771080494 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006554451771080494 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:392.37109375 - response_length/max:994.0 - response_length/min:156.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:392.37109375 - response_length_non_aborted/max:994.0 - response_length_non_aborted/min:156.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.375 - prompt_length/max:503.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011118687689304352 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7090869843959808) - timing_s/agent_loop/generate_sequences/max:np.float64(7.438317820429802) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.064897694450337) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.438317820429802) - timing_
(TaskRunner pid=2001144) Training Progress: 86%|████████▌ | 86/100 [1:00:10<08:30, 36.47s/it]
(TaskRunner pid=2001144) step:87 - global_seqlen/min:20852 - global_seqlen/max:28667 - global_seqlen/minmax_diff:7815 - global_seqlen/balanced_min:22647 - global_seqlen/balanced_max:23125 - global_seqlen/mean:22715.375 - actor/entropy:0.45559191703796387 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2939149737358093 - training/rollout_probs_diff_mean:0.0058854687958955765 - training/rollout_probs_diff_std:0.012017061933875084 - training/rollout_actor_probs_pearson_corr:0.999066174030304 - rollout_corr/rollout_is_mean:1.0001784563064575 - rollout_corr/rollout_is_ratio_fraction_high:9.426581527804956e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.426581527804956e-06 - rollout_corr/rollout_is_max:2.7162554264068604 - rollout_corr/rollout_is_min:0.4282951056957245 - rollout_corr/rollout_is_std:0.04057878628373146 - rollout_corr/rollout_is_eff_sample_size:0.9983566631311757 - rollout_corr/rollout_is_seq_mean:1.0001453161239624 - rollout_corr/rollout_is_seq_std:0.002258741995319724 - rollout_corr/rollout_is_seq_max:1.0086182355880737 - rollout_corr/rollout_is_seq_min:0.991485595703125 - rollout_corr/rollout_is_seq_max_deviation:0.00861823558807373 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5985011835582554) - rollout_corr/training_log_ppl:np.float64(0.463498319266364) - rollout_corr/kl:np.float64(0.000988110573959311) - rollout_corr/k3_kl:np.float64(0.001016859462879438) - rollout_corr/rollout_ppl:np.float64(1.5968896732665598) - rollout_corr/rollout_log_ppl:np.float64(0.46251020778436214) - rollout_corr/log_ppl_diff:np.float64(0.000988111482001841) - rollout_corr/log_ppl_abs_diff:np.float64(0.002121948986314237) - rollout_corr/log_ppl_diff_max:np.float64(0.009548842906951904) - rollout_corr/log_ppl_diff_min:np.float64(-0.005356699228286743) - rollout_corr/ppl_ratio:np.float64(1.0009918422438204) - rollout_corr/chi2_token:np.float64(0.002101767575368285) - rollout_corr/chi2_seq:np.float64(2.397576132323593) - actor/pg_loss:np.float64(0.00023399945348501205) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006639745015490917) - actor/ppo_kl:np.float64(0.0002603144333157559) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.39275551587343216) - perf/mfu/actor:np.float64(0.06633563003961235) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(103.89756393432617) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:1 - critic/score/mean:0.45703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.45703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0044941226951777935 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0044941226951777935 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:414.38671875 - response_length/max:1987.0 - response_length/min:195.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:414.38671875 - response_length_non_aborted/max:1987.0 - response_length_non_aborted/min:195.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.46875 - prompt_length/max:845.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.225014269351959e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3614048715680838) - timing_s/agent_loop/generate_sequences/max:np.float64(12.320071894675493) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.414744761634211) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.320071894675493) - timing_s/
(TaskRunner pid=2001144) Training Progress: 87%|████████▋ | 87/100 [1:00:41<07:31, 34.75s/it]
(TaskRunner pid=2001144) step:88 - global_seqlen/min:19581 - global_seqlen/max:25075 - global_seqlen/minmax_diff:5494 - global_seqlen/balanced_min:21216 - global_seqlen/balanced_max:21291 - global_seqlen/mean:21234.75 - actor/entropy:0.4249110817909241 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102376401424408 - training/rollout_probs_diff_mean:0.0056200479157269 - training/rollout_probs_diff_std:0.011939858086407185 - training/rollout_actor_probs_pearson_corr:0.9990661144256592 - rollout_corr/rollout_is_mean:1.000091314315796 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.978983164008241e-05 - rollout_corr/rollout_is_max:1.9390289783477783 - rollout_corr/rollout_is_min:0.3959798514842987 - rollout_corr/rollout_is_std:0.039931733161211014 - rollout_corr/rollout_is_eff_sample_size:0.9984083515542347 - rollout_corr/rollout_is_seq_mean:1.000091552734375 - rollout_corr/rollout_is_seq_std:0.002440214855596423 - rollout_corr/rollout_is_seq_max:1.0066535472869873 - rollout_corr/rollout_is_seq_min:0.9942281246185303 - rollout_corr/rollout_is_seq_max_deviation:0.006653547286987305 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5667019374668598) - rollout_corr/training_log_ppl:np.float64(0.44311231054598466) - rollout_corr/kl:np.float64(0.0009632572312519017) - rollout_corr/k3_kl:np.float64(0.0009601595706953958) - rollout_corr/rollout_ppl:np.float64(1.565158303361386) - rollout_corr/rollout_log_ppl:np.float64(0.4421490548993461) - rollout_corr/log_ppl_diff:np.float64(0.0009632556466385722) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020528037566691637) - rollout_corr/log_ppl_diff_max:np.float64(0.007056713104248047) - rollout_corr/log_ppl_diff_min:np.float64(-0.005707859992980957) - rollout_corr/ppl_ratio:np.float64(1.0009666790720075) - rollout_corr/chi2_token:np.float64(0.0019129414577037096) - rollout_corr/chi2_seq:np.float64(1.3019321039319038) - actor/pg_loss:np.float64(8.734781295061111e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005212269857111096) - actor/ppo_kl:np.float64(0.00022033117137221225) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3579503148794174) - perf/mfu/actor:np.float64(0.06245543674564276) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(103.91491317749023) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:1 - critic/score/mean:0.546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0003240584919694811 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0003240584919694811 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:394.7734375 - response_length/max:1392.0 - response_length/min:184.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:394.7734375 - response_length_non_aborted/max:1392.0 - response_length_non_aborted/min:184.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.8125 - prompt_length/max:482.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010274350643157959 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1902959402650595) - timing_s/agent_loop/generate_sequences/max:np.float64(9.604242615401745) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.197183445976407) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.604242615401745) - timing_s/agent_loop/slowest/tool_
(TaskRunner pid=2001144) Training Progress: 88%|████████▊ | 88/100 [1:01:08<06:30, 32.58s/it]
(TaskRunner pid=2001144) step:89 - global_seqlen/min:18826 - global_seqlen/max:27590 - global_seqlen/minmax_diff:8764 - global_seqlen/balanced_min:22527 - global_seqlen/balanced_max:22698 - global_seqlen/mean:22556.75 - actor/entropy:0.44242504239082336 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7417161464691162 - training/rollout_probs_diff_mean:0.005717849358916283 - training/rollout_probs_diff_std:0.012119048275053501 - training/rollout_actor_probs_pearson_corr:0.9990301728248596 - rollout_corr/rollout_is_mean:1.0001914501190186 - rollout_corr/rollout_is_ratio_fraction_high:3.77821852453053e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.77821852453053e-05 - rollout_corr/rollout_is_max:2.390676259994507 - rollout_corr/rollout_is_min:0.09278460592031479 - rollout_corr/rollout_is_std:0.040012259036302567 - rollout_corr/rollout_is_eff_sample_size:0.9984022912552583 - rollout_corr/rollout_is_seq_mean:1.000243067741394 - rollout_corr/rollout_is_seq_std:0.002152640139684081 - rollout_corr/rollout_is_seq_max:1.010016918182373 - rollout_corr/rollout_is_seq_min:0.994374692440033 - rollout_corr/rollout_is_seq_max_deviation:0.010016918182373047 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.570118747651577) - rollout_corr/training_log_ppl:np.float64(0.44658595003420487) - rollout_corr/kl:np.float64(0.000813204325964989) - rollout_corr/k3_kl:np.float64(0.000954299051727503) - rollout_corr/rollout_ppl:np.float64(1.56883160257712) - rollout_corr/rollout_log_ppl:np.float64(0.44577274698531255) - rollout_corr/log_ppl_diff:np.float64(0.0008132030488923192) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019919712794944644) - rollout_corr/log_ppl_diff_max:np.float64(0.009704381227493286) - rollout_corr/log_ppl_diff_min:np.float64(-0.005835831165313721) - rollout_corr/ppl_ratio:np.float64(1.0008164767641574) - rollout_corr/chi2_token:np.float64(0.0021558518055826426) - rollout_corr/chi2_seq:np.float64(0.8406954142265022) - actor/pg_loss:np.float64(-2.941116690635681e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002567589408499771) - actor/ppo_kl:np.float64(0.00019581326003503818) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3205021396279335) - perf/mfu/actor:np.float64(0.06640479906223654) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(103.89166259765625) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:1 - critic/score/mean:0.52734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.52734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019068196415901184 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0019068196415901184 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:413.5546875 - response_length/max:1818.0 - response_length/min:144.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:413.5546875 - response_length_non_aborted/max:1818.0 - response_length_non_aborted/min:144.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.34375 - prompt_length/max:720.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011474452912807465 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7104199025779963) - timing_s/agent_loop/generate_sequences/max:np.float64(11.53880456648767) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.351657953993708) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.53880456648767) - timing_s/age
(TaskRunner pid=2001144) Training Progress: 89%|████████▉ | 89/100 [1:01:38<05:49, 31.77s/it]
(TaskRunner pid=2001144) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90}
(TaskRunner pid=2001144) validation generation end
(TaskRunner pid=2001144) [prompt] system
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2001144) <reasoning>
(TaskRunner pid=2001144) ...
(TaskRunner pid=2001144) </reasoning>
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) ...
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) A
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) user
(TaskRunner pid=2001144) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2001144) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2001144) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2001144) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2001144) Please reason step by step.
(TaskRunner pid=2001144) assistant
(TaskRunner pid=2001144) <think>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) </think>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) [response] <reasoning>
(TaskRunner pid=2001144) The question asks which of the given molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". This product contains a substituted aryl group with a Cl atom and a trifluoromethyl group (C(F)(F)F), which suggests that the reactant must have a similar structure to match the product.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Looking at the options:
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) - **Option A**: This molecule has a structure that includes a substituted aryl group with a Cl atom and a C(F)(F)F group, which matches the product. The structure also includes a nitrogen atom and a carbon chain that seems to fit the pattern of the product's structure. This is a strong candidate.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) - **Option B**: This option has a structure that includes a Cl atom but does not have the C(F)(F)F group. This is a mismatch with the product, which has the C(F)(F)F group.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) - **Option C**: This option has a structure that includes a Cl atom and a substituted aryl group, but the overall structure does not match the product. It includes an oxygen atom and a different arrangement that doesn't align with the product's structure.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) - **Option D**: This option includes a C(F)(F)F group but does not have the Cl atom in the correct position. The overall structure does not match the product's structure either.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Therefore, the only option that matches the product's structure, including the Cl and C(F)(F)F groups, is **Option A**.
(TaskRunner pid=2001144) </reasoning>
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) A
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144) [ground_truth] A
(TaskRunner pid=2001144) [score] 1.0
(TaskRunner pid=2001144) [acc] 1.0
(TaskRunner pid=2001144) [pred] A
(TaskRunner pid=2001144) [incorrect_format] 1
(TaskRunner pid=2001144) [feedback]
(TaskRunner pid=2001144) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90
(WorkerDict pid=2001534) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80/actor
(TaskRunner pid=2001144) step:90 - global_seqlen/min:17727 - global_seqlen/max:26019 - global_seqlen/minmax_diff:8292 - global_seqlen/balanced_min:21832 - global_seqlen/balanced_max:21835 - global_seqlen/mean:21833.375 - actor/entropy:0.4232009947299957 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39975857734680176 - training/rollout_probs_diff_mean:0.0054819597862660885 - training/rollout_probs_diff_std:0.011842027306556702 - training/rollout_actor_probs_pearson_corr:0.9990738034248352 - rollout_corr/rollout_is_mean:1.000044345855713 - rollout_corr/rollout_is_ratio_fraction_high:2.8976828616578132e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.761260010534897e-05 - rollout_corr/rollout_is_max:2.442352294921875 - rollout_corr/rollout_is_min:0.1256181299686432 - rollout_corr/rollout_is_std:0.039298299700021744 - rollout_corr/rollout_is_eff_sample_size:0.9984581438546785 - rollout_corr/rollout_is_seq_mean:1.000123143196106 - rollout_corr/rollout_is_seq_std:0.0021049731876701117 - rollout_corr/rollout_is_seq_max:1.006156086921692 - rollout_corr/rollout_is_seq_min:0.9943211078643799 - rollout_corr/rollout_is_seq_max_deviation:0.0061560869216918945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.559266529045999) - rollout_corr/training_log_ppl:np.float64(0.43788518285145983) - rollout_corr/kl:np.float64(0.000856765673598936) - rollout_corr/k3_kl:np.float64(0.0008866318072477952) - rollout_corr/rollout_ppl:np.float64(1.5579178975895047) - rollout_corr/rollout_log_ppl:np.float64(0.43702841736376286) - rollout_corr/log_ppl_diff:np.float64(0.0008567654876969755) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018750238814391196) - rollout_corr/log_ppl_diff_max:np.float64(0.011012017726898193) - rollout_corr/log_ppl_diff_min:np.float64(-0.007498234510421753) - rollout_corr/ppl_ratio:np.float64(1.0008598149288446) - rollout_corr/chi2_token:np.float64(0.001842310419306159) - rollout_corr/chi2_seq:np.float64(1.0038544649723917) - actor/pg_loss:np.float64(-2.9308139346539974e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00031073657191882376) - actor/ppo_kl:np.float64(0.00016516280089007296) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.28973547369241714) - perf/mfu/actor:np.float64(0.06426962290682042) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(103.81690979003906) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6788690476190476) - val-aux/sciknoweval/reward/std@16:np.float64(0.12678625522448414) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7299333333333333) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.09878477424667818) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6280190476190477) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.11524351572376498) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6785571428571427) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.12620272944655192) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7671238095238095) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.0683093031243327) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5827666666666667) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.0942999773142172) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6873952380952382) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.10066758417348941) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7923666666666667) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.04451381518779437) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5454285714285714) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07116584822412066) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6923095238095238) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07268256787209855) - val-aux/sciknoweval/reward/
(TaskRunner pid=2001144) Training Progress: 90%|█████████ | 90/100 [1:03:33<09:26, 56.70s/it]
(TaskRunner pid=2001144) step:91 - global_seqlen/min:18135 - global_seqlen/max:28169 - global_seqlen/minmax_diff:10034 - global_seqlen/balanced_min:22032 - global_seqlen/balanced_max:22048 - global_seqlen/mean:22044.625 - actor/entropy:0.46319979429244995 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2930457592010498 - training/rollout_probs_diff_mean:0.005693704355508089 - training/rollout_probs_diff_std:0.011664330959320068 - training/rollout_actor_probs_pearson_corr:0.9991028308868408 - rollout_corr/rollout_is_mean:1.0002021789550781 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.8793896237621084e-05 - rollout_corr/rollout_is_max:1.9660584926605225 - rollout_corr/rollout_is_min:0.08076423406600952 - rollout_corr/rollout_is_std:0.04058172553777695 - rollout_corr/rollout_is_eff_sample_size:0.9983565443133803 - rollout_corr/rollout_is_seq_mean:1.0002031326293945 - rollout_corr/rollout_is_seq_std:0.002250741235911846 - rollout_corr/rollout_is_seq_max:1.0072354078292847 - rollout_corr/rollout_is_seq_min:0.9932303428649902 - rollout_corr/rollout_is_seq_max_deviation:0.007235407829284668 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6091117658652365) - rollout_corr/training_log_ppl:np.float64(0.4688153638271615) - rollout_corr/kl:np.float64(0.0007379944695662388) - rollout_corr/k3_kl:np.float64(0.0009104512178055302) - rollout_corr/rollout_ppl:np.float64(1.6078901127912104) - rollout_corr/rollout_log_ppl:np.float64(0.46807737060589716) - rollout_corr/log_ppl_diff:np.float64(0.0007379932212643325) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020341871422715485) - rollout_corr/log_ppl_diff_max:np.float64(0.009881436824798584) - rollout_corr/log_ppl_diff_min:np.float64(-0.005900025367736816) - rollout_corr/ppl_ratio:np.float64(1.000741575146094) - rollout_corr/chi2_token:np.float64(0.002125032711774111) - rollout_corr/chi2_seq:np.float64(1.5019807838834822) - actor/pg_loss:np.float64(-4.655460361391306e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004238625938342011) - actor/ppo_kl:np.float64(8.059631479184759e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6602651104331017) - perf/mfu/actor:np.float64(0.06468134287005539) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(105.94984817504883) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:1 - critic/score/mean:0.57421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.57421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00483832648023963 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00483832648023963 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:402.76953125 - response_length/max:1238.0 - response_length/min:173.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:402.76953125 - response_length_non_aborted/max:1238.0 - response_length_non_aborted/min:173.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.125 - prompt_length/max:1009.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.341934204101562e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.089082647114992) - timing_s/agent_loop/generate_sequences/max:np.float64(8.856052953749895) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.206110581246321) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.856052953749895) - timing_s/agent_loop/slowes
(TaskRunner pid=2001144) Training Progress: 91%|█████████ | 91/100 [1:04:00<07:10, 47.85s/it]
(TaskRunner pid=2001144) step:92 - global_seqlen/min:17900 - global_seqlen/max:29108 - global_seqlen/minmax_diff:11208 - global_seqlen/balanced_min:22818 - global_seqlen/balanced_max:23999 - global_seqlen/mean:22973.25 - actor/entropy:0.4327438771724701 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5128510594367981 - training/rollout_probs_diff_mean:0.005590232089161873 - training/rollout_probs_diff_std:0.01210536528378725 - training/rollout_actor_probs_pearson_corr:0.9989980459213257 - rollout_corr/rollout_is_mean:1.0000828504562378 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.6302251828601584e-05 - rollout_corr/rollout_is_max:1.9660102128982544 - rollout_corr/rollout_is_min:0.14373043179512024 - rollout_corr/rollout_is_std:0.040419839322566986 - rollout_corr/rollout_is_eff_sample_size:0.9983692579778656 - rollout_corr/rollout_is_seq_mean:1.0001684427261353 - rollout_corr/rollout_is_seq_std:0.002204200020059943 - rollout_corr/rollout_is_seq_max:1.0092856884002686 - rollout_corr/rollout_is_seq_min:0.9952259659767151 - rollout_corr/rollout_is_seq_max_deviation:0.009285688400268555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5814849096350372) - rollout_corr/training_log_ppl:np.float64(0.4503559048171155) - rollout_corr/kl:np.float64(0.0008601158773160478) - rollout_corr/k3_kl:np.float64(0.0008870534830975885) - rollout_corr/rollout_ppl:np.float64(1.580087968148291) - rollout_corr/rollout_log_ppl:np.float64(0.44949578505475074) - rollout_corr/log_ppl_diff:np.float64(0.0008601197623647749) - rollout_corr/log_ppl_abs_diff:np.float64(0.001828863809350878) - rollout_corr/log_ppl_diff_max:np.float64(0.006230533123016357) - rollout_corr/log_ppl_diff_min:np.float64(-0.006673961877822876) - rollout_corr/ppl_ratio:np.float64(1.000862859422341) - rollout_corr/chi2_token:np.float64(0.0018361934926360846) - rollout_corr/chi2_seq:np.float64(0.7432863691356033) - actor/pg_loss:np.float64(0.0004688943736255169) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004661728883093019) - actor/ppo_kl:np.float64(0.00015811839311830056) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.39724695682525635) - perf/mfu/actor:np.float64(0.06690770125677487) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(104.98439407348633) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:1 - critic/score/mean:0.578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01029509212821722 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01029509212821722 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:430.4140625 - response_length/max:2525.0 - response_length/min:187.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:430.4140625 - response_length_non_aborted/max:2525.0 - response_length_non_aborted/min:187.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:287.5 - prompt_length/max:436.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001069679856300354 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1963943894952536) - timing_s/agent_loop/generate_sequences/max:np.float64(15.043243188410997) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.516788047621958) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.043243188410997) - timing_s/agent_loop/slowest/tool_c
(TaskRunner pid=2001144) Training Progress: 92%|█████████▏| 92/100 [1:04:34<05:48, 43.60s/it]
(TaskRunner pid=2001144) step:93 - global_seqlen/min:17752 - global_seqlen/max:29433 - global_seqlen/minmax_diff:11681 - global_seqlen/balanced_min:22027 - global_seqlen/balanced_max:22046 - global_seqlen/mean:22038.875 - actor/entropy:0.44752171635627747 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4668269157409668 - training/rollout_probs_diff_mean:0.005631833802908659 - training/rollout_probs_diff_std:0.011742200702428818 - training/rollout_actor_probs_pearson_corr:0.9991103410720825 - rollout_corr/rollout_is_mean:1.000031590461731 - rollout_corr/rollout_is_ratio_fraction_high:1.8502243619877845e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.700448723975569e-05 - rollout_corr/rollout_is_max:5.393937110900879 - rollout_corr/rollout_is_min:0.48396989703178406 - rollout_corr/rollout_is_std:0.040105994790792465 - rollout_corr/rollout_is_eff_sample_size:0.9983939733176294 - rollout_corr/rollout_is_seq_mean:1.000080943107605 - rollout_corr/rollout_is_seq_std:0.0022998463828116655 - rollout_corr/rollout_is_seq_max:1.0187183618545532 - rollout_corr/rollout_is_seq_min:0.993992805480957 - rollout_corr/rollout_is_seq_max_deviation:0.018718361854553223 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5782851097173989) - rollout_corr/training_log_ppl:np.float64(0.4492686166195199) - rollout_corr/kl:np.float64(0.0007467737673465535) - rollout_corr/k3_kl:np.float64(0.0010051427101416266) - rollout_corr/rollout_ppl:np.float64(1.577059440780431) - rollout_corr/rollout_log_ppl:np.float64(0.44852184265619144) - rollout_corr/log_ppl_diff:np.float64(0.0007467739633284509) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019104572129435837) - rollout_corr/log_ppl_diff_max:np.float64(0.008987128734588623) - rollout_corr/log_ppl_diff_min:np.float64(-0.014498472213745117) - rollout_corr/ppl_ratio:np.float64(1.0007501747459173) - rollout_corr/chi2_token:np.float64(0.0036972654052078724) - rollout_corr/chi2_seq:np.float64(7.838378930930048) - actor/pg_loss:np.float64(0.0001622772542759776) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00038849973861943) - actor/ppo_kl:np.float64(-4.164984736121369e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3390331566333771) - perf/mfu/actor:np.float64(0.06457848599642167) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(104.35161209106445) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:1 - critic/score/mean:0.8671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002061843639239669 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002061843639239669 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:422.24609375 - response_length/max:1127.0 - response_length/min:159.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:422.24609375 - response_length_non_aborted/max:1127.0 - response_length_non_aborted/min:159.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:709.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014812126755714417 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8472609333693981) - timing_s/agent_loop/generate_sequences/max:np.float64(8.41776162572205) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.422026272332005) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.41776162572205) - timing_s/ag
(TaskRunner pid=2001144) Training Progress: 93%|█████████▎| 93/100 [1:05:01<04:29, 38.53s/it]
(TaskRunner pid=2001144) step:94 - global_seqlen/min:17585 - global_seqlen/max:23714 - global_seqlen/minmax_diff:6129 - global_seqlen/balanced_min:21396 - global_seqlen/balanced_max:21403 - global_seqlen/mean:21400.0 - actor/entropy:0.45375704765319824 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5147505402565002 - training/rollout_probs_diff_mean:0.005501046776771545 - training/rollout_probs_diff_std:0.011572875082492828 - training/rollout_actor_probs_pearson_corr:0.9991158843040466 - rollout_corr/rollout_is_mean:1.0000462532043457 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.391435014549643e-06 - rollout_corr/rollout_is_max:1.6395314931869507 - rollout_corr/rollout_is_min:0.4596215486526489 - rollout_corr/rollout_is_std:0.03904571011662483 - rollout_corr/rollout_is_eff_sample_size:0.9984779908574707 - rollout_corr/rollout_is_seq_mean:0.9999992251396179 - rollout_corr/rollout_is_seq_std:0.001986921299248934 - rollout_corr/rollout_is_seq_max:1.0053033828735352 - rollout_corr/rollout_is_seq_min:0.9915086627006531 - rollout_corr/rollout_is_seq_max_deviation:0.008491337299346924 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5910439868457615) - rollout_corr/training_log_ppl:np.float64(0.4584993905154988) - rollout_corr/kl:np.float64(0.0006600743450874136) - rollout_corr/k3_kl:np.float64(0.0008585889312371364) - rollout_corr/rollout_ppl:np.float64(1.5899825398810208) - rollout_corr/rollout_log_ppl:np.float64(0.45783931750338525) - rollout_corr/log_ppl_diff:np.float64(0.0006600730121135712) - rollout_corr/log_ppl_abs_diff:np.float64(0.001664652954787016) - rollout_corr/log_ppl_diff_max:np.float64(0.00827866792678833) - rollout_corr/log_ppl_diff_min:np.float64(-0.005626559257507324) - rollout_corr/ppl_ratio:np.float64(1.000662395497784) - rollout_corr/chi2_token:np.float64(0.002135754795745015) - rollout_corr/chi2_seq:np.float64(0.7371289066504687) - actor/pg_loss:np.float64(0.00011552649084478617) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024462193687213585) - actor/ppo_kl:np.float64(-0.0001251153578014641) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3210117071866989) - perf/mfu/actor:np.float64(0.06280807868086452) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(104.10160827636719) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:1 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0004918763879686594 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0004918763879686594 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:415.9375 - response_length/max:1041.0 - response_length/min:171.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:415.9375 - response_length_non_aborted/max:1041.0 - response_length_non_aborted/min:171.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:252.8125 - prompt_length/max:384.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.969063103199005e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.02528010122478) - timing_s/agent_loop/generate_sequences/max:np.float64(7.924375336617231) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.402820943199913) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.924375336617231) - timing_s/agent_loop/slowest/tool_c
(TaskRunner pid=2001144) Training Progress: 94%|█████████▍| 94/100 [1:05:27<03:29, 34.90s/it]
(TaskRunner pid=2001144) step:95 - global_seqlen/min:18675 - global_seqlen/max:24139 - global_seqlen/minmax_diff:5464 - global_seqlen/balanced_min:21192 - global_seqlen/balanced_max:21207 - global_seqlen/mean:21202.75 - actor/entropy:0.46913227438926697 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2692910432815552 - training/rollout_probs_diff_mean:0.00569215789437294 - training/rollout_probs_diff_std:0.011702047660946846 - training/rollout_actor_probs_pearson_corr:0.9991575479507446 - rollout_corr/rollout_is_mean:1.0001296997070312 - rollout_corr/rollout_is_ratio_fraction_high:9.780334039533045e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.780334039533045e-06 - rollout_corr/rollout_is_max:2.882702589035034 - rollout_corr/rollout_is_min:0.4166666567325592 - rollout_corr/rollout_is_std:0.03997201472520828 - rollout_corr/rollout_is_eff_sample_size:0.9984051431514943 - rollout_corr/rollout_is_seq_mean:1.000004768371582 - rollout_corr/rollout_is_seq_std:0.002265624701976776 - rollout_corr/rollout_is_seq_max:1.006104588508606 - rollout_corr/rollout_is_seq_min:0.9921392798423767 - rollout_corr/rollout_is_seq_max_deviation:0.007860720157623291 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6265227515250444) - rollout_corr/training_log_ppl:np.float64(0.4783231141627766) - rollout_corr/kl:np.float64(0.0010214125970700039) - rollout_corr/k3_kl:np.float64(0.0008742390372162845) - rollout_corr/rollout_ppl:np.float64(1.6248298487626016) - rollout_corr/rollout_log_ppl:np.float64(0.47730169957503676) - rollout_corr/log_ppl_diff:np.float64(0.001021414587739855) - rollout_corr/log_ppl_abs_diff:np.float64(0.001925189804751426) - rollout_corr/log_ppl_diff_max:np.float64(0.007709681987762451) - rollout_corr/log_ppl_diff_min:np.float64(-0.0052089691162109375) - rollout_corr/ppl_ratio:np.float64(1.001024599885568) - rollout_corr/chi2_token:np.float64(0.0014709951356053352) - rollout_corr/chi2_seq:np.float64(0.8471907007042319) - actor/pg_loss:np.float64(-4.1293445974588394e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022785279179515783) - actor/ppo_kl:np.float64(0.00019291428430534552) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2586103118956089) - perf/mfu/actor:np.float64(0.06273351897098865) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(104.06320571899414) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0014303737552836537 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0014303737552836537 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:399.3984375 - response_length/max:1169.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:399.3984375 - response_length_non_aborted/max:1169.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.1875 - prompt_length/max:646.0 - prompt_length/min:159.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010355748236179352 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2836139257997274) - timing_s/agent_loop/generate_sequences/max:np.float64(8.44287264533341) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.251615054956346) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.44287264533341) - timing_s/agent_
(TaskRunner pid=2001144) Training Progress: 95%|█████████▌| 95/100 [1:05:54<02:41, 32.36s/it]
(TaskRunner pid=2001144) step:96 - global_seqlen/min:19447 - global_seqlen/max:28024 - global_seqlen/minmax_diff:8577 - global_seqlen/balanced_min:22386 - global_seqlen/balanced_max:22975 - global_seqlen/mean:22461.25 - actor/entropy:0.48031309247016907 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6418519020080566 - training/rollout_probs_diff_mean:0.005504278931766748 - training/rollout_probs_diff_std:0.011381902731955051 - training/rollout_actor_probs_pearson_corr:0.9991637468338013 - rollout_corr/rollout_is_mean:0.9999828934669495 - rollout_corr/rollout_is_ratio_fraction_high:1.7949132598005235e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.589826519601047e-05 - rollout_corr/rollout_is_max:2.0751261711120605 - rollout_corr/rollout_is_min:0.2610307037830353 - rollout_corr/rollout_is_std:0.03877149894833565 - rollout_corr/rollout_is_eff_sample_size:0.9984989083131864 - rollout_corr/rollout_is_seq_mean:0.9999107122421265 - rollout_corr/rollout_is_seq_std:0.0020033454056829214 - rollout_corr/rollout_is_seq_max:1.0061962604522705 - rollout_corr/rollout_is_seq_min:0.9932329058647156 - rollout_corr/rollout_is_seq_max_deviation:0.006767094135284424 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6467337417416275) - rollout_corr/training_log_ppl:np.float64(0.49085872640716843) - rollout_corr/kl:np.float64(0.001185998961239998) - rollout_corr/k3_kl:np.float64(0.0008846716466450744) - rollout_corr/rollout_ppl:np.float64(1.6446700925007463) - rollout_corr/rollout_log_ppl:np.float64(0.4896727246523369) - rollout_corr/log_ppl_diff:np.float64(0.0011860017548315227) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019397185533307493) - rollout_corr/log_ppl_diff_max:np.float64(0.011907339096069336) - rollout_corr/log_ppl_diff_min:np.float64(-0.0048629045486450195) - rollout_corr/ppl_ratio:np.float64(1.001189422328025) - rollout_corr/chi2_token:np.float64(0.001117313513532281) - rollout_corr/chi2_seq:np.float64(0.5539289293810725) - actor/pg_loss:np.float64(-1.2987409718334675e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002163373605981178) - actor/ppo_kl:np.float64(0.0002927634003206947) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2742002569139004) - perf/mfu/actor:np.float64(0.0654678219755767) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(104.08746719360352) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002184184966608882 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.002184184966608882 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:435.2578125 - response_length/max:2031.0 - response_length/min:183.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:435.2578125 - response_length_non_aborted/max:2031.0 - response_length_non_aborted/min:183.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.65625 - prompt_length/max:456.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001114383339881897 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0028380136936903) - timing_s/agent_loop/generate_sequences/max:np.float64(12.787106862291694) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.617529367977113) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.787106862291694) - timing_s/a
(TaskRunner pid=2001144) Training Progress: 96%|█████████▌| 96/100 [1:06:25<02:07, 31.98s/it]
(TaskRunner pid=2001144) step:97 - global_seqlen/min:18523 - global_seqlen/max:29238 - global_seqlen/minmax_diff:10715 - global_seqlen/balanced_min:22764 - global_seqlen/balanced_max:22779 - global_seqlen/mean:22775.875 - actor/entropy:0.47625014185905457 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27380454540252686 - training/rollout_probs_diff_mean:0.005669302772730589 - training/rollout_probs_diff_std:0.011558400467038155 - training/rollout_actor_probs_pearson_corr:0.9991499185562134 - rollout_corr/rollout_is_mean:1.0002156496047974 - rollout_corr/rollout_is_ratio_fraction_high:6.551914702868089e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.359878276882228e-06 - rollout_corr/rollout_is_max:2.671459674835205 - rollout_corr/rollout_is_min:0.47194260358810425 - rollout_corr/rollout_is_std:0.04106355458498001 - rollout_corr/rollout_is_eff_sample_size:0.9983172171770474 - rollout_corr/rollout_is_seq_mean:1.0001957416534424 - rollout_corr/rollout_is_seq_std:0.0020187110640108585 - rollout_corr/rollout_is_seq_max:1.0113143920898438 - rollout_corr/rollout_is_seq_min:0.9922095537185669 - rollout_corr/rollout_is_seq_max_deviation:0.01131439208984375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6419487777166069) - rollout_corr/training_log_ppl:np.float64(0.4878401471651159) - rollout_corr/kl:np.float64(0.0009260169798737117) - rollout_corr/k3_kl:np.float64(0.0010060044570536775) - rollout_corr/rollout_ppl:np.float64(1.640379909425974) - rollout_corr/rollout_log_ppl:np.float64(0.48691413091728464) - rollout_corr/log_ppl_diff:np.float64(0.000926016247831285) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018741809763014317) - rollout_corr/log_ppl_diff_max:np.float64(0.009277880191802979) - rollout_corr/log_ppl_diff_min:np.float64(-0.004834771156311035) - rollout_corr/ppl_ratio:np.float64(1.0009288769215345) - rollout_corr/chi2_token:np.float64(0.0021942330058664083) - rollout_corr/chi2_seq:np.float64(1.3588568300474435) - actor/pg_loss:np.float64(-8.591543883085251e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003755323916720954) - actor/ppo_kl:np.float64(0.00024774550554695907) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3222628980875015) - perf/mfu/actor:np.float64(0.06698155411012262) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(103.99562072753906) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:1 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013039479963481426 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013039479963481426 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:417.33984375 - response_length/max:1394.0 - response_length/min:160.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:417.33984375 - response_length_non_aborted/max:1394.0 - response_length_non_aborted/min:160.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:294.40625 - prompt_length/max:900.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.660741150379181e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8591227810829878) - timing_s/agent_loop/generate_sequences/max:np.float64(9.921397039666772) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.444997106147639) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.921397039666772) - timi
(TaskRunner pid=2001144) Training Progress: 97%|█████████▋| 97/100 [1:06:53<01:32, 30.88s/it]
(TaskRunner pid=2001144) step:98 - global_seqlen/min:18062 - global_seqlen/max:27492 - global_seqlen/minmax_diff:9430 - global_seqlen/balanced_min:22173 - global_seqlen/balanced_max:22175 - global_seqlen/mean:22173.875 - actor/entropy:0.4679155945777893 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3663342595100403 - training/rollout_probs_diff_mean:0.005862907040864229 - training/rollout_probs_diff_std:0.012160965241491795 - training/rollout_actor_probs_pearson_corr:0.9990532398223877 - rollout_corr/rollout_is_mean:1.0000195503234863 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.9726810680585913e-05 - rollout_corr/rollout_is_max:1.7753713130950928 - rollout_corr/rollout_is_min:0.22344882786273956 - rollout_corr/rollout_is_std:0.041037417948246 - rollout_corr/rollout_is_eff_sample_size:0.9983188804977047 - rollout_corr/rollout_is_seq_mean:1.0000040531158447 - rollout_corr/rollout_is_seq_std:0.0021857365500181913 - rollout_corr/rollout_is_seq_max:1.0071172714233398 - rollout_corr/rollout_is_seq_min:0.9943082332611084 - rollout_corr/rollout_is_seq_max_deviation:0.007117271423339844 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6160536529496312) - rollout_corr/training_log_ppl:np.float64(0.47414031298831105) - rollout_corr/kl:np.float64(0.0009070881798716179) - rollout_corr/k3_kl:np.float64(0.0009147315522568533) - rollout_corr/rollout_ppl:np.float64(1.61455712094903) - rollout_corr/rollout_log_ppl:np.float64(0.47323322045849636) - rollout_corr/log_ppl_diff:np.float64(0.0009070925298146904) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019306147587485611) - rollout_corr/log_ppl_diff_max:np.float64(0.007121026515960693) - rollout_corr/log_ppl_diff_min:np.float64(-0.0068073570728302) - rollout_corr/ppl_ratio:np.float64(1.000910148723051) - rollout_corr/chi2_token:np.float64(0.0018317520152777433) - rollout_corr/chi2_seq:np.float64(0.7488938681781292) - actor/pg_loss:np.float64(9.624834638088942e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003280359201198735) - actor/ppo_kl:np.float64(4.666207246195597e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29307667911052704) - perf/mfu/actor:np.float64(0.06552486264485657) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(103.98208999633789) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:1 - critic/score/mean:0.51953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008212031796574593 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008212031796574593 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:394.21484375 - response_length/max:1051.0 - response_length/min:185.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:394.21484375 - response_length_non_aborted/max:1051.0 - response_length_non_aborted/min:185.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:631.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.7208741307258606e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.246021695435047) - timing_s/agent_loop/generate_sequences/max:np.float64(7.843934180215001) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.206150739606528) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.843934180215001) - timing_s/agent_loop/slowe
(TaskRunner pid=2001144) Training Progress: 98%|█████████▊| 98/100 [1:07:19<00:59, 29.52s/it]
(TaskRunner pid=2001144) step:99 - global_seqlen/min:18598 - global_seqlen/max:27636 - global_seqlen/minmax_diff:9038 - global_seqlen/balanced_min:23083 - global_seqlen/balanced_max:23101 - global_seqlen/mean:23095.625 - actor/entropy:0.4448176920413971 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45196691155433655 - training/rollout_probs_diff_mean:0.005695185158401728 - training/rollout_probs_diff_std:0.012096690014004707 - training/rollout_actor_probs_pearson_corr:0.9990444779396057 - rollout_corr/rollout_is_mean:0.9999328255653381 - rollout_corr/rollout_is_ratio_fraction_high:1.9084516679868102e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.862677320081275e-05 - rollout_corr/rollout_is_max:4.066524982452393 - rollout_corr/rollout_is_min:0.37569519877433777 - rollout_corr/rollout_is_std:0.040833570063114166 - rollout_corr/rollout_is_eff_sample_size:0.9983351575709272 - rollout_corr/rollout_is_seq_mean:0.9999529123306274 - rollout_corr/rollout_is_seq_std:0.002244414761662483 - rollout_corr/rollout_is_seq_max:1.0066914558410645 - rollout_corr/rollout_is_seq_min:0.9925167560577393 - rollout_corr/rollout_is_seq_max_deviation:0.007483243942260742 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5879877875559032) - rollout_corr/training_log_ppl:np.float64(0.4561408114095684) - rollout_corr/kl:np.float64(0.001010150648010466) - rollout_corr/k3_kl:np.float64(0.0009974992898378332) - rollout_corr/rollout_ppl:np.float64(1.5863256505690515) - rollout_corr/rollout_log_ppl:np.float64(0.45513065773411654) - rollout_corr/log_ppl_diff:np.float64(0.0010101536754518747) - rollout_corr/log_ppl_abs_diff:np.float64(0.001963622635230422) - rollout_corr/log_ppl_diff_max:np.float64(0.009279310703277588) - rollout_corr/log_ppl_diff_min:np.float64(-0.005953103303909302) - rollout_corr/ppl_ratio:np.float64(1.0010132703464478) - rollout_corr/chi2_token:np.float64(0.0028357976116240025) - rollout_corr/chi2_seq:np.float64(1.1871257869061083) - actor/pg_loss:np.float64(1.9366852939128876e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00032031702812673757) - actor/ppo_kl:np.float64(6.454805496147742e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2805963382124901) - perf/mfu/actor:np.float64(0.06764870633684238) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(103.94205474853516) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:1 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006598471198230982 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006598471198230982 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:409.36328125 - response_length/max:1187.0 - response_length/min:170.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:409.36328125 - response_length_non_aborted/max:1187.0 - response_length_non_aborted/min:170.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.375 - prompt_length/max:836.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.706616401672363e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4024337120354176) - timing_s/agent_loop/generate_sequences/max:np.float64(8.622185323387384) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.165816112996254) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.622185323387384) - timing_s
(TaskRunner pid=2001144) Training Progress: 99%|█████████▉| 99/100 [1:07:46<00:28, 28.73s/it]
(TaskRunner pid=2001144) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100}
(TaskRunner pid=2001144) validation generation end
(TaskRunner pid=2001144) [prompt] system
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2001144) <reasoning>
(TaskRunner pid=2001144) ...
(TaskRunner pid=2001144) </reasoning>
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) ...
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) A
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) user
(TaskRunner pid=2001144) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2001144) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2001144) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2001144) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2001144) Please reason step by step.
(TaskRunner pid=2001144) assistant
(TaskRunner pid=2001144) <think>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) </think>
(TaskRunner pid=2001144)
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) [response] <reasoning>
(TaskRunner pid=2001144) The question asks which of the given molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". To determine the correct answer, we need to analyze the structure of the product and compare it with the structures of the given options.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) The product has the following key features:
(TaskRunner pid=2001144) - A nitrogen atom (N) bonded to a carbon atom (C1), which is part of a chain.
(TaskRunner pid=2001144) - The chain includes a "Cc2ccc(Cl)cc2C(F)(F)F" fragment, which suggests a substituted benzene ring with a chlorine atom and a trifluoromethyl group (C(F)(F)F).
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Now, let's examine the options:
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) **Option A:**
(TaskRunner pid=2001144) O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2001144) This option contains a fragment "Cc2ccc(Cl)cc2C(F)(F)F" that matches the structure in the product. It also includes the "N1c1ccn(...)" fragment, which is consistent with the product's structure. This matches the product's structure very closely.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) **Option B:**
(TaskRunner pid=2001144) O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2001144) This option has a "C(Cl)" group, but the rest of the structure does not match the product's "Cc2ccc(Cl)cc2C(F)(F)F" fragment. Additionally, the "c1ccc2ccc(Cl)nc2n1" part does not match the product's structure. Therefore, this is not the correct reactant.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) **Option C:**
(TaskRunner pid=2001144) O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2001144) This option includes a "c2ccc(Cl)c(C(F)(F)F)c2" fragment, which is similar to the product's "Cc2ccc(Cl)cc2C(F)(F)F" fragment, but it is not identical. Additionally, the rest of the structure is more complex and does not match the product exactly.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) **Option D:**
(TaskRunner pid=2001144) O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2001144) This option includes a "C(F)(F)F" group, which is part of the product, but the rest of the structure is not similar to the product's main chain. It lacks the "Cc2ccc(Cl)cc2" fragment and the "N1c1ccn(...)" part.
(TaskRunner pid=2001144)
(TaskRunner pid=2001144) Therefore, **Option A** is the only one that matches the structure of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" most closely.
(TaskRunner pid=2001144) </reasoning>
(TaskRunner pid=2001144) <answer>
(TaskRunner pid=2001144) A
(TaskRunner pid=2001144) </answer>
(TaskRunner pid=2001144) [ground_truth] A
(TaskRunner pid=2001144) [score] 1.0
(TaskRunner pid=2001144) [acc] 1.0
(TaskRunner pid=2001144) [pred] A
(TaskRunner pid=2001144) [incorrect_format] 1
(TaskRunner pid=2001144) [feedback]
(TaskRunner pid=2001144) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_100
(WorkerDict pid=2001534) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90/actor
(TaskRunner pid=2001144) step:100 - global_seqlen/min:19995 - global_seqlen/max:27971 - global_seqlen/minmax_diff:7976 - global_seqlen/balanced_min:24776 - global_seqlen/balanced_max:24784 - global_seqlen/mean:24781.75 - actor/entropy:0.397185742855072 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3175773024559021 - training/rollout_probs_diff_mean:0.005019628442823887 - training/rollout_probs_diff_std:0.01128289382904768 - training/rollout_actor_probs_pearson_corr:0.9990952014923096 - rollout_corr/rollout_is_mean:0.9998645186424255 - rollout_corr/rollout_is_ratio_fraction_high:8.167398846126162e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.7171793741872534e-05 - rollout_corr/rollout_is_max:2.2078511714935303 - rollout_corr/rollout_is_min:0.3767692446708679 - rollout_corr/rollout_is_std:0.03857421875 - rollout_corr/rollout_is_eff_sample_size:0.9985136461377951 - rollout_corr/rollout_is_seq_mean:0.999876856803894 - rollout_corr/rollout_is_seq_std:0.0019207203295081854 - rollout_corr/rollout_is_seq_max:1.0077742338180542 - rollout_corr/rollout_is_seq_min:0.9923958778381348 - rollout_corr/rollout_is_seq_max_deviation:0.007774233818054199 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5306207397952676) - rollout_corr/training_log_ppl:np.float64(0.41552202677121386) - rollout_corr/kl:np.float64(0.0008094973310157627) - rollout_corr/k3_kl:np.float64(0.0007973912350962564) - rollout_corr/rollout_ppl:np.float64(1.5293380408547819) - rollout_corr/rollout_log_ppl:np.float64(0.4147125284653157) - rollout_corr/log_ppl_diff:np.float64(0.0008094983058981597) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015918280114419758) - rollout_corr/log_ppl_diff_max:np.float64(0.007098793983459473) - rollout_corr/log_ppl_diff_min:np.float64(-0.007830023765563965) - rollout_corr/ppl_ratio:np.float64(1.0008116397075355) - rollout_corr/chi2_token:np.float64(0.0015719085931777954) - rollout_corr/chi2_seq:np.float64(1.4306007844861597) - actor/pg_loss:np.float64(-0.00012778816744685173) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.191586903194548e-05) - actor/ppo_kl:np.float64(-9.224095602267113e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.19856154546141624) - perf/mfu/actor:np.float64(0.07253216062486065) - perf/max_memory_allocated_gb:np.float64(123.54408025741577) - perf/max_memory_reserved_gb:np.float64(129.306640625) - perf/cpu_memory_used_gb:np.float64(104.02334213256836) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6880952380952381) - val-aux/sciknoweval/reward/std@16:np.float64(0.11930428433281097) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.733652380952381) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.08603291821062148) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6426714285714287) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.11377153742699202) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6873380952380953) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.11903344827555573) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7654619047619048) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.05713592644762481) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5992857142857143) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10105952090465425) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6969476190476189) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.0900863886286685) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7866809523809524) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.03693927335257077) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5585714285714286) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0857140101161578) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6996714285714285) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.06145786769010523) - val-aux/sciknoweval/reward/best@16/m
(TaskRunner pid=2001144) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': "
(TaskRunner pid=2001144) "np.float64(0.6880952380952381), 'val-aux/sciknoweval/reward/std@16': "
(TaskRunner pid=2001144) "np.float64(0.11930428433281097), 'val-aux/sciknoweval/reward/best@2/mean': "
(TaskRunner pid=2001144) "np.float64(0.733652380952381), 'val-aux/sciknoweval/reward/best@2/std': "
(TaskRunner pid=2001144) "np.float64(0.08603291821062148), 'val-aux/sciknoweval/reward/worst@2/mean': "
(TaskRunner pid=2001144) "np.float64(0.6426714285714287), 'val-aux/sciknoweval/reward/worst@2/std': "
(TaskRunner pid=2001144) "np.float64(0.11377153742699202), 'val-aux/sciknoweval/reward/maj@2/mean': "
(TaskRunner pid=2001144) "np.float64(0.6873380952380953), 'val-aux/sciknoweval/reward/maj@2/std': "
(TaskRunner pid=2001144) "np.float64(0.11903344827555573), 'val-aux/sciknoweval/reward/best@4/mean': "
(TaskRunner pid=2001144) "np.float64(0.7654619047619048), 'val-aux/sciknoweval/reward/best@4/std': "
(TaskRunner pid=2001144) "np.float64(0.05713592644762481), 'val-aux/sciknoweval/reward/worst@4/mean': "
(TaskRunner pid=2001144) "np.float64(0.5992857142857143), 'val-aux/sciknoweval/reward/worst@4/std': "
(TaskRunner pid=2001144) "np.float64(0.10105952090465425), 'val-aux/sciknoweval/reward/maj@4/mean': "
(TaskRunner pid=2001144) "np.float64(0.6969476190476189), 'val-aux/sciknoweval/reward/maj@4/std': "
(TaskRunner pid=2001144) "np.float64(0.0900863886286685), 'val-aux/sciknoweval/reward/best@8/mean': "
(TaskRunner pid=2001144) "np.float64(0.7866809523809524), 'val-aux/sciknoweval/reward/best@8/std': "
(TaskRunner pid=2001144) "np.float64(0.03693927335257077), 'val-aux/sciknoweval/reward/worst@8/mean': "
(TaskRunner pid=2001144) "np.float64(0.5585714285714286), 'val-aux/sciknoweval/reward/worst@8/std': "
(TaskRunner pid=2001144) "np.float64(0.0857140101161578), 'val-aux/sciknoweval/reward/maj@8/mean': "
(TaskRunner pid=2001144) "np.float64(0.6996714285714285), 'val-aux/sciknoweval/reward/maj@8/std': "
(TaskRunner pid=2001144) "np.float64(0.06145786769010523), 'val-aux/sciknoweval/reward/best@16/mean': "
(TaskRunner pid=2001144) "np.float64(0.8012238095238096), 'val-aux/sciknoweval/reward/best@16/std': "
(TaskRunner pid=2001144) 'np.float64(0.022230541736786433), '
(TaskRunner pid=2001144) "'val-aux/sciknoweval/reward/worst@16/mean': np.float64(0.5196142857142857), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/reward/worst@16/std': np.float64(0.06804166756222384), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/reward/maj@16/mean': np.float64(0.6995095238095238), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/reward/maj@16/std': np.float64(0.046993204377870766), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/mean@16': np.float64(0.6880952380952381), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/std@16': np.float64(0.11930428433281097), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/best@2/mean': np.float64(0.733652380952381), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/best@2/std': np.float64(0.08603291821062148), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/worst@2/mean': np.float64(0.6426714285714287), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/worst@2/std': np.float64(0.11377153742699202), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/maj@2/mean': np.float64(0.6873380952380953), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/maj@2/std': np.float64(0.11903344827555573), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/best@4/mean': np.float64(0.7654619047619048), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/best@4/std': np.float64(0.05713592644762481), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/worst@4/mean': np.float64(0.5992857142857143), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/worst@4/std': np.float64(0.10105952090465425), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/maj@4/mean': np.float64(0.6969476190476189), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/maj@4/std': np.float64(0.0900863886286685), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/best@8/mean': np.float64(0.7866809523809524), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/best@8/std': np.float64(0.03693927335257077), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/worst@8/mean': np.float64(0.5585714285714286), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/worst@8/std': np.float64(0.0857140101161578), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/maj@8/mean': np.float64(0.6996714285714285), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/maj@8/std': np.float64(0.06145786769010523), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/best@16/mean': np.float64(0.8012238095238096), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/best@16/std': np.float64(0.022230541736786433), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/worst@16/mean': np.float64(0.5196142857142857), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/worst@16/std': np.float64(0.06804166756222384), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/maj@16/mean': np.float64(0.6995095238095238), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/score/maj@16/std': np.float64(0.046993204377870766), "
(TaskRunner pid=2001144) "'val-core/sciknoweval/acc/mean@16': np.float64(0.6880952380952381), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/acc/std@16': np.float64(0.11930428433281097), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/acc/best@2/mean': np.float64(0.733652380952381), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/acc/best@2/std': np.float64(0.08603291821062148), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/acc/worst@2/mean': np.float64(0.6426714285714287), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/acc/worst@2/std': np.float64(0.11377153742699202), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/acc/maj@2/mean': np.float64(0.6873380952380953), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/acc/maj@2/std': np.float64(0.11903344827555573), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/acc/best@4/mean': np.float64(0.7654619047619048), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/acc/best@4/std': np.float64(0.05713592644762481), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/acc/worst@4/mean': np.float64(0.5992857142857143), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/acc/worst@4/std': np.float64(0.10105952090465425), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/acc/maj@4/mean': np.float64(0.6969476190476189), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/acc/maj@4/std': np.float64(0.0900863886286685), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/acc/best@8/mean': np.float64(0.7866809523809524), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/acc/best@8/std': np.float64(0.03693927335257077), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/acc/worst@8/mean': np.float64(0.5585714285714286), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/acc/worst@8/std': np.float64(0.0857140101161578), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/acc/maj@8/mean': np.float64(0.6996714285714285), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/acc/maj@8/std': np.float64(0.06145786769010523), "
(TaskRunner pid=2001144) "'val-core/sciknoweval/acc/best@16/mean': np.float64(0.8012238095238096), "
(TaskRunner pid=2001144) "'val-core/sciknoweval/acc/best@16/std': np.float64(0.022230541736786433), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/acc/worst@16/mean': np.float64(0.5196142857142857), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/acc/worst@16/std': np.float64(0.06804166756222384), "
(TaskRunner pid=2001144) "'val-core/sciknoweval/acc/maj@16/mean': np.float64(0.6995095238095238), "
(TaskRunner pid=2001144) "'val-core/sciknoweval/acc/maj@16/std': np.float64(0.046993204377870766), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/mean@16': "
(TaskRunner pid=2001144) 'np.float64(0.9994047619047619), '
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/std@16': "
(TaskRunner pid=2001144) 'np.float64(0.0023053472298853674), '
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/best@2/mean': "
(TaskRunner pid=2001144) 'np.float64(0.999952380952381), '
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/best@2/std': "
(TaskRunner pid=2001144) 'np.float64(0.0006717493323650422), '
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': "
(TaskRunner pid=2001144) 'np.float64(0.9988666666666667), '
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/worst@2/std': "
(TaskRunner pid=2001144) 'np.float64(0.003080186451482751), '
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': "
(TaskRunner pid=2001144) 'np.float64(0.9993761904761905), '
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/maj@2/std': "
(TaskRunner pid=2001144) 'np.float64(0.0023491098553929386), '
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/best@4/mean': np.float64(1.0), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/best@4/std': np.float64(0.0), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': "
(TaskRunner pid=2001144) 'np.float64(0.9978047619047619), '
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/worst@4/std': "
(TaskRunner pid=2001144) 'np.float64(0.004004909923005107), '
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': "
(TaskRunner pid=2001144) 'np.float64(0.9998999999999999), '
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/maj@4/std': "
(TaskRunner pid=2001144) 'np.float64(0.0009636106375423148), '
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': "
(TaskRunner pid=2001144) 'np.float64(0.9960904761904762), '
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/worst@8/std': "
(TaskRunner pid=2001144) 'np.float64(0.004680790716043936), '
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': np.float64(1.0), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/maj@8/std': np.float64(0.0), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': "
(TaskRunner pid=2001144) 'np.float64(0.9938380952380953), '
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/worst@16/std': "
(TaskRunner pid=2001144) 'np.float64(0.004551356214978213), '
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': np.float64(1.0), "
(TaskRunner pid=2001144) "'val-aux/sciknoweval/incorrect_format/maj@16/std': np.float64(0.0), "
(TaskRunner pid=2001144) "'val-aux/num_turns/min': np.int32(2), 'val-aux/num_turns/max': np.int32(2), "
(TaskRunner pid=2001144) "'val-aux/num_turns/mean': np.float64(2.0)}")
(TaskRunner pid=2001144) Training Progress: 100%|██████████| 100/100 [1:10:26<00:00, 68.18s/it] Training Progress: 100%|██████████| 100/100 [1:10:27<00:00, 42.27s/it]
(TaskRunner pid=2001144) wandb: updating run metadata
(TaskRunner pid=2001144) wandb: uploading output.log; uploading wandb-summary.json
(TaskRunner pid=2001144) wandb: uploading output.log; uploading config.yaml
(TaskRunner pid=2001144) wandb: uploading history steps 98-99, summary, console lines 724-848
(TaskRunner pid=2001144) wandb:
(TaskRunner pid=2001144) wandb: Run history:
(TaskRunner pid=2001144) wandb: actor/entropy ▁▂▂▂▂▂▃▄▃▄▄▄▃▄▄▄▄▅▆▅▆▆▇▇▆▆▆▇▆▆▆▆▆▆▇█▇██▇
(TaskRunner pid=2001144) wandb: actor/grad_norm ▇▄▆▇█▇▇▅▆▅▆▆▆▄▅▄▃▄▃▂▃▅▅▁▄▃▂▃▁▁▂▃▆▃▂▂▇▃▂▂
(TaskRunner pid=2001144) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
(TaskRunner pid=2001144) wandb: actor/lr ▁▃▄▅▇███████████████████████████████████
(TaskRunner pid=2001144) wandb: actor/pg_clipfrac ▁▄▅▆█▇█▄▅▇▆█▇▅▄▆▅▄▄▄▄▅▄▂▃▄▂▂▃▅▃▃▄▂▂▃▂▂▃▁
(TaskRunner pid=2001144) wandb: actor/pg_clipfrac_lower ▁▁▁▁▁▁▄▁█▁▁▁▁▁▄▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
(TaskRunner pid=2001144) wandb: actor/pg_loss ▅▂▅▅▃▅▇▆▄▅▄▇▁▄▄▇▄▇▅▅▃▅▃▃▇▃▆▃▄▇▆▃▆▃▄▄▄▄█▅
(TaskRunner pid=2001144) wandb: actor/ppo_kl ▂▇▃▁▂▇▅▆▁▁▇▃█▇▇▅▇▄▅▅▆▂▄▃▆▃▄▄▅▃▅▄▆▆▅▅▅▇▃▂
(TaskRunner pid=2001144) wandb: critic/advantages/max ▁███████████████████████████████████████
(TaskRunner pid=2001144) wandb: critic/advantages/mean ▇▁▇▇▆█▇▇▇▆▆▅▆▆▆▅▇▇▆▆▆▇▇▆▆▆▆▆▆▆▅▆▆█▅▆█▆▄▇
(TaskRunner pid=2001144) wandb: +204 ...
(TaskRunner pid=2001144) wandb:
(TaskRunner pid=2001144) wandb: Run summary:
(TaskRunner pid=2001144) wandb: actor/entropy 0.39719
(TaskRunner pid=2001144) wandb: actor/grad_norm 0.19856
(TaskRunner pid=2001144) wandb: actor/kl_loss 0
(TaskRunner pid=2001144) wandb: actor/lr 0.0
(TaskRunner pid=2001144) wandb: actor/pg_clipfrac 8e-05
(TaskRunner pid=2001144) wandb: actor/pg_clipfrac_lower 0
(TaskRunner pid=2001144) wandb: actor/pg_loss -0.00013
(TaskRunner pid=2001144) wandb: actor/ppo_kl -9e-05
(TaskRunner pid=2001144) wandb: critic/advantages/max 0.75
(TaskRunner pid=2001144) wandb: critic/advantages/mean -0.0033
(TaskRunner pid=2001144) wandb: +204 ...
(TaskRunner pid=2001144) wandb:
(TaskRunner pid=2001144) wandb: 🚀 View run qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/0ozhfamy
(TaskRunner pid=2001144) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root
(TaskRunner pid=2001144) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)
(TaskRunner pid=2001144) wandb: Find logs at: ./wandb/run-20260702_050732-0ozhfamy/logs
(TaskRunner pid=2001144) Exception ignored in atexit callback: <function _start_and_connect_service.<locals>.teardown_atexit at 0x77b97d11ba60>
(TaskRunner pid=2001144) Traceback (most recent call last):
(TaskRunner pid=2001144) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 65, in teardown_atexit
(TaskRunner pid=2001144) conn.teardown(hooks.exit_code)
(TaskRunner pid=2001144) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 299, in teardown
(TaskRunner pid=2001144) self._asyncer.run(publish_teardown_and_close)
(TaskRunner pid=2001144) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run
(TaskRunner pid=2001144) return future.result()
(TaskRunner pid=2001144) ^^^^^^^^^^^^^^^
(TaskRunner pid=2001144) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result
(TaskRunner pid=2001144) return self.__get_result()
(TaskRunner pid=2001144) ^^^^^^^^^^^^^^^^^^^
(TaskRunner pid=2001144) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
(TaskRunner pid=2001144) raise self._exception
(TaskRunner pid=2001144) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap
(TaskRunner pid=2001144) return await fn()
(TaskRunner pid=2001144) ^^^^^^^^^^
(TaskRunner pid=2001144) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 297, in publish_teardown_and_close
(TaskRunner pid=2001144) await self._client.close()
(TaskRunner pid=2001144) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 69, in close
(TaskRunner pid=2001144) await self._writer.wait_closed()
(TaskRunner pid=2001144) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 364, in wait_closed
(TaskRunner pid=2001144) await self._protocol._get_close_waiter(self)
(TaskRunner pid=2001144) BrokenPipeError: [Errno 32] Broken pipe
main_ppo exit code: 0
[2026-07-02 06:18:06] Finished chemistry grpo
[2026-07-02 06:18:06] Starting physics grpo
Experiment: qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8
Dataset: physics
Method: grpo
Config: baseline_grpo
Model: Qwen/Qwen3-4B
Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet
Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet
Ray tmp: /tmp/ray_q3g_physics_grpo_Qwen3_4B
2026-07-02 06:18:07,837 INFO scripts.py:1364 -- Did not find any active Ray processes.
Experiment: qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8
Config: baseline_grpo
Task: datasets/sciknoweval/physics
Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-GRPO-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_physics_grpo_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/physics +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.actor.policy_loss.loss_mode=vanilla actor_rollout_ref.actor.kl_loss_coef=0.0
ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_physics_grpo_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/physics', 'EXPERIMENT': 'qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}}
2026-07-02 06:18:17,488 INFO worker.py:2007 -- Started a local Ray instance.
/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
warnings.warn(
*** SIGTERM received at time=1782973099 on cpu 1 ***
PC: @ 0x7836548caf16 (unknown) (unknown)
@ 0x783654881050 (unknown) (unknown)
[2026-07-02 06:18:19,228 E 2021347 2021347] logging.cc:474: *** SIGTERM received at time=1782973099 on cpu 1 ***
[2026-07-02 06:18:19,228 E 2021347 2021347] logging.cc:474: PC: @ 0x7836548caf16 (unknown) (unknown)
[2026-07-02 06:18:19,228 E 2021347 2021347] logging.cc:474: @ 0x783654881050 (unknown) (unknown)
Reference in New Issue Copy Permalink