[2026-07-02 06:18:50] Starting L2T batch32 Qwen3-4B GRPO/RLSD_TR dataset-first queue [2026-07-02 06:18:50] Commit: ee6a667 [2026-07-02 06:18:50] Settings: model=Qwen/Qwen3-4B steps=100 train_batch=32 rlsd_tr_batch=32 rollout=8 mini_batch=8 tr_mix=0.1 max_model_len=10240 vllm=0.8 [2026-07-02 06:18:50] Skipping chemistry grpo; completed by previous queue [2026-07-02 06:18:50] Starting chemistry rlsd_tr Experiment: qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Dataset: chemistry Method: rlsd_tr Config: rlsd Model: Qwen/Qwen3-4B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet Ray tmp: /tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_4B 2026-07-02 06:18:52,310 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Config: rlsd Task: datasets/sciknoweval/chemistry Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-RLSD-TR-GRPO-matched-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/chemistry +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.model.use_fused_kernels=False actor_rollout_ref.actor.policy_loss.loss_mode=rlsd actor_rollout_ref.actor.self_distillation.token_reweight_lambda=0.5 actor_rollout_ref.actor.self_distillation.token_reweight_eps_w=0.2 actor_rollout_ref.actor.self_distillation.token_reweight_decay_steps=0 actor_rollout_ref.actor.self_distillation.teacher_regularization=trust-region actor_rollout_ref.actor.self_distillation.teacher_update_rate=0.1 actor_rollout_ref.actor.self_distillation.max_reprompt_len=10240 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/chemistry', 'EXPERIMENT': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 06:19:01,985 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2026458) TaskRunner hostname: mole-workspace-rw, PID: 2026458 (TaskRunner pid=2026458) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2026458) 'calculate_entropy': False, (TaskRunner pid=2026458) 'calculate_sum_pi_squared': False, (TaskRunner pid=2026458) 'checkpoint': {'_target_': (TaskRunner pid=2026458) 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2026458) 'async_save': False, (TaskRunner pid=2026458) 'load_contents': ['model', (TaskRunner pid=2026458) 'optimizer', (TaskRunner pid=2026458) 'extra'], (TaskRunner pid=2026458) 'save_contents': ['model', (TaskRunner pid=2026458) 'optimizer', (TaskRunner pid=2026458) 'extra']}, (TaskRunner pid=2026458) 'clip_ratio': 0.2, (TaskRunner pid=2026458) 'clip_ratio_c': 3.0, (TaskRunner pid=2026458) 'clip_ratio_high': 0.28, (TaskRunner pid=2026458) 'clip_ratio_low': 0.2, (TaskRunner pid=2026458) 'data_loader_seed': 42, (TaskRunner pid=2026458) 'entropy_checkpointing': False, (TaskRunner pid=2026458) 'entropy_coeff': 0, (TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2026458) 'freeze_vision_tower': False, (TaskRunner pid=2026458) 'fsdp_config': (TaskRunner pid=2026458) {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2026458) 'dtype': 'bfloat16', (TaskRunner pid=2026458) 'entropy_checkpointing': False, (TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2026458) 'forward_only': False, (TaskRunner pid=2026458) 'forward_prefetch': False, (TaskRunner pid=2026458) 'fsdp_size': -1, (TaskRunner pid=2026458) 'full_determinism': False, (TaskRunner pid=2026458) 'model_dtype': 'fp32', (TaskRunner pid=2026458) 'offload_policy': False, (TaskRunner pid=2026458) 'optimizer_offload': False, (TaskRunner pid=2026458) 'param_offload': False, (TaskRunner pid=2026458) 'reshard_after_forward': True, (TaskRunner pid=2026458) 'seed': 42, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_orig_params': False, (TaskRunner pid=2026458) 'use_torch_compile': True, (TaskRunner pid=2026458) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2026458) 'grad_clip': 1.0, (TaskRunner pid=2026458) 'kl_loss_coef': 0.001, (TaskRunner pid=2026458) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2026458) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2026458) 'loss_scale_factor': None, (TaskRunner pid=2026458) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2026458) 'betas': [0.9, 0.999], (TaskRunner pid=2026458) 'clip_grad': 1.0, (TaskRunner pid=2026458) 'lr': 1e-06, (TaskRunner pid=2026458) 'lr_scheduler_type': 'constant', (TaskRunner pid=2026458) 'lr_warmup_steps': 10, (TaskRunner pid=2026458) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2026458) 'min_lr_ratio': 0.0, (TaskRunner pid=2026458) 'num_cycles': 0.5, (TaskRunner pid=2026458) 'optimizer': 'AdamW', (TaskRunner pid=2026458) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2026458) 'override_optimizer_config': None, (TaskRunner pid=2026458) 'total_training_steps': -1, (TaskRunner pid=2026458) 'warmup_style': None, (TaskRunner pid=2026458) 'weight_decay': 0.01}, (TaskRunner pid=2026458) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2026458) 'clip_cov_lb': 1.0, (TaskRunner pid=2026458) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2026458) 'clip_cov_ub': 5.0, (TaskRunner pid=2026458) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2026458) 'loss_mode': 'rlsd', (TaskRunner pid=2026458) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2026458) 'ppo_epochs': 1, (TaskRunner pid=2026458) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2026458) 'ppo_micro_batch_size': None, (TaskRunner pid=2026458) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2026458) 'ppo_mini_batch_size': 8, (TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'all_ranks': False, (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'ranks': [], (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'tool': None, (TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2026458) 'analysis': True, (TaskRunner pid=2026458) 'contents': [], (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'level': 'level0'}, (TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'discrete': False}, (TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2026458) 'step_end': None, (TaskRunner pid=2026458) 'step_start': 0}, (TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2026458) 'rollout_n': 8, (TaskRunner pid=2026458) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2026458) 'mode': 'disabled', (TaskRunner pid=2026458) 'record_file': None, (TaskRunner pid=2026458) 'replay_file': None}, (TaskRunner pid=2026458) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2026458) 'alpha': 0.5, (TaskRunner pid=2026458) 'distillation_add_tail': True, (TaskRunner pid=2026458) 'distillation_topk': 100, (TaskRunner pid=2026458) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2026458) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2026458) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'loss_weight': 0.0, (TaskRunner pid=2026458) 'mask': 'all'}, (TaskRunner pid=2026458) 'feedback_template': '\n' (TaskRunner pid=2026458) 'The ' (TaskRunner pid=2026458) 'following ' (TaskRunner pid=2026458) 'is ' (TaskRunner pid=2026458) 'feedback ' (TaskRunner pid=2026458) 'from ' (TaskRunner pid=2026458) 'your ' (TaskRunner pid=2026458) 'unsuccessful ' (TaskRunner pid=2026458) 'earlier ' (TaskRunner pid=2026458) 'attempt:\n' (TaskRunner pid=2026458) '\n' (TaskRunner pid=2026458) '{feedback_raw}', (TaskRunner pid=2026458) 'full_logit_distillation': True, (TaskRunner pid=2026458) 'include_environment_feedback': True, (TaskRunner pid=2026458) 'is_clip': 2, (TaskRunner pid=2026458) 'max_reprompt_len': 10240, (TaskRunner pid=2026458) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2026458) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2026458) '\n' (TaskRunner pid=2026458) 'Correctly ' (TaskRunner pid=2026458) 'solve ' (TaskRunner pid=2026458) 'the ' (TaskRunner pid=2026458) 'original ' (TaskRunner pid=2026458) 'question.', (TaskRunner pid=2026458) 'reprompt_truncation': 'right', (TaskRunner pid=2026458) 'solution_template': '\n' (TaskRunner pid=2026458) 'Correct ' (TaskRunner pid=2026458) 'solution:\n' (TaskRunner pid=2026458) '\n' (TaskRunner pid=2026458) '{successful_previous_attempt}', (TaskRunner pid=2026458) 'srpo_dynamic_weighting': False, (TaskRunner pid=2026458) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2026458) 'success_reward_threshold': 0.5, (TaskRunner pid=2026458) 'teacher_regularization': 'trust-region', (TaskRunner pid=2026458) 'teacher_update_rate': 0.1, (TaskRunner pid=2026458) 'token_reweight_decay_steps': 0, (TaskRunner pid=2026458) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2026458) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2026458) 'shuffle': False, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2026458) 'tau_neg': 1.05, (TaskRunner pid=2026458) 'tau_pos': 1.0, (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_dynamic_bsz': False, (TaskRunner pid=2026458) 'use_fused_kernels': False, (TaskRunner pid=2026458) 'use_kl_loss': False, (TaskRunner pid=2026458) 'use_prefix_grouper': False, (TaskRunner pid=2026458) 'use_remove_padding': True, (TaskRunner pid=2026458) 'use_torch_compile': True}, (TaskRunner pid=2026458) 'hybrid_engine': True, (TaskRunner pid=2026458) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2026458) 'custom_chat_template': None, (TaskRunner pid=2026458) 'enable_activation_offload': False, (TaskRunner pid=2026458) 'enable_gradient_checkpointing': True, (TaskRunner pid=2026458) 'exclude_modules': None, (TaskRunner pid=2026458) 'external_lib': None, (TaskRunner pid=2026458) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2026458) 'hf_config_path': None, (TaskRunner pid=2026458) 'lora_adapter_path': None, (TaskRunner pid=2026458) 'lora_alpha': 16, (TaskRunner pid=2026458) 'lora_rank': 0, (TaskRunner pid=2026458) 'override_config': {}, (TaskRunner pid=2026458) 'path': 'Qwen/Qwen3-4B', (TaskRunner pid=2026458) 'target_modules': 'all-linear', (TaskRunner pid=2026458) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2026458) 'num_shards': 4}, (TaskRunner pid=2026458) 'tokenizer_path': None, (TaskRunner pid=2026458) 'trust_remote_code': True, (TaskRunner pid=2026458) 'use_fused_kernels': False, (TaskRunner pid=2026458) 'use_liger': False, (TaskRunner pid=2026458) 'use_remove_padding': True, (TaskRunner pid=2026458) 'use_shm': False}, (TaskRunner pid=2026458) 'nccl_timeout': 600, (TaskRunner pid=2026458) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2026458) 'entropy_checkpointing': False, (TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2026458) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2026458) 'dtype': 'bfloat16', (TaskRunner pid=2026458) 'entropy_checkpointing': False, (TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2026458) 'forward_only': True, (TaskRunner pid=2026458) 'forward_prefetch': False, (TaskRunner pid=2026458) 'fsdp_size': -1, (TaskRunner pid=2026458) 'full_determinism': False, (TaskRunner pid=2026458) 'model_dtype': 'fp32', (TaskRunner pid=2026458) 'offload_policy': False, (TaskRunner pid=2026458) 'optimizer_offload': False, (TaskRunner pid=2026458) 'param_offload': False, (TaskRunner pid=2026458) 'reshard_after_forward': True, (TaskRunner pid=2026458) 'seed': 42, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_orig_params': False, (TaskRunner pid=2026458) 'use_torch_compile': True, (TaskRunner pid=2026458) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2026458) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2026458) 'log_prob_micro_batch_size': None, (TaskRunner pid=2026458) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2026458) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'all_ranks': False, (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'ranks': [], (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'tool': None, (TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2026458) 'analysis': True, (TaskRunner pid=2026458) 'contents': [], (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'level': 'level0'}, (TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'discrete': False}, (TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2026458) 'step_end': None, (TaskRunner pid=2026458) 'step_start': 0}, (TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2026458) 'rollout_n': 8, (TaskRunner pid=2026458) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2026458) 'mode': 'disabled', (TaskRunner pid=2026458) 'record_file': None, (TaskRunner pid=2026458) 'replay_file': None}, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_torch_compile': True}, (TaskRunner pid=2026458) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2026458) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2026458) 'agent_loop_config_path': None, (TaskRunner pid=2026458) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2026458) 'name': None, (TaskRunner pid=2026458) 'path': None}, (TaskRunner pid=2026458) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2026458) 'num_workers': 8}, (TaskRunner pid=2026458) 'calculate_log_probs': True, (TaskRunner pid=2026458) 'cudagraph_capture_sizes': None, (TaskRunner pid=2026458) 'data_parallel_size': 1, (TaskRunner pid=2026458) 'disable_log_stats': True, (TaskRunner pid=2026458) 'do_sample': True, (TaskRunner pid=2026458) 'dtype': 'bfloat16', (TaskRunner pid=2026458) 'enable_chunked_prefill': True, (TaskRunner pid=2026458) 'enable_prefix_caching': True, (TaskRunner pid=2026458) 'enable_rollout_routing_replay': False, (TaskRunner pid=2026458) 'enforce_eager': False, (TaskRunner pid=2026458) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2026458) 'expert_parallel_size': 1, (TaskRunner pid=2026458) 'free_cache_engine': True, (TaskRunner pid=2026458) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2026458) 'ignore_eos': False, (TaskRunner pid=2026458) 'layered_summon': False, (TaskRunner pid=2026458) 'load_format': 'dummy', (TaskRunner pid=2026458) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2026458) 'log_prob_micro_batch_size': None, (TaskRunner pid=2026458) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2026458) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2026458) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2026458) 'max_model_len': 10240, (TaskRunner pid=2026458) 'max_num_batched_tokens': 10240, (TaskRunner pid=2026458) 'max_num_seqs': 1024, (TaskRunner pid=2026458) 'mode': 'async', (TaskRunner pid=2026458) 'multi_stage_wake_up': False, (TaskRunner pid=2026458) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'format': 'hermes', (TaskRunner pid=2026458) 'interaction_config_path': None, (TaskRunner pid=2026458) 'max_assistant_turns': None, (TaskRunner pid=2026458) 'max_parallel_calls': 1, (TaskRunner pid=2026458) 'max_tool_response_length': 256, (TaskRunner pid=2026458) 'max_user_turns': None, (TaskRunner pid=2026458) 'num_repeat_rollouts': None, (TaskRunner pid=2026458) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2026458) 'tool_config_path': None, (TaskRunner pid=2026458) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2026458) 'use_inference_chat_template': False}, (TaskRunner pid=2026458) 'n': 8, (TaskRunner pid=2026458) 'name': 'vllm', (TaskRunner pid=2026458) 'over_sample_rate': 0, (TaskRunner pid=2026458) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'all_ranks': False, (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'ranks': [], (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'tool': None, (TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2026458) 'analysis': True, (TaskRunner pid=2026458) 'contents': [], (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'level': 'level0'}, (TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'discrete': False}, (TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2026458) 'step_end': None, (TaskRunner pid=2026458) 'step_start': 0}, (TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2026458) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2026458) 'port': 9090, (TaskRunner pid=2026458) 'served_model_name': 'Qwen/Qwen3-4B'}, (TaskRunner pid=2026458) 'prompt_length': 2048, (TaskRunner pid=2026458) 'quantization': None, (TaskRunner pid=2026458) 'quantization_config_file': None, (TaskRunner pid=2026458) 'response_length': 8192, (TaskRunner pid=2026458) 'scheduling_policy': 'fcfs', (TaskRunner pid=2026458) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2026458) 'skip_rollout': False, (TaskRunner pid=2026458) 'skip_tokenizer_init': True, (TaskRunner pid=2026458) 'temperature': 1.0, (TaskRunner pid=2026458) 'tensor_model_parallel_size': 2, (TaskRunner pid=2026458) 'top_k': -1, (TaskRunner pid=2026458) 'top_p': 1.0, (TaskRunner pid=2026458) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2026458) 'backend': None, (TaskRunner pid=2026458) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2026458) 'token2text': False}, (TaskRunner pid=2026458) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2026458) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2026458) 'do_sample': True, (TaskRunner pid=2026458) 'n': 16, (TaskRunner pid=2026458) 'temperature': 0.6, (TaskRunner pid=2026458) 'top_k': -1, (TaskRunner pid=2026458) 'top_p': 0.95}}}, (TaskRunner pid=2026458) 'algorithm': (TaskRunner pid=2026458) {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2026458) 'adv_estimator': 'grpo', (TaskRunner pid=2026458) 'gamma': 1.0, (TaskRunner pid=2026458) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2026458) 'horizon': 10000, (TaskRunner pid=2026458) 'kl_coef': 0.001, (TaskRunner pid=2026458) 'target_kl': 0.1, (TaskRunner pid=2026458) 'type': 'fixed'}, (TaskRunner pid=2026458) 'kl_penalty': 'kl', (TaskRunner pid=2026458) 'lam': 1.0, (TaskRunner pid=2026458) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2026458) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2026458) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2026458) 'loss_type': 'ppo_clip', (TaskRunner pid=2026458) 'rollout_is': 'token', (TaskRunner pid=2026458) 'rollout_is_batch_normalize': False, (TaskRunner pid=2026458) 'rollout_is_threshold': 2.0, (TaskRunner pid=2026458) 'rollout_rs': None, (TaskRunner pid=2026458) 'rollout_rs_threshold': None}, (TaskRunner pid=2026458) 'use_kl_in_reward': False, (TaskRunner pid=2026458) 'use_pf_ppo': False}, (TaskRunner pid=2026458) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2026458) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2026458) 'async_save': False, (TaskRunner pid=2026458) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2026458) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2026458) 'cliprange_value': 0.5, (TaskRunner pid=2026458) 'data_loader_seed': 42, (TaskRunner pid=2026458) 'enable': None, (TaskRunner pid=2026458) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2026458) 'forward_micro_batch_size': None, (TaskRunner pid=2026458) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2026458) 'grad_clip': 1.0, (TaskRunner pid=2026458) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2026458) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2026458) 'enable_activation_offload': False, (TaskRunner pid=2026458) 'enable_gradient_checkpointing': True, (TaskRunner pid=2026458) 'external_lib': None, (TaskRunner pid=2026458) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2026458) 'dtype': 'bfloat16', (TaskRunner pid=2026458) 'entropy_checkpointing': False, (TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2026458) 'forward_only': False, (TaskRunner pid=2026458) 'forward_prefetch': False, (TaskRunner pid=2026458) 'fsdp_size': -1, (TaskRunner pid=2026458) 'full_determinism': False, (TaskRunner pid=2026458) 'model_dtype': 'fp32', (TaskRunner pid=2026458) 'offload_policy': False, (TaskRunner pid=2026458) 'optimizer_offload': False, (TaskRunner pid=2026458) 'param_offload': False, (TaskRunner pid=2026458) 'reshard_after_forward': True, (TaskRunner pid=2026458) 'seed': 42, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_orig_params': False, (TaskRunner pid=2026458) 'use_torch_compile': True, (TaskRunner pid=2026458) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2026458) 'lora_alpha': 16, (TaskRunner pid=2026458) 'lora_rank': 0, (TaskRunner pid=2026458) 'override_config': {}, (TaskRunner pid=2026458) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2026458) 'target_modules': 'all-linear', (TaskRunner pid=2026458) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2026458) 'tokenizer_path': 'Qwen/Qwen3-4B', (TaskRunner pid=2026458) 'trust_remote_code': True, (TaskRunner pid=2026458) 'use_remove_padding': False, (TaskRunner pid=2026458) 'use_shm': False}, (TaskRunner pid=2026458) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2026458) 'betas': [0.9, 0.999], (TaskRunner pid=2026458) 'clip_grad': 1.0, (TaskRunner pid=2026458) 'lr': 1e-05, (TaskRunner pid=2026458) 'lr_scheduler_type': 'constant', (TaskRunner pid=2026458) 'lr_warmup_steps': -1, (TaskRunner pid=2026458) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2026458) 'min_lr_ratio': 0.0, (TaskRunner pid=2026458) 'num_cycles': 0.5, (TaskRunner pid=2026458) 'optimizer': 'AdamW', (TaskRunner pid=2026458) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2026458) 'override_optimizer_config': None, (TaskRunner pid=2026458) 'total_training_steps': -1, (TaskRunner pid=2026458) 'warmup_style': None, (TaskRunner pid=2026458) 'weight_decay': 0.01}, (TaskRunner pid=2026458) 'ppo_epochs': 1, (TaskRunner pid=2026458) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2026458) 'ppo_micro_batch_size': None, (TaskRunner pid=2026458) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2026458) 'ppo_mini_batch_size': 8, (TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'all_ranks': False, (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'ranks': [], (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'tool': None, (TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2026458) 'analysis': True, (TaskRunner pid=2026458) 'contents': [], (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'level': 'level0'}, (TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'discrete': False}, (TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2026458) 'step_end': None, (TaskRunner pid=2026458) 'step_start': 0}, (TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2026458) 'rollout_n': 8, (TaskRunner pid=2026458) 'shuffle': False, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_dynamic_bsz': False}, (TaskRunner pid=2026458) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2026458) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2026458) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2026458) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2026458) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2026458) 'dataloader_num_workers': 8, (TaskRunner pid=2026458) 'filter_overlong_prompts': True, (TaskRunner pid=2026458) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2026458) 'image_key': 'images', (TaskRunner pid=2026458) 'image_patch_size': 14, (TaskRunner pid=2026458) 'max_prompt_length': 2048, (TaskRunner pid=2026458) 'max_response_length': 8192, (TaskRunner pid=2026458) 'prompt_key': 'prompt', (TaskRunner pid=2026458) 'return_full_prompt': False, (TaskRunner pid=2026458) 'return_multi_modal_inputs': True, (TaskRunner pid=2026458) 'return_raw_chat': True, (TaskRunner pid=2026458) 'return_raw_input_ids': False, (TaskRunner pid=2026458) 'reward_fn_key': 'data_source', (TaskRunner pid=2026458) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2026458) 'seed': None, (TaskRunner pid=2026458) 'shuffle': True, (TaskRunner pid=2026458) 'tokenizer': None, (TaskRunner pid=2026458) 'tool_config_path': None, (TaskRunner pid=2026458) 'train_batch_size': 32, (TaskRunner pid=2026458) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet'], (TaskRunner pid=2026458) 'train_max_samples': 3200, (TaskRunner pid=2026458) 'truncation': 'error', (TaskRunner pid=2026458) 'trust_remote_code': True, (TaskRunner pid=2026458) 'use_shm': False, (TaskRunner pid=2026458) 'val_batch_size': None, (TaskRunner pid=2026458) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet'], (TaskRunner pid=2026458) 'val_max_samples': -1, (TaskRunner pid=2026458) 'validation_shuffle': False, (TaskRunner pid=2026458) 'video_key': 'videos'}, (TaskRunner pid=2026458) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2026458) 'cuda-memory-usage': 'true', (TaskRunner pid=2026458) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2026458) 'capture-range-end': None, (TaskRunner pid=2026458) 'cuda-graph-trace': 'graph', (TaskRunner pid=2026458) 'cuda-memory-usage': 'true', (TaskRunner pid=2026458) 'kill': 'none', (TaskRunner pid=2026458) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2026458) 'torch_memory': {'context': 'all', (TaskRunner pid=2026458) 'kw_args': {}, (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'stacks': 'all', (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2026458) 'profile_continuous_steps': False, (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'steps': None, (TaskRunner pid=2026458) 'tool': None}, (TaskRunner pid=2026458) 'max_model_len': 10240, (TaskRunner pid=2026458) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_4B', (TaskRunner pid=2026458) 'include_dashboard': False, (TaskRunner pid=2026458) 'num_cpus': None, (TaskRunner pid=2026458) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2026458) 'TASK': 'datasets/sciknoweval/chemistry', (TaskRunner pid=2026458) 'USER': 'root'}}}, (TaskRunner pid=2026458) 'timeline_json_file': None}, (TaskRunner pid=2026458) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2026458) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2026458) 'name': 'custom_reward_manager', (TaskRunner pid=2026458) 'path': None}, (TaskRunner pid=2026458) 'name': 'naive', (TaskRunner pid=2026458) 'source': 'register'}, (TaskRunner pid=2026458) 'reward_model': {'enable': False, (TaskRunner pid=2026458) 'enable_resource_pool': False, (TaskRunner pid=2026458) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2026458) 'launch_reward_fn_async': False, (TaskRunner pid=2026458) 'max_length': None, (TaskRunner pid=2026458) 'micro_batch_size': None, (TaskRunner pid=2026458) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2026458) 'model': {'external_lib': None, (TaskRunner pid=2026458) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2026458) 'forward_prefetch': False, (TaskRunner pid=2026458) 'fsdp_size': -1, (TaskRunner pid=2026458) 'param_offload': False, (TaskRunner pid=2026458) 'reshard_after_forward': True, (TaskRunner pid=2026458) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2026458) 'input_tokenizer': 'Qwen/Qwen3-4B', (TaskRunner pid=2026458) 'override_config': {}, (TaskRunner pid=2026458) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2026458) 'trust_remote_code': False, (TaskRunner pid=2026458) 'use_fused_kernels': False, (TaskRunner pid=2026458) 'use_remove_padding': False, (TaskRunner pid=2026458) 'use_shm': False}, (TaskRunner pid=2026458) 'n_gpus_per_node': 8, (TaskRunner pid=2026458) 'nnodes': 0, (TaskRunner pid=2026458) 'num_workers': 1, (TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'all_ranks': False, (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'ranks': [], (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'tool': None, (TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2026458) 'analysis': True, (TaskRunner pid=2026458) 'contents': [], (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'level': 'level0'}, (TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'discrete': False}, (TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2026458) 'step_end': None, (TaskRunner pid=2026458) 'step_start': 0}, (TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2026458) 'reward_loop_class_name': None, (TaskRunner pid=2026458) 'reward_loop_module_path': None, (TaskRunner pid=2026458) 'reward_loop_source': 'register', (TaskRunner pid=2026458) 'reward_manager': 'naive', (TaskRunner pid=2026458) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2026458) 'cudagraph_capture_sizes': None, (TaskRunner pid=2026458) 'data_parallel_size': 1, (TaskRunner pid=2026458) 'disable_log_stats': True, (TaskRunner pid=2026458) 'dtype': 'bfloat16', (TaskRunner pid=2026458) 'enable_chunked_prefill': True, (TaskRunner pid=2026458) 'enable_prefix_caching': True, (TaskRunner pid=2026458) 'enforce_eager': True, (TaskRunner pid=2026458) 'engine_kwargs': {}, (TaskRunner pid=2026458) 'expert_parallel_size': 1, (TaskRunner pid=2026458) 'free_cache_engine': True, (TaskRunner pid=2026458) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2026458) 'limit_images': None, (TaskRunner pid=2026458) 'load_format': 'auto', (TaskRunner pid=2026458) 'max_model_len': None, (TaskRunner pid=2026458) 'max_num_batched_tokens': 8192, (TaskRunner pid=2026458) 'max_num_seqs': 1024, (TaskRunner pid=2026458) 'name': '???', (TaskRunner pid=2026458) 'prompt_length': 2048, (TaskRunner pid=2026458) 'response_length': 2048, (TaskRunner pid=2026458) 'skip_tokenizer_init': False, (TaskRunner pid=2026458) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2026458) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2026458) 'memory_limit_mb': 1024, (TaskRunner pid=2026458) 'url': None}, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_dynamic_bsz': False, (TaskRunner pid=2026458) 'use_reward_loop': False}, (TaskRunner pid=2026458) 'trainer': {'balance_batch': True, (TaskRunner pid=2026458) 'critic_warmup': 0, (TaskRunner pid=2026458) 'default_hdfs_dir': None, (TaskRunner pid=2026458) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2026458) 'del_local_ckpt_after_load': False, (TaskRunner pid=2026458) 'device': 'cuda', (TaskRunner pid=2026458) 'esi_redundant_time': 0, (TaskRunner pid=2026458) 'experiment_name': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2026458) 'group_name': 'QWEN3-RLSD-TR-GRPO-matched-generalization', (TaskRunner pid=2026458) 'log_val_generations': 0, (TaskRunner pid=2026458) 'logger': ['console', 'wandb'], (TaskRunner pid=2026458) 'max_actor_ckpt_to_keep': 1, (TaskRunner pid=2026458) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2026458) 'n_gpus_per_node': 8, (TaskRunner pid=2026458) 'nnodes': 1, (TaskRunner pid=2026458) 'project_name': 'SDPO-root', (TaskRunner pid=2026458) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2026458) 'resume_from_path': None, (TaskRunner pid=2026458) 'resume_mode': 'auto', (TaskRunner pid=2026458) 'rollout_data_dir': None, (TaskRunner pid=2026458) 'save_freq': 10, (TaskRunner pid=2026458) 'test_freq': 10, (TaskRunner pid=2026458) 'total_epochs': 30, (TaskRunner pid=2026458) 'total_training_steps': 100, (TaskRunner pid=2026458) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2026458) 'val_before_train': False, (TaskRunner pid=2026458) 'val_only': False, (TaskRunner pid=2026458) 'validation_data_dir': None}, (TaskRunner pid=2026458) 'transfer_queue': {'enable': False}, (TaskRunner pid=2026458) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/chemistry', (TaskRunner pid=2026458) 'dir': '/users/root/SDPO', (TaskRunner pid=2026458) 'log_dir': '/users/root/output', (TaskRunner pid=2026458) 'task': 'datasets/sciknoweval/chemistry'}} (TaskRunner pid=2026458) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2026458) use_critic=need_critic(config), (TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2026458) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2026458) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /Qwen/Qwen3-4B/resolve/main/tokenizer_config.json (Caused by SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1010)')))"), '(Request ID: eb9682da-997b-4a49-aefd-e3ec8c865f3b)')' thrown while requesting HEAD https://huggingface.co/Qwen/Qwen3-4B/resolve/main/tokenizer_config.json (TaskRunner pid=2026458) WARNING:2026-07-02 06:19:07,289:'(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /Qwen/Qwen3-4B/resolve/main/tokenizer_config.json (Caused by SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1010)')))"), '(Request ID: eb9682da-997b-4a49-aefd-e3ec8c865f3b)')' thrown while requesting HEAD https://huggingface.co/Qwen/Qwen3-4B/resolve/main/tokenizer_config.json (TaskRunner pid=2026458) Retrying in 1s [Retry 1/5]. (TaskRunner pid=2026458) WARNING:2026-07-02 06:19:07,290:Retrying in 1s [Retry 1/5]. (TaskRunner pid=2026458) Using dataset class: RLHFDataset (TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2026458) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2026458) dataset len: 1890 (TaskRunner pid=2026458) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2026458) WARNING:2026-07-02 06:19:12,124:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2026458) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/1890 [00:00 (TaskRunner pid=2026458) bind role actor_rollout_ref method chat_completion to class .WorkerDict'> (TaskRunner pid=2026458) bind role actor_rollout_ref method generate to class .WorkerDict'> (TaskRunner pid=2026458) bind role actor_rollout_ref method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2026458) bind role actor_rollout_ref method sleep to class .WorkerDict'> (TaskRunner pid=2026458) bind role actor_rollout_ref method wake_up to class .WorkerDict'> (TaskRunner pid=2026458) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 210/210 [00:00<00:00, 301.55 examples/s] (TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2026458) self.use_critic = need_critic(self.config) (WorkerDict pid=2026872) [W702 06:19:21.430329774 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:52047 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2026871) [Gloo] Rank 3 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2026873) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2026873) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2026869) [W702 06:19:22.501122826 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:52047 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2026868) Model config after override: Qwen3Config { (WorkerDict pid=2026868) "architectures": [ (WorkerDict pid=2026868) "Qwen3ForCausalLM" (WorkerDict pid=2026868) ], (WorkerDict pid=2026868) "attention_bias": false, (WorkerDict pid=2026868) "attention_dropout": 0.0, (WorkerDict pid=2026868) "dtype": "bfloat16", (WorkerDict pid=2026868) "eos_token_id": 151645, (WorkerDict pid=2026868) "head_dim": 128, (WorkerDict pid=2026868) "hidden_act": "silu", (WorkerDict pid=2026868) "hidden_size": 2560, (WorkerDict pid=2026868) "initializer_range": 0.02, (WorkerDict pid=2026868) "intermediate_size": 9728, (WorkerDict pid=2026868) "layer_types": [ (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention" (WorkerDict pid=2026868) ], (WorkerDict pid=2026868) "max_position_embeddings": 40960, (WorkerDict pid=2026868) "max_window_layers": 36, (WorkerDict pid=2026868) "model_type": "qwen3", (WorkerDict pid=2026868) "num_attention_heads": 32, (WorkerDict pid=2026868) "num_hidden_layers": 36, (WorkerDict pid=2026868) "num_key_value_heads": 8, (WorkerDict pid=2026868) "pad_token_id": 151643, (WorkerDict pid=2026868) "rms_norm_eps": 1e-06, (WorkerDict pid=2026868) "rope_scaling": null, (WorkerDict pid=2026868) "rope_theta": 1000000, (WorkerDict pid=2026868) "sliding_window": null, (WorkerDict pid=2026868) "tie_word_embeddings": true, (WorkerDict pid=2026868) "transformers_version": "4.57.1", (WorkerDict pid=2026868) "use_cache": true, (WorkerDict pid=2026868) "use_sliding_window": false, (WorkerDict pid=2026868) "vocab_size": 151936 (WorkerDict pid=2026868) } (WorkerDict pid=2026868) (WorkerDict pid=2026873) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2026873) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2026873) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2026873) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2026868) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2026871) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2026871) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2026870) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.94s/it] [repeated 12x across cluster] (WorkerDict pid=2026870) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.97s/it] [repeated 4x across cluster] (WorkerDict pid=2026868) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 3.00s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:06<00:00, 2.01s/it] [repeated 2x across cluster] (WorkerDict pid=2026868) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2026868) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2026868) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2026868) Actor use_remove_padding=True (WorkerDict pid=2026868) Actor use_fused_kernels=False (WorkerDict pid=2026868) Actor use_prefix_grouper=False (WorkerDict pid=2026871) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2026871) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2026871) [Gloo] Rank 1 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2026871) [Gloo] Rank 1 is connected to 1 peer ranks. Expected number of connected peer ranks is : 1 (WorkerDict pid=2026871) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 (WorkerDict pid=2026871) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2026871) warnings.warn( (WorkerDict pid=2026869) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2026869) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2026868) reference model: Qwen/Qwen3-4B (WorkerDict pid=2026871) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 8x across cluster] (WorkerDict pid=2026871) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 8x across cluster] (WorkerDict pid=2026868) Model config after override: Qwen3Config { (WorkerDict pid=2026868) "architectures": [ (WorkerDict pid=2026868) "Qwen3ForCausalLM" (WorkerDict pid=2026868) ], (WorkerDict pid=2026868) "attention_bias": false, (WorkerDict pid=2026868) "attention_dropout": 0.0, (WorkerDict pid=2026868) "dtype": "bfloat16", (WorkerDict pid=2026868) "eos_token_id": 151645, (WorkerDict pid=2026868) "head_dim": 128, (WorkerDict pid=2026868) "hidden_act": "silu", (WorkerDict pid=2026868) "hidden_size": 2560, (WorkerDict pid=2026868) "initializer_range": 0.02, (WorkerDict pid=2026868) "intermediate_size": 9728, (WorkerDict pid=2026868) "layer_types": [ (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention" (WorkerDict pid=2026868) ], (WorkerDict pid=2026868) "max_position_embeddings": 40960, (WorkerDict pid=2026868) "max_window_layers": 36, (WorkerDict pid=2026868) "model_type": "qwen3", (WorkerDict pid=2026868) "num_attention_heads": 32, (WorkerDict pid=2026868) "num_hidden_layers": 36, (WorkerDict pid=2026868) "num_key_value_heads": 8, (WorkerDict pid=2026868) "pad_token_id": 151643, (WorkerDict pid=2026868) "rms_norm_eps": 1e-06, (WorkerDict pid=2026868) "rope_scaling": null, (WorkerDict pid=2026868) "rope_theta": 1000000, (WorkerDict pid=2026868) "sliding_window": null, (WorkerDict pid=2026868) "tie_word_embeddings": true, (WorkerDict pid=2026868) "transformers_version": "4.57.1", (WorkerDict pid=2026868) "use_cache": true, (WorkerDict pid=2026868) "use_sliding_window": false, (WorkerDict pid=2026868) "vocab_size": 151936 (WorkerDict pid=2026868) } (WorkerDict pid=2026868) (WorkerDict pid=2026871) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2026868) Ref use_remove_padding=True (WorkerDict pid=2026868) Ref use_fused_kernels=False (WorkerDict pid=2026868) Ref use_prefix_grouper=False (TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2026458) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2026874) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.90s/it] [repeated 12x across cluster] (WorkerDict pid=2026873) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.89s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.95s/it] [repeated 2x across cluster] (WorkerDict pid=2026874) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.95s/it] [repeated 4x across cluster] (vLLMHttpServer pid=2027750) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2027750) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2027750) ['serve', (vLLMHttpServer pid=2027750) 'Qwen/Qwen3-4B', (vLLMHttpServer pid=2027750) '--dtype', (vLLMHttpServer pid=2027750) 'bfloat16', (vLLMHttpServer pid=2027750) '--load_format', (vLLMHttpServer pid=2027750) 'dummy', (vLLMHttpServer pid=2027750) '--trust_remote_code', (vLLMHttpServer pid=2027750) '--max_model_len', (vLLMHttpServer pid=2027750) '40960', (vLLMHttpServer pid=2027750) '--max_num_seqs', (vLLMHttpServer pid=2027750) '1024', (vLLMHttpServer pid=2027750) '--enable_chunked_prefill', (vLLMHttpServer pid=2027750) '--max_num_batched_tokens', (vLLMHttpServer pid=2027750) '10240', (vLLMHttpServer pid=2027750) '--enable_prefix_caching', (vLLMHttpServer pid=2027750) '--enable_sleep_mode', (vLLMHttpServer pid=2027750) '--logprobs_mode', (vLLMHttpServer pid=2027750) 'processed_logprobs', (vLLMHttpServer pid=2027750) '--disable_custom_all_reduce', (vLLMHttpServer pid=2027750) '--gpu_memory_utilization', (vLLMHttpServer pid=2027750) '0.8', (vLLMHttpServer pid=2027750) '--disable_log_stats', (vLLMHttpServer pid=2027750) '--tensor_parallel_size', (vLLMHttpServer pid=2027750) '2', (vLLMHttpServer pid=2027750) '--seed', (vLLMHttpServer pid=2027750) '0', (vLLMHttpServer pid=2027750) '--override_generation_config', (vLLMHttpServer pid=2027750) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2027750) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2027750) '--hf_overrides', (vLLMHttpServer pid=2027750) '{}', (vLLMHttpServer pid=2027750) '--scheduling_policy', (vLLMHttpServer pid=2027750) 'fcfs'] (WorkerDict pid=2026872) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2026872) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (vLLMHttpServer pid=2027750) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2027750) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2027750) WARNING 07-02 06:20:18 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2026868) WARNING 07-02 06:20:26 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2027751) WARNING 07-02 06:20:19 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2026872) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2026868) 2026-07-02 06:20:40,724 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2026868) 2026-07-02 06:20:40,743 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (vLLMHttpServer pid=2027753) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 3x across cluster] (vLLMHttpServer pid=2027753) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 3x across cluster] (vLLMHttpServer pid=2027753) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2027753) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which molecule is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". This product contains a complex structure with a benzene ring substituted with a chlorine atom and a trifluoromethyl group (C(F)(F)F). The key part of the product is the "Cc2ccc(Cl)cc2C(F)(F)F" fragment, which suggests that the reactant must contain a similar structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Looking at the options: (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option A contains a ketone group (O=C) and a nitrogen-containing ring, but it does not match the specific substituents in the product. (TaskRunner pid=2026458) - Option B contains a chlorine atom and a nitrogen, but it does not include the trifluoromethyl group (C(F)(F)F). (TaskRunner pid=2026458) - Option C contains a trifluoromethyl group (C(F)(F)F) and a chlorine atom, and it includes a structure that matches the "Cc2ccc(Cl)cc2" part of the product. (TaskRunner pid=2026458) - Option D contains a trifluoromethyl group but lacks the chlorine atom in the correct position. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C is the only one that includes both the chlorine and the trifluoromethyl group in the correct positions, matching the structure of the product. Therefore, it is the correct reactant. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) C (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 0.0 (TaskRunner pid=2026458) [acc] 0.0 (TaskRunner pid=2026458) [pred] C (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2026458) step:10 - global_seqlen/min:17490 - global_seqlen/max:20901 - global_seqlen/minmax_diff:3411 - global_seqlen/balanced_min:19357 - global_seqlen/balanced_max:19367 - global_seqlen/mean:19364.75 - actor/entropy:0.24537719786167145 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4744035601615906 - training/rollout_probs_diff_mean:0.005785205401480198 - training/rollout_probs_diff_std:0.014700246043503284 - training/rollout_actor_probs_pearson_corr:0.9979208707809448 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:0.9998391270637512 - rollout_corr/rollout_is_ratio_fraction_high:1.1633588655968197e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.980152829783037e-05 - rollout_corr/rollout_is_max:2.010491132736206 - rollout_corr/rollout_is_min:0.23784410953521729 - rollout_corr/rollout_is_std:0.04395073652267456 - rollout_corr/rollout_is_eff_sample_size:0.9980714630669532 - rollout_corr/rollout_is_seq_mean:0.9998910427093506 - rollout_corr/rollout_is_seq_std:0.0027966259513050318 - rollout_corr/rollout_is_seq_max:1.0104008913040161 - rollout_corr/rollout_is_seq_min:0.9919784069061279 - rollout_corr/rollout_is_seq_max_deviation:0.010400891304016113 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3026330736465752) - rollout_corr/training_log_ppl:np.float64(0.26078601760673337) - rollout_corr/kl:np.float64(0.0013049291076470126) - rollout_corr/k3_kl:np.float64(0.0011964667813799679) - rollout_corr/rollout_ppl:np.float64(1.3009595507755876) - rollout_corr/rollout_log_ppl:np.float64(0.25948108817101456) - rollout_corr/log_ppl_diff:np.float64(0.0013049294357188046) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024894755915738642) - rollout_corr/log_ppl_diff_max:np.float64(0.017253637313842773) - rollout_corr/log_ppl_diff_min:np.float64(-0.008967161178588867) - rollout_corr/ppl_ratio:np.float64(1.0013103527016938) - rollout_corr/chi2_token:np.float64(0.002327824244275689) - rollout_corr/chi2_seq:np.float64(0.852935308823362) - actor/pg_loss:np.float64(0.00642693554982543) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010866172510759498) - actor/ppo_kl:np.float64(0.00015771930741692586) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(177288.61759452336) - self_distillation/token_reweight_w_std:np.float64(2583470.6754472964) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0025011864490807) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1041747338604182) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.5560944229364395) - perf/mfu/actor:np.float64(0.03574857500362935) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(141.2105484008789) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.4398809523809524) - val-aux/sciknoweval/reward/std@16:np.float64(0.2550574423136475) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5466857142857143) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.22323689062871058) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.3330666666666666) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.20976723560246902) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.44009047619047625) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.25489768792129186) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6363809523809525) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.16780774930381429) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.2502714285714286) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.14555290883567562) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.4538952380952381) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.209952783081799) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7015523809523809) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.1130187029836996) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.1943) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.087999239033555) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.4615809523809523) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.16058361739063698) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7451095238095238) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.06576006624725168) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.16135714285714284) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04477536267518606) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.46405714285714283) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11915383713937476) - val-aux/sciknoweval/score/mean@16:np.float64(0.4398809523809524) - val-aux/sciknoweval/score/std@16:np.float64(0.2550574423136475) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.5466857142857143) - val-aux/sciknoweval/score/best@2/std:np.float64(0.22323689062871058) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.3330666666666666) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.20976723560246902) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.44009047619047625) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.25489768792129186) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6363809523809525) - val-aux/sciknoweval/score/best@4/std:np.float64(0.16780774930381429) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.2502714285714286) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.14555290883567562) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.4538952380952381) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.209952783081799) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7015523809523809) - val-aux/sciknoweval/score/best@8/std:np.float64(0.1130187029836996) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.1943) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.087999239033555) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.4615809523809523) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.16058361739063698) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7451095238095238) - val-aux/sciknoweval/score/best@16/std:np.float64(0.06576006624725168) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.16135714285714284) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04477536267518606) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.46405714285714283) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11915383713937476) - val-core/sciknoweval/acc/mean@16:np.float64(0.4398809523809524) - val-aux/sciknoweval/acc/std@16:np.float64(0.2550574423136475) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.5466857142857143) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.22323689062871058) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.3330666666666666) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.20976723560246902) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.44009047619047625) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.25489768792129186) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6363809523809525) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.16780774930381429) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.2502714285714286) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.14555290883567562) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.4538952380952381) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.209952783081799) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7015523809523809) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.1130187029836996) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.1943) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.087999239033555) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.4615809523809523) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.16058361739063698) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7451095238095238) - val-core/sciknoweval/acc/best@16/std:np.float64(0.06576006624725168) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.16135714285714284) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04477536267518606) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.46405714285714283) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11915383713937476) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.999702380952381) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0011526736149426837) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999904761904761) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0002127457895589398) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9994571428571428) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0015133869749850571) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9997476190476191) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0010668260935279934) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9989428571428571) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0019790054535129544) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9999190476190476) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0006155763481184213) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9981333333333333) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0023247461032216924) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9999904761904761) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0002127457895589398) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9970238095238095) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0023053472298853674) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001270969514735043 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001270969514735043 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:335.7734375 - response_length/max:1122.0 - response_length/min:122.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:335.7734375 - response_length_non_aborted/max:1122.0 - response_length_non_aborted/min:122.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.375 - prompt_length/max:701.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.7655776143074036e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4987823646515608) - timing_s/agent_loop/generate_sequences/max:np.float64(7.95271722227335) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.580353087221738) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.95271722227335) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:190 - timing_s/agent_loop/slowest/response_length:1122 - timing_s/gen:13.539362616837025 - timing_s/reward:0.05532005988061428 - timing_s/old_log_prob:2.3085584342479706 - timing_s/adv:0.48644337616860867 - timing_s/update_actor:15.44553491845727 - timing_s/step:31.924703411757946 - timing_s/testing:117.72085026092827 - timing_s/save_checkpoint:6.645242588594556 - timing_s/stop_profile:0.0001191999763250351 - timing_per_token_ms/adv:0.0031400055265921887 - timing_per_token_ms/gen:0.1575113731919894 - timing_per_token_ms/update_actor:0.09970135761149299 - perf/total_num_tokens:154918 - perf/time_per_step:31.924703411757946 - perf/throughput:606.5757213226894 (TaskRunner pid=2026458) Training Progress: 10%|█ | 10/100 [08:48<1:56:19, 77.55s/it] (TaskRunner pid=2026458) step:11 - global_seqlen/min:17717 - global_seqlen/max:22105 - global_seqlen/minmax_diff:4388 - global_seqlen/balanced_min:20139 - global_seqlen/balanced_max:20427 - global_seqlen/mean:20176.875 - actor/entropy:0.28003615140914917 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2986578345298767 - training/rollout_probs_diff_mean:0.006229304242879152 - training/rollout_probs_diff_std:0.014922214671969414 - training/rollout_actor_probs_pearson_corr:0.9980378746986389 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:0.9996895790100098 - rollout_corr/rollout_is_ratio_fraction_high:2.3571841666125692e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.071552681736648e-05 - rollout_corr/rollout_is_max:2.091331720352173 - rollout_corr/rollout_is_min:0.056948471814394 - rollout_corr/rollout_is_std:0.04615355655550957 - rollout_corr/rollout_is_eff_sample_size:0.9978729526678736 - rollout_corr/rollout_is_seq_mean:0.9996609091758728 - rollout_corr/rollout_is_seq_std:0.002769676735624671 - rollout_corr/rollout_is_seq_max:1.0097713470458984 - rollout_corr/rollout_is_seq_min:0.9914740920066833 - rollout_corr/rollout_is_seq_max_deviation:0.009771347045898438 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3445568550378084) - rollout_corr/training_log_ppl:np.float64(0.2928216397704091) - rollout_corr/kl:np.float64(0.0013540790551616055) - rollout_corr/k3_kl:np.float64(0.001210083299042708) - rollout_corr/rollout_ppl:np.float64(1.3426842936314642) - rollout_corr/rollout_log_ppl:np.float64(0.2914675589709077) - rollout_corr/log_ppl_diff:np.float64(0.0013540807995013893) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025300010456703603) - rollout_corr/log_ppl_diff_max:np.float64(0.011778771877288818) - rollout_corr/log_ppl_diff_min:np.float64(-0.006693422794342041) - rollout_corr/ppl_ratio:np.float64(1.001359449699521) - rollout_corr/chi2_token:np.float64(0.0021713071037083864) - rollout_corr/chi2_seq:np.float64(0.5860715808812529) - actor/pg_loss:np.float64(0.009074105182662606) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010309692956980143) - actor/ppo_kl:np.float64(-0.00014117327255291912) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.421875) - self_distillation/token_reweight_w_mean:np.float64(239610.312335568) - self_distillation/token_reweight_w_std:np.float64(3316290.997419097) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019973907619715) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14387543653720059) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.6877006888389587) - perf/mfu/actor:np.float64(0.037142691887104826) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.9859390258789) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006922754924744368 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006922754924744368 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:331.43359375 - response_length/max:1206.0 - response_length/min:142.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:331.43359375 - response_length_non_aborted/max:1206.0 - response_length_non_aborted/min:142.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.09375 - prompt_length/max:620.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012538395822048187 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7413379289209843) - timing_s/agent_loop/generate_sequences/max:np.float64(8.170563256368041) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.67358700458135) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.170563256368041) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:1206 - timing_s/gen:14.099589532241225 - timing_s/reward:0.05785256810486317 - timing_s/old_log_prob:2.514467492699623 - timing_s/adv:0.5351101234555244 - timing_s/update_actor:15.359056254848838 - timing_s/step:32.6669915933162 - timing_s/stop_profile:0.0001175031065940857 - timing_per_token_ms/adv:0.003315120177527023 - timing_per_token_ms/gen:0.16617664186407563 - timing_per_token_ms/update_actor:0.09515259582349124 - perf/total_num_tokens:161415 - perf/time_per_step:32.6669915933162 - perf/throughput:617.6532951423746 (TaskRunner pid=2026458) Training Progress: 11%|█ | 11/100 [09:20<1:34:40, 63.83s/it] (TaskRunner pid=2026458) step:12 - global_seqlen/min:17937 - global_seqlen/max:22358 - global_seqlen/minmax_diff:4421 - global_seqlen/balanced_min:19749 - global_seqlen/balanced_max:19763 - global_seqlen/mean:19757.25 - actor/entropy:0.2599850594997406 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29290565848350525 - training/rollout_probs_diff_mean:0.00601433590054512 - training/rollout_probs_diff_std:0.014930167235434055 - training/rollout_actor_probs_pearson_corr:0.9979494214057922 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.4609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:0.999984085559845 - rollout_corr/rollout_is_ratio_fraction_high:2.223754199803807e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.33563111780677e-05 - rollout_corr/rollout_is_max:2.084258556365967 - rollout_corr/rollout_is_min:0.48210135102272034 - rollout_corr/rollout_is_std:0.04539177566766739 - rollout_corr/rollout_is_eff_sample_size:0.9979434670549656 - rollout_corr/rollout_is_seq_mean:0.9999362230300903 - rollout_corr/rollout_is_seq_std:0.002637566300109029 - rollout_corr/rollout_is_seq_max:1.0081006288528442 - rollout_corr/rollout_is_seq_min:0.9917522072792053 - rollout_corr/rollout_is_seq_max_deviation:0.008247792720794678 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3254084084182978) - rollout_corr/training_log_ppl:np.float64(0.2779389991774224) - rollout_corr/kl:np.float64(0.0013704395118168122) - rollout_corr/k3_kl:np.float64(0.0011261843454803966) - rollout_corr/rollout_ppl:np.float64(1.3235342632979155) - rollout_corr/rollout_log_ppl:np.float64(0.27656855969689786) - rollout_corr/log_ppl_diff:np.float64(0.0013704394805245101) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024474224192090333) - rollout_corr/log_ppl_diff_max:np.float64(0.011240929365158081) - rollout_corr/log_ppl_diff_min:np.float64(-0.010027527809143066) - rollout_corr/ppl_ratio:np.float64(1.001375619089231) - rollout_corr/chi2_token:np.float64(0.001807245658710599) - rollout_corr/chi2_seq:np.float64(1.2864351340103894) - actor/pg_loss:np.float64(0.0068493077997118235) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010652919690983254) - actor/ppo_kl:np.float64(0.00020145679512317827) - actor/pg_clipfrac_lower:np.float64(1.669337689236272e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4609375) - self_distillation/token_reweight_w_mean:np.float64(145999.17218339257) - self_distillation/token_reweight_w_std:np.float64(2374372.460695313) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004515508189797) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1246801737579517) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.5914773046970367) - perf/mfu/actor:np.float64(0.03625096042724267) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.91592407226562) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.4609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004239031113684177 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004239031113684177 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:351.3203125 - response_length/max:1148.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:351.3203125 - response_length_non_aborted/max:1148.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.09375 - prompt_length/max:665.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.013755116611719131 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6791318338364363) - timing_s/agent_loop/generate_sequences/max:np.float64(8.12350033968687) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.846359725270304) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.12350033968687) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:1148 - timing_s/gen:13.899831658229232 - timing_s/reward:0.05922466889023781 - timing_s/old_log_prob:2.4094871655106544 - timing_s/adv:0.5697984360158443 - timing_s/update_actor:15.474863145500422 - timing_s/step:32.48048596084118 - timing_s/stop_profile:0.0001192968338727951 - timing_per_token_ms/adv:0.003604995862378648 - timing_per_token_ms/gen:0.15454904109752532 - timing_per_token_ms/update_actor:0.09790623154475206 - perf/total_num_tokens:158058 - perf/time_per_step:32.48048596084118 - perf/throughput:608.2806157463147 (TaskRunner pid=2026458) Training Progress: 12%|█▏ | 12/100 [09:53<1:19:39, 54.31s/it] (TaskRunner pid=2026458) step:13 - global_seqlen/min:17460 - global_seqlen/max:22387 - global_seqlen/minmax_diff:4927 - global_seqlen/balanced_min:19538 - global_seqlen/balanced_max:19656 - global_seqlen/mean:19562.0 - actor/entropy:0.28330478072166443 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2932348847389221 - training/rollout_probs_diff_mean:0.006090556271374226 - training/rollout_probs_diff_std:0.014429626986384392 - training/rollout_actor_probs_pearson_corr:0.998181939125061 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.68359375 - self_distillation/correct_sample_fraction:0.47265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.68359375 - rollout_corr/rollout_is_mean:0.9998684525489807 - rollout_corr/rollout_is_ratio_fraction_high:2.3373224394163117e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.3373224394163117e-05 - rollout_corr/rollout_is_max:2.1884407997131348 - rollout_corr/rollout_is_min:0.4281519055366516 - rollout_corr/rollout_is_std:0.04431807994842529 - rollout_corr/rollout_is_eff_sample_size:0.9980391641302073 - rollout_corr/rollout_is_seq_mean:0.9998126029968262 - rollout_corr/rollout_is_seq_std:0.0026776245795190334 - rollout_corr/rollout_is_seq_max:1.009365439414978 - rollout_corr/rollout_is_seq_min:0.9913782477378845 - rollout_corr/rollout_is_seq_max_deviation:0.009365439414978027 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3591533433645964) - rollout_corr/training_log_ppl:np.float64(0.30308022926328704) - rollout_corr/kl:np.float64(0.0012214116412607723) - rollout_corr/k3_kl:np.float64(0.0011643693818541578) - rollout_corr/rollout_ppl:np.float64(1.3574783634394407) - rollout_corr/rollout_log_ppl:np.float64(0.3018588193517644) - rollout_corr/log_ppl_diff:np.float64(0.0012214099115226418) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024031201901379973) - rollout_corr/log_ppl_diff_max:np.float64(0.011142253875732422) - rollout_corr/log_ppl_diff_min:np.float64(-0.00809323787689209) - rollout_corr/ppl_ratio:np.float64(1.0012265297118574) - rollout_corr/chi2_token:np.float64(0.0022032533306628466) - rollout_corr/chi2_seq:np.float64(1.1880777331534773) - actor/pg_loss:np.float64(0.008368437876924872) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011567205356186605) - actor/ppo_kl:np.float64(-1.6140308314760432e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.68359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.47265625) - self_distillation/token_reweight_w_mean:np.float64(93498.56042674859) - self_distillation/token_reweight_w_std:np.float64(1510537.690864286) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999996354104951) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12338341507711448) - self_distillation/empty_target_batch:np.float64(0.31640625) - actor/grad_norm:np.float64(0.620448037981987) - perf/mfu/actor:np.float64(0.036300476348807696) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.69003295898438) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.47265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.47265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003617006354033947 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003617006354033947 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:334.25 - response_length/max:1159.0 - response_length/min:121.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:334.25 - response_length_non_aborted/max:1159.0 - response_length_non_aborted/min:121.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.0625 - prompt_length/max:612.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014793872833251953 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3433514721691608) - timing_s/agent_loop/generate_sequences/max:np.float64(7.973012773320079) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6053173123218585) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.973012773320079) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:1159 - timing_s/gen:13.718764901161194 - timing_s/reward:0.05917247012257576 - timing_s/old_log_prob:2.3764582853764296 - timing_s/adv:0.49192045256495476 - timing_s/update_actor:15.186074122786522 - timing_s/step:31.921324100345373 - timing_s/stop_profile:0.00019265897572040558 - timing_per_token_ms/adv:0.003143342018741404 - timing_per_token_ms/gen:0.1603258800154403 - timing_per_token_ms/update_actor:0.09703809760496448 - perf/total_num_tokens:156496 - perf/time_per_step:31.921324100345373 - perf/throughput:612.8191906609648 (TaskRunner pid=2026458) Training Progress: 13%|█▎ | 13/100 [10:25<1:08:56, 47.54s/it] (TaskRunner pid=2026458) step:14 - global_seqlen/min:16941 - global_seqlen/max:22560 - global_seqlen/minmax_diff:5619 - global_seqlen/balanced_min:18841 - global_seqlen/balanced_max:18844 - global_seqlen/mean:18842.75 - actor/entropy:0.2922348976135254 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5466010570526123 - training/rollout_probs_diff_mean:0.006264000199735165 - training/rollout_probs_diff_std:0.014866359531879425 - training/rollout_actor_probs_pearson_corr:0.9981194138526917 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:1.0002233982086182 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.292350346688181e-05 - rollout_corr/rollout_is_max:1.9629987478256226 - rollout_corr/rollout_is_min:0.12186871469020844 - rollout_corr/rollout_is_std:0.04516667500138283 - rollout_corr/rollout_is_eff_sample_size:0.9979653119171001 - rollout_corr/rollout_is_seq_mean:1.0002156496047974 - rollout_corr/rollout_is_seq_std:0.00266577722504735 - rollout_corr/rollout_is_seq_max:1.0086764097213745 - rollout_corr/rollout_is_seq_min:0.9924325942993164 - rollout_corr/rollout_is_seq_max_deviation:0.008676409721374512 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.356894207186997) - rollout_corr/training_log_ppl:np.float64(0.30215946782846004) - rollout_corr/kl:np.float64(0.0010239662146345752) - rollout_corr/k3_kl:np.float64(0.0012049657070747344) - rollout_corr/rollout_ppl:np.float64(1.3555219103582203) - rollout_corr/rollout_log_ppl:np.float64(0.3011355001362972) - rollout_corr/log_ppl_diff:np.float64(0.0010239676921628416) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023661727900616825) - rollout_corr/log_ppl_diff_max:np.float64(0.017396926879882812) - rollout_corr/log_ppl_diff_min:np.float64(-0.007369279861450195) - rollout_corr/ppl_ratio:np.float64(1.0010292823426425) - rollout_corr/chi2_token:np.float64(0.0027081489097326994) - rollout_corr/chi2_seq:np.float64(1.2070772459264845) - actor/pg_loss:np.float64(0.00797818333376199) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010150298585358541) - actor/ppo_kl:np.float64(0.00015036190526984683) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.453125) - self_distillation/token_reweight_w_mean:np.float64(119982.02733938815) - self_distillation/token_reweight_w_std:np.float64(1933813.0637409594) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014771707355976) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1388165877724532) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.6455883979797363) - perf/mfu/actor:np.float64(0.034641753809947065) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.4944953918457) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0026480345986783504 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0026480345986783504 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:321.3984375 - response_length/max:682.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:321.3984375 - response_length_non_aborted/max:682.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.4375 - prompt_length/max:666.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012210384011268616 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7070676069706678) - timing_s/agent_loop/generate_sequences/max:np.float64(5.696014670655131) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4708188238437288) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.696014670655131) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:413 - timing_s/agent_loop/slowest/response_length:682 - timing_s/gen:11.448512280359864 - timing_s/reward:0.05713308043777943 - timing_s/old_log_prob:2.4212917368859053 - timing_s/adv:0.4922778233885765 - timing_s/update_actor:15.44989874586463 - timing_s/step:29.956744581460953 - timing_s/stop_profile:0.00011506862938404083 - timing_per_token_ms/adv:0.003265697837288722 - timing_per_token_ms/gen:0.1391442704047238 - timing_per_token_ms/update_actor:0.10249232958209809 - perf/total_num_tokens:150742 - perf/time_per_step:29.956744581460953 - perf/throughput:628.9985865707529 (TaskRunner pid=2026458) Training Progress: 14%|█▍ | 14/100 [10:55<1:00:33, 42.25s/it] (TaskRunner pid=2026458) step:15 - global_seqlen/min:17798 - global_seqlen/max:24465 - global_seqlen/minmax_diff:6667 - global_seqlen/balanced_min:20570 - global_seqlen/balanced_max:20574 - global_seqlen/mean:20572.75 - actor/entropy:0.28748324513435364 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4564783573150635 - training/rollout_probs_diff_mean:0.005869686137884855 - training/rollout_probs_diff_std:0.014366100542247295 - training/rollout_actor_probs_pearson_corr:0.9982949495315552 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.4921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:1.0001139640808105 - rollout_corr/rollout_is_ratio_fraction_high:5.4864267440279946e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.3891413952223957e-05 - rollout_corr/rollout_is_max:3.0497446060180664 - rollout_corr/rollout_is_min:0.38640856742858887 - rollout_corr/rollout_is_std:0.045057009905576706 - rollout_corr/rollout_is_eff_sample_size:0.9979744538008674 - rollout_corr/rollout_is_seq_mean:1.0000189542770386 - rollout_corr/rollout_is_seq_std:0.002783029805868864 - rollout_corr/rollout_is_seq_max:1.0093011856079102 - rollout_corr/rollout_is_seq_min:0.9919822812080383 - rollout_corr/rollout_is_seq_max_deviation:0.009301185607910156 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3641661452129483) - rollout_corr/training_log_ppl:np.float64(0.3061258476227522) - rollout_corr/kl:np.float64(0.0008984394618103408) - rollout_corr/k3_kl:np.float64(0.0010934443635051139) - rollout_corr/rollout_ppl:np.float64(1.3629517373628914) - rollout_corr/rollout_log_ppl:np.float64(0.3052274066139944) - rollout_corr/log_ppl_diff:np.float64(0.0008984410087577999) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023149027838371694) - rollout_corr/log_ppl_diff_max:np.float64(0.015007197856903076) - rollout_corr/log_ppl_diff_min:np.float64(-0.0058860182762146) - rollout_corr/ppl_ratio:np.float64(1.00090327905491) - rollout_corr/chi2_token:np.float64(0.0026275489944964647) - rollout_corr/chi2_seq:np.float64(1.554575996240601) - actor/pg_loss:np.float64(0.006172901834361255) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010921142766164849) - actor/ppo_kl:np.float64(-0.00014767027690254508) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4921875) - self_distillation/token_reweight_w_mean:np.float64(168825.68510642461) - self_distillation/token_reweight_w_std:np.float64(2411026.6848952193) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999142882879823) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13169058857602067) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.5341654643416405) - perf/mfu/actor:np.float64(0.03786336781112096) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.43387985229492) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036114403046667576 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0036114403046667576 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:355.9921875 - response_length/max:841.0 - response_length/min:154.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:355.9921875 - response_length_non_aborted/max:841.0 - response_length_non_aborted/min:154.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.90625 - prompt_length/max:494.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000149507075548172 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8174101784825325) - timing_s/agent_loop/generate_sequences/max:np.float64(6.822324391454458) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8521247571407002) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.822324391454458) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:187 - timing_s/agent_loop/slowest/response_length:841 - timing_s/gen:12.53348901681602 - timing_s/reward:0.06065330095589161 - timing_s/old_log_prob:2.2750260774046183 - timing_s/adv:0.5170386601239443 - timing_s/update_actor:15.529155055060983 - timing_s/step:31.002764070406556 - timing_s/stop_profile:0.00011553429067134857 - timing_per_token_ms/adv:0.003141526170079014 - timing_per_token_ms/gen:0.13752813458002525 - timing_per_token_ms/update_actor:0.09435512422416172 - perf/total_num_tokens:164582 - perf/time_per_step:31.002764070406556 - perf/throughput:663.5779298026383 (TaskRunner pid=2026458) Training Progress: 15%|█▌ | 15/100 [11:26<55:04, 38.87s/it] (TaskRunner pid=2026458) step:16 - global_seqlen/min:17436 - global_seqlen/max:21462 - global_seqlen/minmax_diff:4026 - global_seqlen/balanced_min:19472 - global_seqlen/balanced_max:19474 - global_seqlen/mean:19473.375 - actor/entropy:0.29451751708984375 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34674251079559326 - training/rollout_probs_diff_mean:0.006398364901542664 - training/rollout_probs_diff_std:0.014944251626729965 - training/rollout_actor_probs_pearson_corr:0.9980317950248718 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.64453125 - self_distillation/correct_sample_fraction:0.42578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.64453125 - rollout_corr/rollout_is_mean:1.000139832496643 - rollout_corr/rollout_is_ratio_fraction_high:3.57470526068937e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001429882104275748 - rollout_corr/rollout_is_max:2.075308322906494 - rollout_corr/rollout_is_min:0.33335795998573303 - rollout_corr/rollout_is_std:0.04721960052847862 - rollout_corr/rollout_is_eff_sample_size:0.9977759819851924 - rollout_corr/rollout_is_seq_mean:1.0001648664474487 - rollout_corr/rollout_is_seq_std:0.002871463308110833 - rollout_corr/rollout_is_seq_max:1.0102380514144897 - rollout_corr/rollout_is_seq_min:0.9917946457862854 - rollout_corr/rollout_is_seq_max_deviation:0.010238051414489746 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.36143590323627) - rollout_corr/training_log_ppl:np.float64(0.3041792264266405) - rollout_corr/kl:np.float64(0.0010337699030245062) - rollout_corr/k3_kl:np.float64(0.0012385786424147227) - rollout_corr/rollout_ppl:np.float64(1.3600022280588746) - rollout_corr/rollout_log_ppl:np.float64(0.30314545703004114) - rollout_corr/log_ppl_diff:np.float64(0.0010337693965993822) - rollout_corr/log_ppl_abs_diff:np.float64(0.002399776072707027) - rollout_corr/log_ppl_diff_max:np.float64(0.011399328708648682) - rollout_corr/log_ppl_diff_min:np.float64(-0.006810277700424194) - rollout_corr/ppl_ratio:np.float64(1.0010383201297373) - rollout_corr/chi2_token:np.float64(0.0029240259900689125) - rollout_corr/chi2_seq:np.float64(1.0353804978076369) - actor/pg_loss:np.float64(0.007192879682406783) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010274524142914743) - actor/ppo_kl:np.float64(4.0873823671461196e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.64453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.42578125) - self_distillation/token_reweight_w_mean:np.float64(90371.76209130394) - self_distillation/token_reweight_w_std:np.float64(1389709.4737052019) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000855000689626) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11968303663888946) - self_distillation/empty_target_batch:np.float64(0.35546875) - actor/grad_norm:np.float64(0.6043514311313629) - perf/mfu/actor:np.float64(0.03566211221390109) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.67253494262695) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006435959134250879 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006435959134250879 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:327.82421875 - response_length/max:698.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:327.82421875 - response_length_non_aborted/max:698.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.71875 - prompt_length/max:859.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010755844414234161 - timing_s/agent_loop/generate_sequences/min:np.float64(1.786530127748847) - timing_s/agent_loop/generate_sequences/max:np.float64(5.909618901088834) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.657506644296518) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.909618901088834) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:356 - timing_s/agent_loop/slowest/response_length:670 - timing_s/gen:11.721529113128781 - timing_s/reward:0.05861745588481426 - timing_s/old_log_prob:2.29558939486742 - timing_s/adv:0.5112862680107355 - timing_s/update_actor:15.580748109146953 - timing_s/step:30.256891636177897 - timing_s/stop_profile:0.00016321614384651184 - timing_per_token_ms/adv:0.003281957210875975 - timing_per_token_ms/gen:0.13967004412531464 - timing_per_token_ms/update_actor:0.10001314685530213 - perf/total_num_tokens:155787 - perf/time_per_step:30.256891636177897 - perf/throughput:643.6013069074107 (TaskRunner pid=2026458) Training Progress: 16%|█▌ | 16/100 [11:56<50:48, 36.29s/it] (TaskRunner pid=2026458) step:17 - global_seqlen/min:19903 - global_seqlen/max:25273 - global_seqlen/minmax_diff:5370 - global_seqlen/balanced_min:22184 - global_seqlen/balanced_max:22190 - global_seqlen/mean:22187.375 - actor/entropy:0.28030383586883545 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4096241295337677 - training/rollout_probs_diff_mean:0.00557515537366271 - training/rollout_probs_diff_std:0.014061576686799526 - training/rollout_actor_probs_pearson_corr:0.9982938766479492 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.640625 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.640625 - rollout_corr/rollout_is_mean:1.000001311302185 - rollout_corr/rollout_is_ratio_fraction_high:2.1693151211366057e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.4232874390436336e-05 - rollout_corr/rollout_is_max:3.007925271987915 - rollout_corr/rollout_is_min:0.3316154181957245 - rollout_corr/rollout_is_std:0.0431625172495842 - rollout_corr/rollout_is_eff_sample_size:0.9981403427818611 - rollout_corr/rollout_is_seq_mean:1.0001276731491089 - rollout_corr/rollout_is_seq_std:0.002614374505355954 - rollout_corr/rollout_is_seq_max:1.0077310800552368 - rollout_corr/rollout_is_seq_min:0.9923279881477356 - rollout_corr/rollout_is_seq_max_deviation:0.007731080055236816 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.381326341535896) - rollout_corr/training_log_ppl:np.float64(0.3175459819030948) - rollout_corr/kl:np.float64(0.0007249800520758853) - rollout_corr/k3_kl:np.float64(0.0011385135364321286) - rollout_corr/rollout_ppl:np.float64(1.3803315898403525) - rollout_corr/rollout_log_ppl:np.float64(0.316821001295466) - rollout_corr/log_ppl_diff:np.float64(0.0007249806076288223) - rollout_corr/log_ppl_abs_diff:np.float64(0.00215923844370991) - rollout_corr/log_ppl_diff_max:np.float64(0.00994020700454712) - rollout_corr/log_ppl_diff_min:np.float64(-0.007171809673309326) - rollout_corr/ppl_ratio:np.float64(1.0007290567737073) - rollout_corr/chi2_token:np.float64(0.003144999034702778) - rollout_corr/chi2_seq:np.float64(0.9032414378598332) - actor/pg_loss:np.float64(0.0055801779963076115) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008305092796945246) - actor/ppo_kl:np.float64(-0.00022399483597723702) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.640625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(213885.15984977316) - self_distillation/token_reweight_w_std:np.float64(3187382.02780142) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009852752555162) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11256717666401528) - self_distillation/empty_target_batch:np.float64(0.359375) - actor/grad_norm:np.float64(0.5271423608064651) - perf/mfu/actor:np.float64(0.04032452755142399) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.51458740234375) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0036620749160647392 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0036620749160647392 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:360.13671875 - response_length/max:1140.0 - response_length/min:135.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:360.13671875 - response_length_non_aborted/max:1140.0 - response_length_non_aborted/min:135.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:333.21875 - prompt_length/max:617.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014656782150268555 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6168197989463806) - timing_s/agent_loop/generate_sequences/max:np.float64(8.074994288384914) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7389801555254962) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.074994288384914) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1140 - timing_s/gen:13.71630746126175 - timing_s/reward:0.06177904084324837 - timing_s/old_log_prob:2.311936927959323 - timing_s/adv:0.5291431322693825 - timing_s/update_actor:15.853100046515465 - timing_s/step:32.560671512037516 - timing_s/stop_profile:0.00012019649147987366 - timing_per_token_ms/adv:0.0029811048640802622 - timing_per_token_ms/gen:0.14877496026098758 - timing_per_token_ms/update_actor:0.08931374287469487 - perf/total_num_tokens:177499 - perf/time_per_step:32.560671512037516 - perf/throughput:681.4163826995226 (TaskRunner pid=2026458) Training Progress: 17%|█▋ | 17/100 [12:29<48:40, 35.18s/it] (TaskRunner pid=2026458) step:18 - global_seqlen/min:17147 - global_seqlen/max:20375 - global_seqlen/minmax_diff:3228 - global_seqlen/balanced_min:18455 - global_seqlen/balanced_max:18458 - global_seqlen/mean:18457.0 - actor/entropy:0.3130648136138916 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2772509455680847 - training/rollout_probs_diff_mean:0.0063330200500786304 - training/rollout_probs_diff_std:0.014608096331357956 - training/rollout_actor_probs_pearson_corr:0.9982773065567017 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73046875 - self_distillation/correct_sample_fraction:0.421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73046875 - rollout_corr/rollout_is_mean:0.9997703433036804 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.43533785478212e-05 - rollout_corr/rollout_is_max:1.9900730848312378 - rollout_corr/rollout_is_min:0.4134674072265625 - rollout_corr/rollout_is_std:0.04487474635243416 - rollout_corr/rollout_is_eff_sample_size:0.9979895323387393 - rollout_corr/rollout_is_seq_mean:0.9998025298118591 - rollout_corr/rollout_is_seq_std:0.002652442781254649 - rollout_corr/rollout_is_seq_max:1.0070677995681763 - rollout_corr/rollout_is_seq_min:0.9913365244865417 - rollout_corr/rollout_is_seq_max_deviation:0.008663475513458252 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3969450276345015) - rollout_corr/training_log_ppl:np.float64(0.3299181046313606) - rollout_corr/kl:np.float64(0.001080212333377517) - rollout_corr/k3_kl:np.float64(0.0011240256234259505) - rollout_corr/rollout_ppl:np.float64(1.3953958731144667) - rollout_corr/rollout_log_ppl:np.float64(0.32883789180777967) - rollout_corr/log_ppl_diff:np.float64(0.0010802128235809505) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023538178065791726) - rollout_corr/log_ppl_diff_max:np.float64(0.011510014533996582) - rollout_corr/log_ppl_diff_min:np.float64(-0.009586125612258911) - rollout_corr/ppl_ratio:np.float64(1.0010849123355001) - rollout_corr/chi2_token:np.float64(0.0023459065705537796) - rollout_corr/chi2_seq:np.float64(0.5731152268126607) - actor/pg_loss:np.float64(0.007650406565517187) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011541137814674585) - actor/ppo_kl:np.float64(-0.00018551408525979696) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.421875) - self_distillation/token_reweight_w_mean:np.float64(258180.61296664365) - self_distillation/token_reweight_w_std:np.float64(3445719.439609512) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0022555205505341) - self_distillation/token_reweight_w_clip_frac:np.float64(0.15918678505113348) - self_distillation/empty_target_batch:np.float64(0.26953125) - actor/grad_norm:np.float64(0.6539853662252426) - perf/mfu/actor:np.float64(0.033737344150768206) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.57009506225586) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005579437594860792 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005579437594860792 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:303.5 - response_length/max:713.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:303.5 - response_length_non_aborted/max:713.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.28125 - prompt_length/max:540.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013168901205062866 - timing_s/agent_loop/generate_sequences/min:np.float64(1.151022594422102) - timing_s/agent_loop/generate_sequences/max:np.float64(5.789204815402627) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2287484776461497) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.789204815402627) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:260 - timing_s/agent_loop/slowest/response_length:713 - timing_s/gen:11.655897682532668 - timing_s/reward:0.05685456842184067 - timing_s/old_log_prob:2.3437488954514265 - timing_s/adv:0.4995372127741575 - timing_s/update_actor:15.640153270214796 - timing_s/step:30.285941103473306 - timing_s/stop_profile:0.00014618411660194397 - timing_per_token_ms/adv:0.0033831148939031095 - timing_per_token_ms/gen:0.15001927618580968 - timing_per_token_ms/update_actor:0.10592291048257298 - perf/total_num_tokens:147656 - perf/time_per_step:30.285941103473306 - perf/throughput:609.4246811397015 (TaskRunner pid=2026458) Training Progress: 18%|█▊ | 18/100 [12:59<46:05, 33.73s/it] (TaskRunner pid=2026458) step:19 - global_seqlen/min:16712 - global_seqlen/max:22903 - global_seqlen/minmax_diff:6191 - global_seqlen/balanced_min:19739 - global_seqlen/balanced_max:19742 - global_seqlen/mean:19740.875 - actor/entropy:0.30067509412765503 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2716989815235138 - training/rollout_probs_diff_mean:0.006011276505887508 - training/rollout_probs_diff_std:0.014068786054849625 - training/rollout_actor_probs_pearson_corr:0.9983283281326294 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:1.0000412464141846 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.48191299760947e-05 - rollout_corr/rollout_is_max:1.967125415802002 - rollout_corr/rollout_is_min:0.3074234426021576 - rollout_corr/rollout_is_std:0.04442286118865013 - rollout_corr/rollout_is_eff_sample_size:0.9980306147615733 - rollout_corr/rollout_is_seq_mean:1.0000803470611572 - rollout_corr/rollout_is_seq_std:0.0029041871894150972 - rollout_corr/rollout_is_seq_max:1.0110079050064087 - rollout_corr/rollout_is_seq_min:0.9903981685638428 - rollout_corr/rollout_is_seq_max_deviation:0.011007905006408691 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.375831812620163) - rollout_corr/training_log_ppl:np.float64(0.31493956671329215) - rollout_corr/kl:np.float64(0.0009361723424987645) - rollout_corr/k3_kl:np.float64(0.00108607468507671) - rollout_corr/rollout_ppl:np.float64(1.3745251446962357) - rollout_corr/rollout_log_ppl:np.float64(0.31400339302490465) - rollout_corr/log_ppl_diff:np.float64(0.0009361736883874983) - rollout_corr/log_ppl_abs_diff:np.float64(0.002310327283339575) - rollout_corr/log_ppl_diff_max:np.float64(0.008226662874221802) - rollout_corr/log_ppl_diff_min:np.float64(-0.010857582092285156) - rollout_corr/ppl_ratio:np.float64(1.000940806698054) - rollout_corr/chi2_token:np.float64(0.002460781019181013) - rollout_corr/chi2_seq:np.float64(1.263379932148382) - actor/pg_loss:np.float64(0.005378377623856068) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007215913469735824) - actor/ppo_kl:np.float64(-2.377988029422795e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_w_mean:np.float64(92620.61167126987) - self_distillation/token_reweight_w_std:np.float64(1450653.645121337) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001238340511918) - self_distillation/token_reweight_w_clip_frac:np.float64(0.096618483774364) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.5270146429538727) - perf/mfu/actor:np.float64(0.03604524222111994) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.61618423461914) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0024989761877804995 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0024989761877804995 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:314.77734375 - response_length/max:828.0 - response_length/min:154.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:314.77734375 - response_length_non_aborted/max:828.0 - response_length_non_aborted/min:154.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.125 - prompt_length/max:836.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014412589371204376 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8398964125663042) - timing_s/agent_loop/generate_sequences/max:np.float64(6.303931085392833) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4352486536299693) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.303931085392833) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:213 - timing_s/agent_loop/slowest/response_length:828 - timing_s/gen:12.11710455827415 - timing_s/reward:0.05806134268641472 - timing_s/old_log_prob:2.3140441831201315 - timing_s/adv:0.5250083282589912 - timing_s/update_actor:15.580280201509595 - timing_s/step:30.68251054175198 - timing_s/stop_profile:0.00012233667075634003 - timing_per_token_ms/adv:0.0033243734653288625 - timing_per_token_ms/gen:0.150368000177136 - timing_per_token_ms/update_actor:0.09865494944822352 - perf/total_num_tokens:157927 - perf/time_per_step:30.68251054175198 - perf/throughput:643.3917776427427 (TaskRunner pid=2026458) Training Progress: 19%|█▉ | 19/100 [13:30<44:19, 32.83s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which molecule is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". This product contains a structure with a substituted benzene ring (with a Cl group), a trifluoromethyl group (C(F)(F)F), and a nitrogen-containing amine group. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Looking at the options: (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option A contains a structure with a substituted benzene ring, a nitrogen atom, and a carbonyl group. It also includes the Cl and trifluoromethyl groups, which match the product's structure. However, it has an extra nitrogen and a different arrangement that may not be the correct reactant. (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option B has a similar structure but with a different arrangement of substituents and a different nitrogen configuration, which does not match the product's structure exactly. (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option C includes a substituted benzene ring, a carbonyl group, and a different arrangement of substituents. It includes a hydroxyl group and a different type of substitution, which does not match the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option D has a substituted benzene ring, a carbonyl group, and a different arrangement of substituents. It includes a different type of substitution and does not match the product's structure exactly. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A is the only one that matches the structure of the product, including the correct substituents and arrangement of atoms. Therefore, it is the correct reactant. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10/actor (TaskRunner pid=2026458) step:20 - global_seqlen/min:17486 - global_seqlen/max:22655 - global_seqlen/minmax_diff:5169 - global_seqlen/balanced_min:20031 - global_seqlen/balanced_max:20228 - global_seqlen/mean:20058.625 - actor/entropy:0.297139972448349 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3528301417827606 - training/rollout_probs_diff_mean:0.006166450679302216 - training/rollout_probs_diff_std:0.014436931349337101 - training/rollout_actor_probs_pearson_corr:0.9982419013977051 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.56640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.000038743019104 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.690288890036754e-05 - rollout_corr/rollout_is_max:1.9574382305145264 - rollout_corr/rollout_is_min:0.3535598814487457 - rollout_corr/rollout_is_std:0.04514159634709358 - rollout_corr/rollout_is_eff_sample_size:0.9979664991652689 - rollout_corr/rollout_is_seq_mean:1.0000238418579102 - rollout_corr/rollout_is_seq_std:0.0029090906027704477 - rollout_corr/rollout_is_seq_max:1.0110317468643188 - rollout_corr/rollout_is_seq_min:0.9888052940368652 - rollout_corr/rollout_is_seq_max_deviation:0.011194705963134766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3683372386731207) - rollout_corr/training_log_ppl:np.float64(0.31030508255935274) - rollout_corr/kl:np.float64(0.0011023225397535086) - rollout_corr/k3_kl:np.float64(0.001434143366623175) - rollout_corr/rollout_ppl:np.float64(1.3668043981306255) - rollout_corr/rollout_log_ppl:np.float64(0.30920275836251676) - rollout_corr/log_ppl_diff:np.float64(0.0011023241968359798) - rollout_corr/log_ppl_abs_diff:np.float64(0.002514752763090655) - rollout_corr/log_ppl_diff_max:np.float64(0.0131492018699646) - rollout_corr/log_ppl_diff_min:np.float64(-0.008862733840942383) - rollout_corr/ppl_ratio:np.float64(1.0011082254350185) - rollout_corr/chi2_token:np.float64(0.0035093913320451975) - rollout_corr/chi2_seq:np.float64(1.2149556565564126) - actor/pg_loss:np.float64(0.009395288652740419) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015388868646368792) - actor/ppo_kl:np.float64(4.407900779046514e-05) - actor/pg_clipfrac_lower:np.float64(3.007869418070186e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.56640625) - self_distillation/token_reweight_w_mean:np.float64(178377.74242206803) - self_distillation/token_reweight_w_std:np.float64(2601166.713610011) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996962018776685) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1573005250247661) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.68055609613657) - perf/mfu/actor:np.float64(0.03693197062950951) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.50461959838867) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.5101190476190476) - val-aux/sciknoweval/reward/std@16:np.float64(0.24920759452591995) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6139380952380953) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.2043060724638079) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.40814285714285714) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.21773506231934722) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5119238095238096) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.24829066308997053) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6943) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.144133820391533) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.32112857142857143) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1656815962127734) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.5277333333333334) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.20232169992437612) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7490619047619047) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09145129929830753) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.2561380952380952) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.11226817567713847) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.5346095238095239) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1512467176955187) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7826571428571428) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.051125776744021086) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.21173809523809525) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06785839628022725) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.5356190476190476) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11104188480484187) - val-aux/sciknoweval/score/mean@16:np.float64(0.5101190476190476) - val-aux/sciknoweval/score/std@16:np.float64(0.24920759452591995) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6139380952380953) - val-aux/sciknoweval/score/best@2/std:np.float64(0.2043060724638079) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.40814285714285714) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.21773506231934722) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.5119238095238096) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.24829066308997053) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6943) - val-aux/sciknoweval/score/best@4/std:np.float64(0.144133820391533) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.32112857142857143) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1656815962127734) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.5277333333333334) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.20232169992437612) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7490619047619047) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09145129929830753) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.2561380952380952) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.11226817567713847) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.5346095238095239) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.1512467176955187) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7826571428571428) - val-aux/sciknoweval/score/best@16/std:np.float64(0.051125776744021086) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.21173809523809525) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06785839628022725) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.5356190476190476) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11104188480484187) - val-core/sciknoweval/acc/mean@16:np.float64(0.5101190476190476) - val-aux/sciknoweval/acc/std@16:np.float64(0.24920759452591995) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6139380952380953) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.2043060724638079) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.40814285714285714) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.21773506231934722) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.5119238095238096) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.24829066308997053) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6943) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.144133820391533) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.32112857142857143) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1656815962127734) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.5277333333333334) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.20232169992437612) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7490619047619047) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09145129929830753) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.2561380952380952) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.11226817567713847) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.5346095238095239) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.1512467176955187) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7826571428571428) - val-core/sciknoweval/acc/best@16/std:np.float64(0.051125776744021086) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.21173809523809525) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06785839628022725) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.5356190476190476) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11104188480484187) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.999702380952381) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0011526736149426837) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999809523809524) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.000300566358820181) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9992904761904762) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0016956593912030376) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9996047619047619) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0013137248630794105) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9988428571428571) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.002042357581276089) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9999285714285715) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0005788212276734866) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9980380952380952) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0023437861108329258) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9968761904761905) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.002262100242179584) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011333633214235306 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011333633214235306 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:343.23828125 - response_length/max:1220.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:343.23828125 - response_length_non_aborted/max:1220.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.59375 - prompt_length/max:392.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000131312757730484 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3023900520056486) - timing_s/agent_loop/generate_sequences/max:np.float64(8.421228867024183) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7250610580740613) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.421228867024183) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:1220 - timing_s/gen:14.119003845378757 - timing_s/reward:0.058908019214868546 - timing_s/old_log_prob:2.3049094192683697 - timing_s/adv:0.5378871113061905 - timing_s/update_actor:15.319367559626698 - timing_s/step:32.428459821268916 - timing_s/testing:112.78920979425311 - timing_s/save_checkpoint:6.872485619038343 - timing_s/stop_profile:0.00016718730330467224 - timing_per_token_ms/adv:0.003351968986571802 - timing_per_token_ms/gen:0.16068242321386106 - timing_per_token_ms/update_actor:0.09546621191399397 - perf/total_num_tokens:160469 - perf/time_per_step:32.428459821268916 - perf/throughput:618.5500363123663 (TaskRunner pid=2026458) Training Progress: 20%|██ | 20/100 [16:02<1:31:32, 68.66s/it] (TaskRunner pid=2026458) step:21 - global_seqlen/min:17387 - global_seqlen/max:23009 - global_seqlen/minmax_diff:5622 - global_seqlen/balanced_min:19296 - global_seqlen/balanced_max:19303 - global_seqlen/mean:19300.625 - actor/entropy:0.2956748902797699 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.42548468708992004 - training/rollout_probs_diff_mean:0.005995167884975672 - training/rollout_probs_diff_std:0.014325084164738655 - training/rollout_actor_probs_pearson_corr:0.9982481002807617 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76171875 - self_distillation/correct_sample_fraction:0.375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76171875 - rollout_corr/rollout_is_mean:0.999878466129303 - rollout_corr/rollout_is_ratio_fraction_high:1.2208670341351535e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.325202022911981e-05 - rollout_corr/rollout_is_max:2.6692886352539062 - rollout_corr/rollout_is_min:0.4042225778102875 - rollout_corr/rollout_is_std:0.044611647725105286 - rollout_corr/rollout_is_eff_sample_size:0.9980132789914753 - rollout_corr/rollout_is_seq_mean:0.9997992515563965 - rollout_corr/rollout_is_seq_std:0.0029755644500255585 - rollout_corr/rollout_is_seq_max:1.0099884271621704 - rollout_corr/rollout_is_seq_min:0.9911137819290161 - rollout_corr/rollout_is_seq_max_deviation:0.00998842716217041 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3673860565759242) - rollout_corr/training_log_ppl:np.float64(0.3087902419210877) - rollout_corr/kl:np.float64(0.002253606636003269) - rollout_corr/k3_kl:np.float64(0.0017760526020538236) - rollout_corr/rollout_ppl:np.float64(1.36424843640998) - rollout_corr/rollout_log_ppl:np.float64(0.30653663334669545) - rollout_corr/log_ppl_diff:np.float64(0.002253608574392274) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032565151050221175) - rollout_corr/log_ppl_diff_max:np.float64(0.021895110607147217) - rollout_corr/log_ppl_diff_min:np.float64(-0.006019324064254761) - rollout_corr/ppl_ratio:np.float64(1.002263783942908) - rollout_corr/chi2_token:np.float64(0.002287261188030243) - rollout_corr/chi2_seq:np.float64(0.5934691540896893) - actor/pg_loss:np.float64(0.010032993159256876) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001869809015715873) - actor/ppo_kl:np.float64(0.000984730247550747) - actor/pg_clipfrac_lower:np.float64(2.9235126021376345e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76171875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.375) - self_distillation/token_reweight_w_mean:np.float64(353075.78875684226) - self_distillation/token_reweight_w_std:np.float64(4698832.619528098) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0033803705591708) - self_distillation/token_reweight_w_clip_frac:np.float64(0.17246504218201153) - self_distillation/empty_target_batch:np.float64(0.23828125) - actor/grad_norm:np.float64(0.7826535254716873) - perf/mfu/actor:np.float64(0.03582340340916603) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.28646850585938) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004340182524174452 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004340182524174452 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:319.95703125 - response_length/max:825.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:319.95703125 - response_length_non_aborted/max:825.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.1875 - prompt_length/max:669.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.8999285101890564e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3953026235103607) - timing_s/agent_loop/generate_sequences/max:np.float64(6.321475323289633) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4004087638677447) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.321475323289633) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:348 - timing_s/agent_loop/slowest/response_length:825 - timing_s/gen:12.068734215572476 - timing_s/reward:0.05136189982295036 - timing_s/old_log_prob:2.455154914408922 - timing_s/adv:0.532898997887969 - timing_s/update_actor:15.235047852620482 - timing_s/step:30.448000945150852 - timing_s/stop_profile:0.00011887960135936737 - timing_per_token_ms/adv:0.003451306614992837 - timing_per_token_ms/gen:0.14734320057102976 - timing_per_token_ms/update_actor:0.09866939446663309 - perf/total_num_tokens:154405 - perf/time_per_step:30.448000945150852 - perf/throughput:633.8880846321642 (TaskRunner pid=2026458) Training Progress: 21%|██ | 21/100 [16:33<1:15:19, 57.20s/it] (TaskRunner pid=2026458) step:22 - global_seqlen/min:15143 - global_seqlen/max:18708 - global_seqlen/minmax_diff:3565 - global_seqlen/balanced_min:16965 - global_seqlen/balanced_max:16966 - global_seqlen/mean:16965.375 - actor/entropy:0.31038597226142883 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2530360817909241 - training/rollout_probs_diff_mean:0.0064188591204583645 - training/rollout_probs_diff_std:0.014548459090292454 - training/rollout_actor_probs_pearson_corr:0.9982183575630188 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.66796875 - self_distillation/correct_sample_fraction:0.375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.66796875 - rollout_corr/rollout_is_mean:1.0001835823059082 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.267485928721726e-05 - rollout_corr/rollout_is_max:1.9917649030685425 - rollout_corr/rollout_is_min:0.3914262652397156 - rollout_corr/rollout_is_std:0.0448860339820385 - rollout_corr/rollout_is_eff_sample_size:0.9979898885301821 - rollout_corr/rollout_is_seq_mean:1.0002237558364868 - rollout_corr/rollout_is_seq_std:0.0030402897391468287 - rollout_corr/rollout_is_seq_max:1.0082043409347534 - rollout_corr/rollout_is_seq_min:0.9909875988960266 - rollout_corr/rollout_is_seq_max_deviation:0.009012401103973389 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3948297398164868) - rollout_corr/training_log_ppl:np.float64(0.32873041287530214) - rollout_corr/kl:np.float64(0.0012066373147963105) - rollout_corr/k3_kl:np.float64(0.0013965993947522293) - rollout_corr/rollout_ppl:np.float64(1.3931263061240315) - rollout_corr/rollout_log_ppl:np.float64(0.32752377312863246) - rollout_corr/log_ppl_diff:np.float64(0.0012066397466696799) - rollout_corr/log_ppl_abs_diff:np.float64(0.002761501877103001) - rollout_corr/log_ppl_diff_max:np.float64(0.011306077241897583) - rollout_corr/log_ppl_diff_min:np.float64(-0.009509235620498657) - rollout_corr/ppl_ratio:np.float64(1.0012132688425481) - rollout_corr/chi2_token:np.float64(0.003218145342543721) - rollout_corr/chi2_seq:np.float64(1.0662864169571549) - actor/pg_loss:np.float64(0.006237241672351956) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020408303180374787) - actor/ppo_kl:np.float64(0.0003451099556370796) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.66796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.375) - self_distillation/token_reweight_w_mean:np.float64(326844.6732725387) - self_distillation/token_reweight_w_std:np.float64(4298094.462517299) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0025643010158092) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1402466203435324) - self_distillation/empty_target_batch:np.float64(0.33203125) - actor/grad_norm:np.float64(0.6192770302295685) - perf/mfu/actor:np.float64(0.03161979996245398) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.7820701599121) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003102818038314581 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003102818038314581 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:274.60546875 - response_length/max:646.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:274.60546875 - response_length_non_aborted/max:646.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.5625 - prompt_length/max:592.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012409500777721405 - timing_s/agent_loop/generate_sequences/min:np.float64(1.460020076483488) - timing_s/agent_loop/generate_sequences/max:np.float64(5.160831518471241) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.023891647448181) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.160831518471241) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:221 - timing_s/agent_loop/slowest/response_length:646 - timing_s/gen:10.945641493424773 - timing_s/reward:0.05562561750411987 - timing_s/old_log_prob:2.356655828654766 - timing_s/adv:0.4728220421820879 - timing_s/update_actor:15.27205784060061 - timing_s/step:29.190949207171798 - timing_s/stop_profile:0.0001829788088798523 - timing_per_token_ms/adv:0.003483728197741635 - timing_per_token_ms/gen:0.15570124032240534 - timing_per_token_ms/update_actor:0.11252372730193563 - perf/total_num_tokens:135723 - perf/time_per_step:29.190949207171798 - perf/throughput:581.1861368259944 (TaskRunner pid=2026458) Training Progress: 22%|██▏ | 22/100 [17:02<1:03:27, 48.81s/it] (TaskRunner pid=2026458) step:23 - global_seqlen/min:15206 - global_seqlen/max:25332 - global_seqlen/minmax_diff:10126 - global_seqlen/balanced_min:18411 - global_seqlen/balanced_max:25520 - global_seqlen/mean:19300.375 - actor/entropy:0.264564573764801 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4770219326019287 - training/rollout_probs_diff_mean:0.0052871485240757465 - training/rollout_probs_diff_std:0.013520004227757454 - training/rollout_actor_probs_pearson_corr:0.998356282711029 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.59375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999374747276306 - rollout_corr/rollout_is_ratio_fraction_high:3.601138087105937e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.0018966905772686e-05 - rollout_corr/rollout_is_max:2.1046066284179688 - rollout_corr/rollout_is_min:0.36882883310317993 - rollout_corr/rollout_is_std:0.041905514895915985 - rollout_corr/rollout_is_eff_sample_size:0.9982467687554272 - rollout_corr/rollout_is_seq_mean:0.9998911023139954 - rollout_corr/rollout_is_seq_std:0.0030532306991517544 - rollout_corr/rollout_is_seq_max:1.0095295906066895 - rollout_corr/rollout_is_seq_min:0.9908995628356934 - rollout_corr/rollout_is_seq_max_deviation:0.009529590606689453 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3668543784879148) - rollout_corr/training_log_ppl:np.float64(0.30833037884258374) - rollout_corr/kl:np.float64(0.0014712483040568713) - rollout_corr/k3_kl:np.float64(0.0014252641368557306) - rollout_corr/rollout_ppl:np.float64(1.364856572356075) - rollout_corr/rollout_log_ppl:np.float64(0.30685912866283616) - rollout_corr/log_ppl_diff:np.float64(0.0014712501797475852) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029776765513815917) - rollout_corr/log_ppl_diff_max:np.float64(0.01450282335281372) - rollout_corr/log_ppl_diff_min:np.float64(-0.008856803178787231) - rollout_corr/ppl_ratio:np.float64(1.0014787362888455) - rollout_corr/chi2_token:np.float64(0.003022435586899519) - rollout_corr/chi2_seq:np.float64(1.3533644466660917) - actor/pg_loss:np.float64(0.007422412978485227) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020335467470431468) - actor/ppo_kl:np.float64(0.00035265562636155323) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59375) - self_distillation/token_reweight_w_mean:np.float64(345985.51773530385) - self_distillation/token_reweight_w_std:np.float64(4731856.136310147) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988346283789724) - self_distillation/token_reweight_w_clip_frac:np.float64(0.16046884056413546) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.7161731868982315) - perf/mfu/actor:np.float64(0.03389860046243484) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1311912536621) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008347137831151485 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008347137831151485 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:325.41796875 - response_length/max:8192.0 - response_length/min:126.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:325.41796875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:277.71875 - prompt_length/max:766.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010981783270835876 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5618780367076397) - timing_s/agent_loop/generate_sequences/max:np.float64(49.69344619102776) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4613353605309385) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.69344619102776) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:230 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.576194578781724 - timing_s/reward:0.05657547526061535 - timing_s/old_log_prob:2.56242498382926 - timing_s/adv:0.5329509787261486 - timing_s/update_actor:16.533158656209707 - timing_s/step:75.34719325602055 - timing_s/stop_profile:0.00011706724762916565 - timing_per_token_ms/adv:0.0034516879770869 - timing_per_token_ms/gen:0.6671251464916721 - timing_per_token_ms/update_actor:0.1070779625798055 - perf/total_num_tokens:154403 - perf/time_per_step:75.34719325602055 - perf/throughput:256.1525408706292 (TaskRunner pid=2026458) Training Progress: 23%|██▎ | 23/100 [18:17<1:12:52, 56.79s/it] (TaskRunner pid=2026458) step:24 - global_seqlen/min:16231 - global_seqlen/max:19060 - global_seqlen/minmax_diff:2829 - global_seqlen/balanced_min:17704 - global_seqlen/balanced_max:17708 - global_seqlen/mean:17706.25 - actor/entropy:0.29770269989967346 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5271099209785461 - training/rollout_probs_diff_mean:0.0061988383531570435 - training/rollout_probs_diff_std:0.014574181288480759 - training/rollout_actor_probs_pearson_corr:0.9981999397277832 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.5078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:1.000082015991211 - rollout_corr/rollout_is_ratio_fraction_high:2.8622949685086496e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011449179874034598 - rollout_corr/rollout_is_max:2.555720567703247 - rollout_corr/rollout_is_min:0.15318159759044647 - rollout_corr/rollout_is_std:0.04580353945493698 - rollout_corr/rollout_is_eff_sample_size:0.997906784084707 - rollout_corr/rollout_is_seq_mean:1.000064492225647 - rollout_corr/rollout_is_seq_std:0.0029369699768722057 - rollout_corr/rollout_is_seq_max:1.0110825300216675 - rollout_corr/rollout_is_seq_min:0.9918853640556335 - rollout_corr/rollout_is_seq_max_deviation:0.01108253002166748 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.36986547941342) - rollout_corr/training_log_ppl:np.float64(0.31092395351151936) - rollout_corr/kl:np.float64(0.0015822704315340275) - rollout_corr/k3_kl:np.float64(0.0015203862882344765) - rollout_corr/rollout_ppl:np.float64(1.3676151768304408) - rollout_corr/rollout_log_ppl:np.float64(0.3093416829360649) - rollout_corr/log_ppl_diff:np.float64(0.0015822705754544586) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029158030229154974) - rollout_corr/log_ppl_diff_max:np.float64(0.013341128826141357) - rollout_corr/log_ppl_diff_min:np.float64(-0.009624361991882324) - rollout_corr/ppl_ratio:np.float64(1.0015899618156254) - rollout_corr/chi2_token:np.float64(0.0028670839965343475) - rollout_corr/chi2_seq:np.float64(0.6717999817337841) - actor/pg_loss:np.float64(0.0065518481424078345) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001698189102626202) - actor/ppo_kl:np.float64(0.000539918119613958) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5078125) - self_distillation/token_reweight_w_mean:np.float64(259653.62712082197) - self_distillation/token_reweight_w_std:np.float64(3592355.994360844) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997991549316794) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1357375134830363) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.6166152656078339) - perf/mfu/actor:np.float64(0.03283040904184733) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1153450012207) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:0 - critic/score/mean:0.5078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003980379085987806 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003980379085987806 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:272.9453125 - response_length/max:645.0 - response_length/min:116.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:272.9453125 - response_length_non_aborted/max:645.0 - response_length_non_aborted/min:116.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.375 - prompt_length/max:848.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011000782251358032 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4089452624320984) - timing_s/agent_loop/generate_sequences/max:np.float64(5.263506431132555) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0320224383394816) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.263506431132555) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:645 - timing_s/gen:11.14209609106183 - timing_s/reward:0.05296533182263374 - timing_s/old_log_prob:2.2817067373543978 - timing_s/adv:0.506876740604639 - timing_s/update_actor:15.04843096062541 - timing_s/step:29.098722390830517 - timing_s/stop_profile:0.00011877715587615967 - timing_per_token_ms/adv:0.003578374448320784 - timing_per_token_ms/gen:0.15945982899307082 - timing_per_token_ms/update_actor:0.10623671698288323 - perf/total_num_tokens:141650 - perf/time_per_step:29.098722390830517 - perf/throughput:608.4889144679262 (TaskRunner pid=2026458) Training Progress: 24%|██▍ | 24/100 [18:46<1:01:25, 48.49s/it] (TaskRunner pid=2026458) step:25 - global_seqlen/min:15795 - global_seqlen/max:22433 - global_seqlen/minmax_diff:6638 - global_seqlen/balanced_min:18523 - global_seqlen/balanced_max:18527 - global_seqlen/mean:18525.875 - actor/entropy:0.2758452296257019 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25533485412597656 - training/rollout_probs_diff_mean:0.005769852548837662 - training/rollout_probs_diff_std:0.013884530402719975 - training/rollout_actor_probs_pearson_corr:0.9983004331588745 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.78515625 - self_distillation/correct_sample_fraction:0.48828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78515625 - rollout_corr/rollout_is_mean:1.0001115798950195 - rollout_corr/rollout_is_ratio_fraction_high:1.414367125107674e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.828734250215348e-05 - rollout_corr/rollout_is_max:2.6026298999786377 - rollout_corr/rollout_is_min:0.3732539415359497 - rollout_corr/rollout_is_std:0.04336366429924965 - rollout_corr/rollout_is_eff_sample_size:0.9981233594863631 - rollout_corr/rollout_is_seq_mean:1.000120759010315 - rollout_corr/rollout_is_seq_std:0.003118946449831128 - rollout_corr/rollout_is_seq_max:1.0095044374465942 - rollout_corr/rollout_is_seq_min:0.990989625453949 - rollout_corr/rollout_is_seq_max_deviation:0.009504437446594238 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3535170801915228) - rollout_corr/training_log_ppl:np.float64(0.2983562831650488) - rollout_corr/kl:np.float64(0.000989430469498842) - rollout_corr/k3_kl:np.float64(0.0011073322453967194) - rollout_corr/rollout_ppl:np.float64(1.3521247510798275) - rollout_corr/rollout_log_ppl:np.float64(0.29736685164971277) - rollout_corr/log_ppl_diff:np.float64(0.0009894315153360367) - rollout_corr/log_ppl_abs_diff:np.float64(0.002447564620524645) - rollout_corr/log_ppl_diff_max:np.float64(0.01045176386833191) - rollout_corr/log_ppl_diff_min:np.float64(-0.008554220199584961) - rollout_corr/ppl_ratio:np.float64(1.0009945430792868) - rollout_corr/chi2_token:np.float64(0.002472628140822053) - rollout_corr/chi2_seq:np.float64(0.6732058441266418) - actor/pg_loss:np.float64(0.005739172571338713) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010865399485737726) - actor/ppo_kl:np.float64(0.0001121951680964628) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78515625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78515625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48828125) - self_distillation/token_reweight_w_mean:np.float64(393425.2932063611) - self_distillation/token_reweight_w_std:np.float64(5274741.3204780845) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0018663464579731) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12906738638412207) - self_distillation/empty_target_batch:np.float64(0.21484375) - actor/grad_norm:np.float64(0.6542618125677109) - perf/mfu/actor:np.float64(0.034142301102186606) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1241111755371) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:0 - critic/score/mean:0.48828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005874927621334791 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005874927621334791 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:276.18359375 - response_length/max:807.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:276.18359375 - response_length_non_aborted/max:807.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.75 - prompt_length/max:646.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014953315258026123 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3853862024843693) - timing_s/agent_loop/generate_sequences/max:np.float64(5.845100851729512) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.959879736532457) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.845100851729512) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:213 - timing_s/agent_loop/slowest/response_length:807 - timing_s/gen:11.540503876283765 - timing_s/reward:0.05447870306670666 - timing_s/old_log_prob:2.3400896415114403 - timing_s/adv:0.5113725643604994 - timing_s/update_actor:15.447554072365165 - timing_s/step:29.938440999016166 - timing_s/stop_profile:0.00012017786502838135 - timing_per_token_ms/adv:0.0034503941403611123 - timing_per_token_ms/gen:0.16322509478075561 - timing_per_token_ms/update_actor:0.10422958478590866 - perf/total_num_tokens:148207 - perf/time_per_step:29.938440999016166 - perf/throughput:618.7989214471387 (TaskRunner pid=2026458) Training Progress: 25%|██▌ | 25/100 [19:16<53:40, 42.94s/it] (TaskRunner pid=2026458) step:26 - global_seqlen/min:16232 - global_seqlen/max:20091 - global_seqlen/minmax_diff:3859 - global_seqlen/balanced_min:18014 - global_seqlen/balanced_max:18018 - global_seqlen/mean:18016.5 - actor/entropy:0.28236204385757446 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3797455430030823 - training/rollout_probs_diff_mean:0.005817539524286985 - training/rollout_probs_diff_std:0.014095457270741463 - training/rollout_actor_probs_pearson_corr:0.9982751607894897 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71875 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71875 - rollout_corr/rollout_is_mean:1.0000920295715332 - rollout_corr/rollout_is_ratio_fraction_high:1.4651584933744743e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.790950960246846e-05 - rollout_corr/rollout_is_max:2.0633111000061035 - rollout_corr/rollout_is_min:0.2806716561317444 - rollout_corr/rollout_is_std:0.044501952826976776 - rollout_corr/rollout_is_eff_sample_size:0.9980238466152811 - rollout_corr/rollout_is_seq_mean:1.0001037120819092 - rollout_corr/rollout_is_seq_std:0.0033811419270932674 - rollout_corr/rollout_is_seq_max:1.010282039642334 - rollout_corr/rollout_is_seq_min:0.9834394454956055 - rollout_corr/rollout_is_seq_max_deviation:0.01656055450439453 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3481695111840963) - rollout_corr/training_log_ppl:np.float64(0.294715576863382) - rollout_corr/kl:np.float64(0.00122398861193318) - rollout_corr/k3_kl:np.float64(0.0011840371912512637) - rollout_corr/rollout_ppl:np.float64(1.3464941019192338) - rollout_corr/rollout_log_ppl:np.float64(0.2934915872756392) - rollout_corr/log_ppl_diff:np.float64(0.0012239895877428353) - rollout_corr/log_ppl_abs_diff:np.float64(0.002889484982006252) - rollout_corr/log_ppl_diff_max:np.float64(0.026334106922149658) - rollout_corr/log_ppl_diff_min:np.float64(-0.00859677791595459) - rollout_corr/ppl_ratio:np.float64(1.0012316501233727) - rollout_corr/chi2_token:np.float64(0.002297403523698449) - rollout_corr/chi2_seq:np.float64(1.2100839058402926) - actor/pg_loss:np.float64(0.004832616308704019) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006622324299314641) - actor/ppo_kl:np.float64(0.00027824501646955824) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(57253.49309117114) - self_distillation/token_reweight_w_std:np.float64(780711.6470303052) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9960764090064913) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10869685839861631) - self_distillation/empty_target_batch:np.float64(0.28125) - actor/grad_norm:np.float64(0.5899917781352997) - perf/mfu/actor:np.float64(0.03340075709542584) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.18029022216797) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:0 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0030969788786023855 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.0030969788786023855 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:266.609375 - response_length/max:611.0 - response_length/min:109.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:266.609375 - response_length_non_aborted/max:611.0 - response_length_non_aborted/min:109.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.40625 - prompt_length/max:709.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014692358672618866 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2525565940886736) - timing_s/agent_loop/generate_sequences/max:np.float64(4.980492563918233) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.909446031990228) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.980492563918233) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:406 - timing_s/agent_loop/slowest/response_length:611 - timing_s/gen:10.703484870493412 - timing_s/reward:0.056105511263012886 - timing_s/old_log_prob:2.315383156761527 - timing_s/adv:0.49405837804079056 - timing_s/update_actor:15.426462525501847 - timing_s/step:29.082023203372955 - timing_s/stop_profile:0.0001284964382648468 - timing_per_token_ms/adv:0.003427818791391159 - timing_per_token_ms/gen:0.156823021603666 - timing_per_token_ms/update_actor:0.10703010105668309 - perf/total_num_tokens:144132 - perf/time_per_step:29.082023203372955 - perf/throughput:619.5064172120746 (TaskRunner pid=2026458) Training Progress: 26%|██▌ | 26/100 [19:46<47:51, 38.80s/it] (TaskRunner pid=2026458) step:27 - global_seqlen/min:14932 - global_seqlen/max:19899 - global_seqlen/minmax_diff:4967 - global_seqlen/balanced_min:17014 - global_seqlen/balanced_max:17065 - global_seqlen/mean:17022.125 - actor/entropy:0.27002352476119995 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2583366632461548 - training/rollout_probs_diff_mean:0.005672288592904806 - training/rollout_probs_diff_std:0.013904700987040997 - training/rollout_actor_probs_pearson_corr:0.9982785582542419 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76171875 - self_distillation/correct_sample_fraction:0.5 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76171875 - rollout_corr/rollout_is_mean:1.0000253915786743 - rollout_corr/rollout_is_ratio_fraction_high:5.697275264537893e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.4243188161344733e-05 - rollout_corr/rollout_is_max:2.935737371444702 - rollout_corr/rollout_is_min:0.4851767420768738 - rollout_corr/rollout_is_std:0.04337328299880028 - rollout_corr/rollout_is_eff_sample_size:0.9981222906269569 - rollout_corr/rollout_is_seq_mean:0.9999536275863647 - rollout_corr/rollout_is_seq_std:0.002799020381644368 - rollout_corr/rollout_is_seq_max:1.0089412927627563 - rollout_corr/rollout_is_seq_min:0.991137683391571 - rollout_corr/rollout_is_seq_max_deviation:0.008941292762756348 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3306819731369615) - rollout_corr/training_log_ppl:np.float64(0.28223902196623385) - rollout_corr/kl:np.float64(0.0010320211754617503) - rollout_corr/k3_kl:np.float64(0.0012100190330102123) - rollout_corr/rollout_ppl:np.float64(1.3292772802524269) - rollout_corr/rollout_log_ppl:np.float64(0.28120700089493766) - rollout_corr/log_ppl_diff:np.float64(0.0010320210712961853) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025203333934769034) - rollout_corr/log_ppl_diff_max:np.float64(0.011266171932220459) - rollout_corr/log_ppl_diff_min:np.float64(-0.00728076696395874) - rollout_corr/ppl_ratio:np.float64(1.0010370928794146) - rollout_corr/chi2_token:np.float64(0.002755629364401102) - rollout_corr/chi2_seq:np.float64(0.8209603114519268) - actor/pg_loss:np.float64(0.0054146344773471355) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001441872691430035) - actor/ppo_kl:np.float64(-3.021463783170475e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76171875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5) - self_distillation/token_reweight_w_mean:np.float64(149585.24241724727) - self_distillation/token_reweight_w_std:np.float64(2245182.637069527) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0022315112873912) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10743028059368953) - self_distillation/empty_target_batch:np.float64(0.23828125) - actor/grad_norm:np.float64(0.5845905467867851) - perf/mfu/actor:np.float64(0.031846668272466384) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.01116561889648) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:0 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0008261048933491111 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0008261048933491111 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:274.25390625 - response_length/max:877.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:274.25390625 - response_length_non_aborted/max:877.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:257.6875 - prompt_length/max:380.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001650247722864151 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4051890634000301) - timing_s/agent_loop/generate_sequences/max:np.float64(6.235670423135161) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9710693403030746) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.235670423135161) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:275 - timing_s/agent_loop/slowest/response_length:877 - timing_s/gen:12.108512347564101 - timing_s/reward:0.05487024039030075 - timing_s/old_log_prob:2.2881970033049583 - timing_s/adv:0.46778740733861923 - timing_s/update_actor:15.23558795452118 - timing_s/step:30.242075925692916 - timing_s/stop_profile:0.0001263841986656189 - timing_per_token_ms/adv:0.003435142552256396 - timing_per_token_ms/gen:0.17246382013081088 - timing_per_token_ms/update_actor:0.11188077248376142 - perf/total_num_tokens:136177 - perf/time_per_step:30.242075925692916 - perf/throughput:562.8623194328543 (TaskRunner pid=2026458) Training Progress: 27%|██▋ | 27/100 [20:16<44:06, 36.25s/it] (TaskRunner pid=2026458) step:28 - global_seqlen/min:15064 - global_seqlen/max:18912 - global_seqlen/minmax_diff:3848 - global_seqlen/balanced_min:17172 - global_seqlen/balanced_max:17302 - global_seqlen/mean:17191.25 - actor/entropy:0.26276373863220215 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24491259455680847 - training/rollout_probs_diff_mean:0.005732021760195494 - training/rollout_probs_diff_std:0.014186778105795383 - training/rollout_actor_probs_pearson_corr:0.9981420636177063 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9999222159385681 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4750567970622797e-05 - rollout_corr/rollout_is_max:1.8682423830032349 - rollout_corr/rollout_is_min:0.4175392985343933 - rollout_corr/rollout_is_std:0.043118301779031754 - rollout_corr/rollout_is_eff_sample_size:0.9981441433187315 - rollout_corr/rollout_is_seq_mean:0.9998924732208252 - rollout_corr/rollout_is_seq_std:0.0026770406402647495 - rollout_corr/rollout_is_seq_max:1.0065722465515137 - rollout_corr/rollout_is_seq_min:0.9909303188323975 - rollout_corr/rollout_is_seq_max_deviation:0.009069681167602539 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3176847058348358) - rollout_corr/training_log_ppl:np.float64(0.2727040337776998) - rollout_corr/kl:np.float64(0.0012852028201848498) - rollout_corr/k3_kl:np.float64(0.001238845826378565) - rollout_corr/rollout_ppl:np.float64(1.315975757315755) - rollout_corr/rollout_log_ppl:np.float64(0.27141882907017134) - rollout_corr/log_ppl_diff:np.float64(0.0012852047075284645) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026650355284800753) - rollout_corr/log_ppl_diff_max:np.float64(0.011441320180892944) - rollout_corr/log_ppl_diff_min:np.float64(-0.011876940727233887) - rollout_corr/ppl_ratio:np.float64(1.0012910023797303) - rollout_corr/chi2_token:np.float64(0.002501809736713767) - rollout_corr/chi2_seq:np.float64(1.4115001251921058) - actor/pg_loss:np.float64(0.005721935303881764) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001080054133581143) - actor/ppo_kl:np.float64(0.00020560735394870022) - actor/pg_clipfrac_lower:np.float64(3.0048076951061375e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(263679.60727620986) - self_distillation/token_reweight_w_std:np.float64(3103030.912776855) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007451251149178) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10346215419122018) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.6124091595411301) - perf/mfu/actor:np.float64(0.03192732787611963) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.95586776733398) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00606801500543952 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00606801500543952 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:264.8203125 - response_length/max:1012.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:264.8203125 - response_length_non_aborted/max:1012.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.40625 - prompt_length/max:443.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001549404114484787 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3163175880908966) - timing_s/agent_loop/generate_sequences/max:np.float64(6.851215856149793) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.895652609608078) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.851215856149793) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:255 - timing_s/agent_loop/slowest/response_length:1012 - timing_s/gen:12.757756907492876 - timing_s/reward:0.05424255505204201 - timing_s/old_log_prob:2.304752092808485 - timing_s/adv:0.5271155349910259 - timing_s/update_actor:15.178545750677586 - timing_s/step:30.909547282382846 - timing_s/stop_profile:0.00012030825018882751 - timing_per_token_ms/adv:0.0038327312949249324 - timing_per_token_ms/gen:0.18818415947565972 - timing_per_token_ms/update_actor:0.11036534392988864 - perf/total_num_tokens:137530 - perf/time_per_step:30.909547282382846 - perf/throughput:556.179288002652 (TaskRunner pid=2026458) Training Progress: 28%|██▊ | 28/100 [20:47<41:35, 34.66s/it] (TaskRunner pid=2026458) step:29 - global_seqlen/min:14445 - global_seqlen/max:22059 - global_seqlen/minmax_diff:7614 - global_seqlen/balanced_min:19100 - global_seqlen/balanced_max:20098 - global_seqlen/mean:19231.25 - actor/entropy:0.2399301379919052 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5590871572494507 - training/rollout_probs_diff_mean:0.005236997734755278 - training/rollout_probs_diff_std:0.014018451794981956 - training/rollout_actor_probs_pearson_corr:0.9981542229652405 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.5859375 - self_distillation/correct_sample_fraction:0.34375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.5859375 - rollout_corr/rollout_is_mean:1.0000776052474976 - rollout_corr/rollout_is_ratio_fraction_high:2.4095805201795883e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.819161040359177e-05 - rollout_corr/rollout_is_max:3.816781997680664 - rollout_corr/rollout_is_min:0.17573890089988708 - rollout_corr/rollout_is_std:0.04291044920682907 - rollout_corr/rollout_is_eff_sample_size:0.9981623150356519 - rollout_corr/rollout_is_seq_mean:0.9999245405197144 - rollout_corr/rollout_is_seq_std:0.0027032301295548677 - rollout_corr/rollout_is_seq_max:1.008720874786377 - rollout_corr/rollout_is_seq_min:0.992788553237915 - rollout_corr/rollout_is_seq_max_deviation:0.008720874786376953 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3173197288997471) - rollout_corr/training_log_ppl:np.float64(0.27068548381794244) - rollout_corr/kl:np.float64(0.0009471329129642214) - rollout_corr/k3_kl:np.float64(0.001118057121630045) - rollout_corr/rollout_ppl:np.float64(1.3160036713816226) - rollout_corr/rollout_log_ppl:np.float64(0.2697383483755402) - rollout_corr/log_ppl_diff:np.float64(0.0009471354424022138) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024268297711387277) - rollout_corr/log_ppl_diff_max:np.float64(0.010676145553588867) - rollout_corr/log_ppl_diff_min:np.float64(-0.008322864770889282) - rollout_corr/ppl_ratio:np.float64(1.0009521523024887) - rollout_corr/chi2_token:np.float64(0.0025950868148356676) - rollout_corr/chi2_seq:np.float64(1.0168882303405553) - actor/pg_loss:np.float64(0.004738333867862821) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009898098714984371) - actor/ppo_kl:np.float64(-0.0001235502958607526) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.5859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.34375) - self_distillation/token_reweight_w_mean:np.float64(148981.53039407637) - self_distillation/token_reweight_w_std:np.float64(2396815.2922347095) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019751288928092) - self_distillation/token_reweight_w_clip_frac:np.float64(0.088682441302808) - self_distillation/empty_target_batch:np.float64(0.4140625) - actor/grad_norm:np.float64(0.5395381674170494) - perf/mfu/actor:np.float64(0.0350041185371077) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.9663429260254) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:0 - critic/score/mean:0.34375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.014469530433416367 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.014469530433416367 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:324.2265625 - response_length/max:2192.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:324.2265625 - response_length_non_aborted/max:2192.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.75 - prompt_length/max:492.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000128256157040596 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1863264609128237) - timing_s/agent_loop/generate_sequences/max:np.float64(12.72119532711804) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.387789061780495) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.72119532711804) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:291 - timing_s/agent_loop/slowest/response_length:2192 - timing_s/gen:18.591661918908358 - timing_s/reward:0.05764943175017834 - timing_s/old_log_prob:2.3708045054227114 - timing_s/adv:0.6288922913372517 - timing_s/update_actor:15.632321024313569 - timing_s/step:37.36768543533981 - timing_s/stop_profile:0.00012124888598918915 - timing_per_token_ms/adv:0.004087697701249605 - timing_per_token_ms/gen:0.2239905293716821 - timing_per_token_ms/update_actor:0.10160754646937646 - perf/total_num_tokens:153850 - perf/time_per_step:37.36768543533981 - perf/throughput:514.6492156512428 (TaskRunner pid=2026458) Training Progress: 29%|██▉ | 29/100 [21:24<41:59, 35.49s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to analyze the structure of the product and compare it with the options provided. The product contains a key fragment: "Cc2ccc(Cl)cc2C(F)(F)F", which is a benzene ring with a chlorine substituent and a trifluoromethyl group. This fragment is likely derived from a reactant that contains a similar structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A includes a structure with a benzene ring, a chlorine substituent, and a carboxylic acid group, but it does not contain the trifluoromethyl group. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B includes a structure with a benzene ring and a chlorine substituent, but it lacks the trifluoromethyl group and has a different functional group arrangement. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C includes a structure with a benzene ring, a chlorine substituent, and a trifluoromethyl group, and it matches the required fragment in the product. It also includes a carboxylic acid group, which is consistent with the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D includes a structure with a benzene ring and a trifluoromethyl group, but it lacks the chlorine substituent and has a different functional group arrangement. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, Option C is the only correct reactant that matches the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) C (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 0.0 (TaskRunner pid=2026458) [acc] 0.0 (TaskRunner pid=2026458) [pred] C (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20/actor (TaskRunner pid=2026458) step:30 - global_seqlen/min:15398 - global_seqlen/max:27317 - global_seqlen/minmax_diff:11919 - global_seqlen/balanced_min:18528 - global_seqlen/balanced_max:25696 - global_seqlen/mean:20322.875 - actor/entropy:0.21429221332073212 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29036805033683777 - training/rollout_probs_diff_mean:0.0045964838936924934 - training/rollout_probs_diff_std:0.012965062633156776 - training/rollout_actor_probs_pearson_corr:0.9982911944389343 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.5859375 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.5859375 - rollout_corr/rollout_is_mean:0.9999114871025085 - rollout_corr/rollout_is_ratio_fraction_high:1.198825157189276e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.99412560404744e-05 - rollout_corr/rollout_is_max:2.0840299129486084 - rollout_corr/rollout_is_min:0.03206612169742584 - rollout_corr/rollout_is_std:0.04015055671334267 - rollout_corr/rollout_is_eff_sample_size:0.9983904085296447 - rollout_corr/rollout_is_seq_mean:0.9999212622642517 - rollout_corr/rollout_is_seq_std:0.003086200449615717 - rollout_corr/rollout_is_seq_max:1.0128618478775024 - rollout_corr/rollout_is_seq_min:0.9903483986854553 - rollout_corr/rollout_is_seq_max_deviation:0.012861847877502441 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3321797889657319) - rollout_corr/training_log_ppl:np.float64(0.28194430993880815) - rollout_corr/kl:np.float64(0.0011635422050204625) - rollout_corr/k3_kl:np.float64(0.0012278429346110897) - rollout_corr/rollout_ppl:np.float64(1.3305623242631555) - rollout_corr/rollout_log_ppl:np.float64(0.2807807675008007) - rollout_corr/log_ppl_diff:np.float64(0.0011635424380074255) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028164471932541346) - rollout_corr/log_ppl_diff_max:np.float64(0.01170569658279419) - rollout_corr/log_ppl_diff_min:np.float64(-0.010286390781402588) - rollout_corr/ppl_ratio:np.float64(1.0011705257929862) - rollout_corr/chi2_token:np.float64(0.0025684768334031105) - rollout_corr/chi2_seq:np.float64(0.56981481006369) - actor/pg_loss:np.float64(0.004291482153348625) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010325129969714908) - actor/ppo_kl:np.float64(2.040402626590776e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.5859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(97715.03854049812) - self_distillation/token_reweight_w_std:np.float64(1362849.7468459418) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998477473855019) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08768687405972742) - self_distillation/empty_target_batch:np.float64(0.4140625) - actor/grad_norm:np.float64(0.5216539651155472) - perf/mfu/actor:np.float64(0.03586380732978027) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.9196662902832) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.5943452380952381) - val-aux/sciknoweval/reward/std@16:np.float64(0.19385587331070453) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6712571428571429) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.14416552478989664) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5181047619047618) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.18180031114620876) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5949523809523809) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1935592702447506) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7259380952380953) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09597054475819111) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4468809523809524) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.15668932421972914) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6092761904761904) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.14936173723020238) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7630047619047619) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.05768424104510231) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.3826380952380953) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.12589633211761853) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.615552380952381) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10296365255394938) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7833380952380953) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.029000030584979297) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.32762380952380954) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.0895080232788665) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6183523809523809) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07343617415290264) - val-aux/sciknoweval/score/mean@16:np.float64(0.5943452380952381) - val-aux/sciknoweval/score/std@16:np.float64(0.19385587331070453) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6712571428571429) - val-aux/sciknoweval/score/best@2/std:np.float64(0.14416552478989664) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5181047619047618) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.18180031114620876) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.5949523809523809) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1935592702447506) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7259380952380953) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09597054475819111) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4468809523809524) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.15668932421972914) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6092761904761904) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.14936173723020238) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7630047619047619) - val-aux/sciknoweval/score/best@8/std:np.float64(0.05768424104510231) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.3826380952380953) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.12589633211761853) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.615552380952381) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10296365255394938) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7833380952380953) - val-aux/sciknoweval/score/best@16/std:np.float64(0.029000030584979297) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.32762380952380954) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.0895080232788665) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6183523809523809) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07343617415290264) - val-core/sciknoweval/acc/mean@16:np.float64(0.5943452380952381) - val-aux/sciknoweval/acc/std@16:np.float64(0.19385587331070453) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6712571428571429) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.14416552478989664) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5181047619047618) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.18180031114620876) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.5949523809523809) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1935592702447506) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7259380952380953) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09597054475819111) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4468809523809524) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.15668932421972914) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6092761904761904) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.14936173723020238) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7630047619047619) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.05768424104510231) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.3826380952380953) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.12589633211761853) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.615552380952381) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10296365255394938) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7833380952380953) - val-core/sciknoweval/acc/best@16/std:np.float64(0.029000030584979297) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.32762380952380954) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.0895080232788665) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6183523809523809) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07343617415290264) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.08896331489086151 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.08896331489086151 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:325.83984375 - response_length/max:8192.0 - response_length/min:95.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:325.83984375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:309.25 - prompt_length/max:516.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012153573334217072 - timing_s/agent_loop/generate_sequences/min:np.float64(0.691241292282939) - timing_s/agent_loop/generate_sequences/max:np.float64(49.47414601966739) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9101952763448935) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.47414601966739) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:218 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.4048219807446 - timing_s/reward:0.05735262483358383 - timing_s/old_log_prob:2.4137737192213535 - timing_s/adv:0.4771828204393387 - timing_s/update_actor:16.52942644432187 - timing_s/step:74.96756593696773 - timing_s/testing:65.94198310934007 - timing_s/save_checkpoint:6.872431704774499 - timing_s/stop_profile:9.614601731300354e-05 - timing_per_token_ms/adv:0.002935010551160568 - timing_per_token_ms/gen:0.6642069409667878 - timing_per_token_ms/update_actor:0.10166761865829681 - perf/total_num_tokens:162583 - perf/time_per_step:74.96756593696773 - perf/throughput:271.0889001930161 (TaskRunner pid=2026458) Training Progress: 30%|███ | 30/100 [23:52<1:20:43, 69.19s/it] (TaskRunner pid=2026458) step:31 - global_seqlen/min:15697 - global_seqlen/max:20497 - global_seqlen/minmax_diff:4800 - global_seqlen/balanced_min:18142 - global_seqlen/balanced_max:18440 - global_seqlen/mean:18179.75 - actor/entropy:0.24378487467765808 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4056262969970703 - training/rollout_probs_diff_mean:0.005305483937263489 - training/rollout_probs_diff_std:0.013869675807654858 - training/rollout_actor_probs_pearson_corr:0.9981986880302429 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:1.0002444982528687 - rollout_corr/rollout_is_ratio_fraction_high:5.420200977823697e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.485351620242e-05 - rollout_corr/rollout_is_max:2.6529436111450195 - rollout_corr/rollout_is_min:0.2698281407356262 - rollout_corr/rollout_is_std:0.04314593970775604 - rollout_corr/rollout_is_eff_sample_size:0.9981427181140134 - rollout_corr/rollout_is_seq_mean:1.0003163814544678 - rollout_corr/rollout_is_seq_std:0.0029239451978355646 - rollout_corr/rollout_is_seq_max:1.0140212774276733 - rollout_corr/rollout_is_seq_min:0.9939373135566711 - rollout_corr/rollout_is_seq_max_deviation:0.01402127742767334 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3019663002341986) - rollout_corr/training_log_ppl:np.float64(0.2597778052149806) - rollout_corr/kl:np.float64(0.001051263411028458) - rollout_corr/k3_kl:np.float64(0.0012008980110067569) - rollout_corr/rollout_ppl:np.float64(1.3005597270093858) - rollout_corr/rollout_log_ppl:np.float64(0.2587265418842435) - rollout_corr/log_ppl_diff:np.float64(0.0010512633307371289) - rollout_corr/log_ppl_abs_diff:np.float64(0.002555768267484382) - rollout_corr/log_ppl_diff_max:np.float64(0.010198384523391724) - rollout_corr/log_ppl_diff_min:np.float64(-0.010389238595962524) - rollout_corr/ppl_ratio:np.float64(1.00105693587102) - rollout_corr/chi2_token:np.float64(0.002714824862778187) - rollout_corr/chi2_seq:np.float64(1.5057584547903389) - actor/pg_loss:np.float64(0.004452571272850037) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007820382315912866) - actor/ppo_kl:np.float64(0.00035900160938950876) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_w_mean:np.float64(99177.65471042739) - self_distillation/token_reweight_w_std:np.float64(1529254.068720376) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9981596393045038) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09003465421847068) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.5145964473485947) - perf/mfu/actor:np.float64(0.0335625674385555) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.96116638183594) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:0 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012466462794691324 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0012466462794691324 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:288.2734375 - response_length/max:769.0 - response_length/min:118.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:288.2734375 - response_length_non_aborted/max:769.0 - response_length_non_aborted/min:118.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.84375 - prompt_length/max:635.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.449354648590088e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2310489993542433) - timing_s/agent_loop/generate_sequences/max:np.float64(5.724285513162613) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1068192758539226) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.724285513162613) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:635 - timing_s/agent_loop/slowest/response_length:769 - timing_s/gen:11.310221493244171 - timing_s/reward:0.053344203159213066 - timing_s/old_log_prob:2.4446758944541216 - timing_s/adv:0.550066627562046 - timing_s/update_actor:15.255971036851406 - timing_s/step:29.709052259102464 - timing_s/stop_profile:0.0001292899250984192 - timing_per_token_ms/adv:0.003782138282718726 - timing_per_token_ms/gen:0.15325918714930176 - timing_per_token_ms/update_actor:0.10489673288171872 - perf/total_num_tokens:145438 - perf/time_per_step:29.709052259102464 - perf/throughput:611.9262856804853 (TaskRunner pid=2026458) Training Progress: 31%|███ | 31/100 [24:22<1:05:57, 57.36s/it] (TaskRunner pid=2026458) step:32 - global_seqlen/min:13741 - global_seqlen/max:20093 - global_seqlen/minmax_diff:6352 - global_seqlen/balanced_min:17020 - global_seqlen/balanced_max:17022 - global_seqlen/mean:17021.5 - actor/entropy:0.2510213851928711 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3174631893634796 - training/rollout_probs_diff_mean:0.005626973696053028 - training/rollout_probs_diff_std:0.014290885999798775 - training/rollout_actor_probs_pearson_corr:0.9980513453483582 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:1.0003660917282104 - rollout_corr/rollout_is_ratio_fraction_high:4.35514775745105e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.35514775745105e-05 - rollout_corr/rollout_is_max:2.588045358657837 - rollout_corr/rollout_is_min:0.49156951904296875 - rollout_corr/rollout_is_std:0.04363907128572464 - rollout_corr/rollout_is_eff_sample_size:0.9981006764057321 - rollout_corr/rollout_is_seq_mean:1.0004558563232422 - rollout_corr/rollout_is_seq_std:0.0026327052619308233 - rollout_corr/rollout_is_seq_max:1.0082557201385498 - rollout_corr/rollout_is_seq_min:0.9930630326271057 - rollout_corr/rollout_is_seq_max_deviation:0.008255720138549805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3111304240301251) - rollout_corr/training_log_ppl:np.float64(0.2656050497171236) - rollout_corr/kl:np.float64(0.0011632751664905072) - rollout_corr/k3_kl:np.float64(0.0012394887605751137) - rollout_corr/rollout_ppl:np.float64(1.3095909063704312) - rollout_corr/rollout_log_ppl:np.float64(0.2644417754927417) - rollout_corr/log_ppl_diff:np.float64(0.0011632742243818939) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027760078373830765) - rollout_corr/log_ppl_diff_max:np.float64(0.027847379446029663) - rollout_corr/log_ppl_diff_min:np.float64(-0.00688682496547699) - rollout_corr/ppl_ratio:np.float64(1.0011707202065736) - rollout_corr/chi2_token:np.float64(0.00264995195902884) - rollout_corr/chi2_seq:np.float64(1.4071137343998998) - actor/pg_loss:np.float64(0.003960146103054285) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013795204131383798) - actor/ppo_kl:np.float64(0.0005964390661219454) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(250012.3793359641) - self_distillation/token_reweight_w_std:np.float64(2860040.7311532954) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007816257420927) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07969121757196262) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.522880308330059) - perf/mfu/actor:np.float64(0.031496540810405246) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.58367156982422) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:0 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019834069535136223 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0019834069535136223 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:269.078125 - response_length/max:628.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:269.078125 - response_length_non_aborted/max:628.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.84375 - prompt_length/max:404.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002002175897359848 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3522796910256147) - timing_s/agent_loop/generate_sequences/max:np.float64(4.971057282760739) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.862827809440205) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.971057282760739) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:286 - timing_s/agent_loop/slowest/response_length:628 - timing_s/gen:10.714683732017875 - timing_s/reward:0.05403785966336727 - timing_s/old_log_prob:2.382686661556363 - timing_s/adv:0.4786151312291622 - timing_s/update_actor:15.277625290676951 - timing_s/step:28.9955370221287 - timing_s/stop_profile:0.00011957064270973206 - timing_per_token_ms/adv:0.003514783738427593 - timing_per_token_ms/gen:0.1555467703968683 - timing_per_token_ms/update_actor:0.1121935881875639 - perf/total_num_tokens:136172 - perf/time_per_step:28.9955370221287 - perf/throughput:587.0386186332606 (TaskRunner pid=2026458) Training Progress: 32%|███▏ | 32/100 [24:51<55:22, 48.86s/it] (TaskRunner pid=2026458) step:33 - global_seqlen/min:15641 - global_seqlen/max:20620 - global_seqlen/minmax_diff:4979 - global_seqlen/balanced_min:18180 - global_seqlen/balanced_max:18182 - global_seqlen/mean:18181.0 - actor/entropy:0.28735455870628357 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9959163069725037 - training/rollout_probs_diff_mean:0.00580747751519084 - training/rollout_probs_diff_std:0.015400327742099762 - training/rollout_actor_probs_pearson_corr:0.9980352520942688 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9997233152389526 - rollout_corr/rollout_is_ratio_fraction_high:1.423528101440752e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014235280104912817 - rollout_corr/rollout_is_max:2.051370859146118 - rollout_corr/rollout_is_min:0.0007158105727285147 - rollout_corr/rollout_is_std:0.04281798005104065 - rollout_corr/rollout_is_eff_sample_size:0.998169085060141 - rollout_corr/rollout_is_seq_mean:0.9997956156730652 - rollout_corr/rollout_is_seq_std:0.003073018742725253 - rollout_corr/rollout_is_seq_max:1.0114701986312866 - rollout_corr/rollout_is_seq_min:0.9903202056884766 - rollout_corr/rollout_is_seq_max_deviation:0.011470198631286621 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3598110149614513) - rollout_corr/training_log_ppl:np.float64(0.30384603922721) - rollout_corr/kl:np.float64(0.0014454446327292203) - rollout_corr/k3_kl:np.float64(0.001423622587481077) - rollout_corr/rollout_ppl:np.float64(1.3577961758710444) - rollout_corr/rollout_log_ppl:np.float64(0.3024005940533243) - rollout_corr/log_ppl_diff:np.float64(0.0014454451738856733) - rollout_corr/log_ppl_abs_diff:np.float64(0.002951021713670343) - rollout_corr/log_ppl_diff_max:np.float64(0.02602332830429077) - rollout_corr/log_ppl_diff_min:np.float64(-0.010346651077270508) - rollout_corr/ppl_ratio:np.float64(1.001454961951822) - rollout_corr/chi2_token:np.float64(0.002324617700651288) - rollout_corr/chi2_seq:np.float64(0.6557224565185606) - actor/pg_loss:np.float64(0.0037985126255080104) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009114227987083723) - actor/ppo_kl:np.float64(7.80257138011109e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(186856.52135437424) - self_distillation/token_reweight_w_std:np.float64(2469605.76739274) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999388714786619) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09890055964933708) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.5402719005942345) - perf/mfu/actor:np.float64(0.03298787544887611) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.31770038604736) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:0 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003960967063903809 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003960967063903809 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:274.40625 - response_length/max:556.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:274.40625 - response_length_non_aborted/max:556.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.75 - prompt_length/max:474.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001493934541940689 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3925712388008833) - timing_s/agent_loop/generate_sequences/max:np.float64(4.813172671943903) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.01079661748372) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.813172671943903) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:198 - timing_s/agent_loop/slowest/response_length:556 - timing_s/gen:10.570713378489017 - timing_s/reward:0.056401949375867844 - timing_s/old_log_prob:2.26583200879395 - timing_s/adv:0.4962475970387459 - timing_s/update_actor:15.83989292383194 - timing_s/step:29.316704677417874 - timing_s/stop_profile:3.923475742340088e-05 - timing_per_token_ms/adv:0.00341185576315072 - timing_per_token_ms/gen:0.1504770723506579 - timing_per_token_ms/update_actor:0.10890416453874883 - perf/total_num_tokens:145448 - perf/time_per_step:29.316704677417874 - perf/throughput:620.1583772818947 (TaskRunner pid=2026458) Training Progress: 33%|███▎ | 33/100 [25:20<48:01, 43.01s/it] (TaskRunner pid=2026458) step:34 - global_seqlen/min:15048 - global_seqlen/max:22716 - global_seqlen/minmax_diff:7668 - global_seqlen/balanced_min:18406 - global_seqlen/balanced_max:18407 - global_seqlen/mean:18406.75 - actor/entropy:0.2822253406047821 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3892279863357544 - training/rollout_probs_diff_mean:0.00599341094493866 - training/rollout_probs_diff_std:0.014615834690630436 - training/rollout_actor_probs_pearson_corr:0.9981156587600708 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:0.9999809265136719 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00012492018868215382 - rollout_corr/rollout_is_max:1.9517436027526855 - rollout_corr/rollout_is_min:0.10854519158601761 - rollout_corr/rollout_is_std:0.04540359228849411 - rollout_corr/rollout_is_eff_sample_size:0.9979426360193112 - rollout_corr/rollout_is_seq_mean:0.9999065399169922 - rollout_corr/rollout_is_seq_std:0.003091080579906702 - rollout_corr/rollout_is_seq_max:1.0100407600402832 - rollout_corr/rollout_is_seq_min:0.9879905581474304 - rollout_corr/rollout_is_seq_max_deviation:0.01200944185256958 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3448055190965533) - rollout_corr/training_log_ppl:np.float64(0.29254834007588215) - rollout_corr/kl:np.float64(0.001122740535322464) - rollout_corr/k3_kl:np.float64(0.0011786174198959998) - rollout_corr/rollout_ppl:np.float64(1.3432867326773703) - rollout_corr/rollout_log_ppl:np.float64(0.2914255977375433) - rollout_corr/log_ppl_diff:np.float64(0.0011227423383388668) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025718869001138955) - rollout_corr/log_ppl_diff_max:np.float64(0.014177501201629639) - rollout_corr/log_ppl_diff_min:np.float64(-0.006468132138252258) - rollout_corr/ppl_ratio:np.float64(1.0011283229105175) - rollout_corr/chi2_token:np.float64(0.002466450445353985) - rollout_corr/chi2_seq:np.float64(0.9479083714541048) - actor/pg_loss:np.float64(0.005841488018631935) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011660458308142552) - actor/ppo_kl:np.float64(-8.996075499190237e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_w_mean:np.float64(205396.02536904486) - self_distillation/token_reweight_w_std:np.float64(2610298.4065771694) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991585344541818) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12581025002873503) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.6120489835739136) - perf/mfu/actor:np.float64(0.03423697654860308) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.21736526489258) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012471199035644531 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012471199035644531 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:281.4296875 - response_length/max:743.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:281.4296875 - response_length_non_aborted/max:743.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.78125 - prompt_length/max:621.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.58306884765625e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4199188239872456) - timing_s/agent_loop/generate_sequences/max:np.float64(5.984945947304368) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2054700024527847) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.984945947304368) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:207 - timing_s/agent_loop/slowest/response_length:743 - timing_s/gen:11.809425836429 - timing_s/reward:0.05546233803033829 - timing_s/old_log_prob:2.252874342724681 - timing_s/adv:0.49194580502808094 - timing_s/update_actor:15.391067944467068 - timing_s/step:30.08788885548711 - timing_s/stop_profile:0.00011718645691871643 - timing_per_token_ms/adv:0.003340797567659153 - timing_per_token_ms/gen:0.16391507976055575 - timing_per_token_ms/update_actor:0.10452054235855779 - perf/total_num_tokens:147254 - perf/time_per_step:30.08788885548711 - perf/throughput:611.7660859626305 (TaskRunner pid=2026458) Training Progress: 34%|███▍ | 34/100 [25:50<43:03, 39.14s/it] (TaskRunner pid=2026458) step:35 - global_seqlen/min:14987 - global_seqlen/max:19906 - global_seqlen/minmax_diff:4919 - global_seqlen/balanced_min:16938 - global_seqlen/balanced_max:16939 - global_seqlen/mean:16938.25 - actor/entropy:0.294697642326355 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22079068422317505 - training/rollout_probs_diff_mean:0.005864571314305067 - training/rollout_probs_diff_std:0.013605127111077309 - training/rollout_actor_probs_pearson_corr:0.9984296560287476 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.5859375 - self_distillation/correct_sample_fraction:0.5078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.5859375 - rollout_corr/rollout_is_mean:1.00005042552948 - rollout_corr/rollout_is_ratio_fraction_high:3.1109037081478164e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.1109037081478164e-05 - rollout_corr/rollout_is_max:2.7191102504730225 - rollout_corr/rollout_is_min:0.37482571601867676 - rollout_corr/rollout_is_std:0.04332791268825531 - rollout_corr/rollout_is_eff_sample_size:0.9981263285522803 - rollout_corr/rollout_is_seq_mean:1.0001163482666016 - rollout_corr/rollout_is_seq_std:0.0030649646650999784 - rollout_corr/rollout_is_seq_max:1.0084450244903564 - rollout_corr/rollout_is_seq_min:0.989280641078949 - rollout_corr/rollout_is_seq_max_deviation:0.010719358921051025 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3609930002130568) - rollout_corr/training_log_ppl:np.float64(0.3051015918608755) - rollout_corr/kl:np.float64(0.0011746971918284999) - rollout_corr/k3_kl:np.float64(0.0011032617354658214) - rollout_corr/rollout_ppl:np.float64(1.3593582352623343) - rollout_corr/rollout_log_ppl:np.float64(0.30392689420841634) - rollout_corr/log_ppl_diff:np.float64(0.0011746976524591446) - rollout_corr/log_ppl_abs_diff:np.float64(0.002850949880667031) - rollout_corr/log_ppl_diff_max:np.float64(0.012322753667831421) - rollout_corr/log_ppl_diff_min:np.float64(-0.010998740792274475) - rollout_corr/ppl_ratio:np.float64(1.0011811451986432) - rollout_corr/chi2_token:np.float64(0.0021009566262364388) - rollout_corr/chi2_seq:np.float64(1.5435644236858934) - actor/pg_loss:np.float64(0.0014815301401540637) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020538957323879004) - actor/ppo_kl:np.float64(0.00030627488916046275) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.5859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5078125) - self_distillation/token_reweight_w_mean:np.float64(94700.63521376462) - self_distillation/token_reweight_w_std:np.float64(1075744.331360741) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004025450907648) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03615615499438718) - self_distillation/empty_target_batch:np.float64(0.4140625) - actor/grad_norm:np.float64(0.3742603175342083) - perf/mfu/actor:np.float64(0.031170659142369998) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1671371459961) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:0 - critic/score/mean:0.5078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0029106393922120333 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0029106393922120333 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:251.1328125 - response_length/max:502.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:251.1328125 - response_length_non_aborted/max:502.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.1875 - prompt_length/max:514.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001547783613204956 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1277420744299889) - timing_s/agent_loop/generate_sequences/max:np.float64(4.495345920324326) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.789818469223974) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.495345920324326) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:355 - timing_s/agent_loop/slowest/response_length:502 - timing_s/gen:10.292290568351746 - timing_s/reward:0.054091084748506546 - timing_s/old_log_prob:2.2543541602790356 - timing_s/adv:0.4955630674958229 - timing_s/update_actor:15.42586961016059 - timing_s/step:28.60832902789116 - timing_s/stop_profile:3.491714596748352e-05 - timing_per_token_ms/adv:0.003657130071700315 - timing_per_token_ms/gen:0.16009162495491905 - timing_per_token_ms/update_actor:0.11383901532153992 - perf/total_num_tokens:135506 - perf/time_per_step:28.60832902789116 - perf/throughput:592.0740768706333 (TaskRunner pid=2026458) Training Progress: 35%|███▌ | 35/100 [26:19<38:59, 35.99s/it] (TaskRunner pid=2026458) step:36 - global_seqlen/min:14431 - global_seqlen/max:20659 - global_seqlen/minmax_diff:6228 - global_seqlen/balanced_min:17408 - global_seqlen/balanced_max:17413 - global_seqlen/mean:17411.75 - actor/entropy:0.2818928360939026 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30500367283821106 - training/rollout_probs_diff_mean:0.005986991338431835 - training/rollout_probs_diff_std:0.014329608529806137 - training/rollout_actor_probs_pearson_corr:0.9982293248176575 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:1.0000543594360352 - rollout_corr/rollout_is_ratio_fraction_high:1.4678683328384068e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.871473331353627e-05 - rollout_corr/rollout_is_max:3.0542242527008057 - rollout_corr/rollout_is_min:0.18286029994487762 - rollout_corr/rollout_is_std:0.044746387749910355 - rollout_corr/rollout_is_eff_sample_size:0.9980018804571298 - rollout_corr/rollout_is_seq_mean:1.0000989437103271 - rollout_corr/rollout_is_seq_std:0.0029557279776781797 - rollout_corr/rollout_is_seq_max:1.0128530263900757 - rollout_corr/rollout_is_seq_min:0.992638111114502 - rollout_corr/rollout_is_seq_max_deviation:0.012853026390075684 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3384348950348794) - rollout_corr/training_log_ppl:np.float64(0.28769120943616144) - rollout_corr/kl:np.float64(0.0010638975770804748) - rollout_corr/k3_kl:np.float64(0.0012365762453327989) - rollout_corr/rollout_ppl:np.float64(1.3369859019294381) - rollout_corr/rollout_log_ppl:np.float64(0.28662730925134383) - rollout_corr/log_ppl_diff:np.float64(0.0010639001848176122) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025838720030151308) - rollout_corr/log_ppl_diff_max:np.float64(0.01416441798210144) - rollout_corr/log_ppl_diff_min:np.float64(-0.009993910789489746) - rollout_corr/ppl_ratio:np.float64(1.0010696118697524) - rollout_corr/chi2_token:np.float64(0.002987496554851532) - rollout_corr/chi2_seq:np.float64(0.8903114418499172) - actor/pg_loss:np.float64(0.0051360129145905375) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000932457276576315) - actor/ppo_kl:np.float64(0.00010139281163645819) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(95758.95235729823) - self_distillation/token_reweight_w_std:np.float64(1392223.179759777) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9973206941504031) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10905869025737047) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.5497298240661621) - perf/mfu/actor:np.float64(0.03256102036135432) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1156120300293) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007596218958497047 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007596218958497047 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:266.1171875 - response_length/max:662.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:266.1171875 - response_length_non_aborted/max:662.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.0 - prompt_length/max:704.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.2092596888542175e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3356867656111717) - timing_s/agent_loop/generate_sequences/max:np.float64(5.081710338592529) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.919884198134241) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.081710338592529) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:379 - timing_s/agent_loop/slowest/response_length:662 - timing_s/gen:10.837040143087506 - timing_s/reward:0.05424739792943001 - timing_s/old_log_prob:2.3121078684926033 - timing_s/adv:0.5094937365502119 - timing_s/update_actor:15.35438490100205 - timing_s/step:29.153722267597914 - timing_s/stop_profile:3.104284405708313e-05 - timing_per_token_ms/adv:0.0036576861641579098 - timing_per_token_ms/gen:0.1590734835905162 - timing_per_token_ms/update_actor:0.11023005227075143 - perf/total_num_tokens:139294 - perf/time_per_step:29.153722267597914 - perf/throughput:597.2393452945732 (TaskRunner pid=2026458) Training Progress: 36%|███▌ | 36/100 [26:48<36:12, 33.94s/it] (TaskRunner pid=2026458) step:37 - global_seqlen/min:15833 - global_seqlen/max:21178 - global_seqlen/minmax_diff:5345 - global_seqlen/balanced_min:18583 - global_seqlen/balanced_max:18587 - global_seqlen/mean:18585.625 - actor/entropy:0.285804808139801 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4652300179004669 - training/rollout_probs_diff_mean:0.005965293850749731 - training/rollout_probs_diff_std:0.014233550056815147 - training/rollout_actor_probs_pearson_corr:0.9982559680938721 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:1.0002163648605347 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.9148995963623747e-05 - rollout_corr/rollout_is_max:1.695287823677063 - rollout_corr/rollout_is_min:0.16980920732021332 - rollout_corr/rollout_is_std:0.04377271980047226 - rollout_corr/rollout_is_eff_sample_size:0.9980885633504123 - rollout_corr/rollout_is_seq_mean:1.000288486480713 - rollout_corr/rollout_is_seq_std:0.0028211281169205904 - rollout_corr/rollout_is_seq_max:1.0087376832962036 - rollout_corr/rollout_is_seq_min:0.9919080138206482 - rollout_corr/rollout_is_seq_max_deviation:0.008737683296203613 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3575056260451674) - rollout_corr/training_log_ppl:np.float64(0.3016296330606565) - rollout_corr/kl:np.float64(0.0007570147956474926) - rollout_corr/k3_kl:np.float64(0.0012164926273499077) - rollout_corr/rollout_ppl:np.float64(1.3564373790286481) - rollout_corr/rollout_log_ppl:np.float64(0.30087261548032984) - rollout_corr/log_ppl_diff:np.float64(0.0007570175803266466) - rollout_corr/log_ppl_abs_diff:np.float64(0.002560925087891519) - rollout_corr/log_ppl_diff_max:np.float64(0.011357903480529785) - rollout_corr/log_ppl_diff_min:np.float64(-0.008871495723724365) - rollout_corr/ppl_ratio:np.float64(1.000762677518651) - rollout_corr/chi2_token:np.float64(0.0033016859088093042) - rollout_corr/chi2_seq:np.float64(1.0105145464185625) - actor/pg_loss:np.float64(0.0031967939576134086) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006711441328661749) - actor/ppo_kl:np.float64(3.3348735911431504e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_w_mean:np.float64(214368.9940047427) - self_distillation/token_reweight_w_std:np.float64(2919033.6983316364) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007499558851123) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07807663449784741) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.4402663931250572) - perf/mfu/actor:np.float64(0.034623907048585915) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.02531814575195) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0031590224243700504 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0031590224243700504 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:268.01953125 - response_length/max:686.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:268.01953125 - response_length_non_aborted/max:686.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.78125 - prompt_length/max:1009.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.923235297203064e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.457766680046916) - timing_s/agent_loop/generate_sequences/max:np.float64(5.311068296432495) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9142332015690044) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.311068296432495) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:258 - timing_s/agent_loop/slowest/response_length:686 - timing_s/gen:11.104413088411093 - timing_s/reward:0.05539845488965511 - timing_s/old_log_prob:2.2851091288030148 - timing_s/adv:0.5124162267893553 - timing_s/update_actor:15.311178967356682 - timing_s/step:29.35571401193738 - timing_s/stop_profile:0.00011339224874973297 - timing_per_token_ms/adv:0.003446320925374821 - timing_per_token_ms/gen:0.1618412412867983 - timing_per_token_ms/update_actor:0.10297729406030656 - perf/total_num_tokens:148685 - perf/time_per_step:29.35571401193738 - perf/throughput:633.1177975246056 (TaskRunner pid=2026458) Training Progress: 37%|███▋ | 37/100 [27:18<34:12, 32.58s/it] (TaskRunner pid=2026458) step:38 - global_seqlen/min:15292 - global_seqlen/max:19759 - global_seqlen/minmax_diff:4467 - global_seqlen/balanced_min:17036 - global_seqlen/balanced_max:17040 - global_seqlen/mean:17038.625 - actor/entropy:0.2831912040710449 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4492228925228119 - training/rollout_probs_diff_mean:0.006005707662552595 - training/rollout_probs_diff_std:0.014447803609073162 - training/rollout_actor_probs_pearson_corr:0.9981924891471863 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9998399019241333 - rollout_corr/rollout_is_ratio_fraction_high:2.9010312573518604e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.3515468860277906e-05 - rollout_corr/rollout_is_max:4.976550579071045 - rollout_corr/rollout_is_min:0.1723451316356659 - rollout_corr/rollout_is_std:0.045169975608587265 - rollout_corr/rollout_is_eff_sample_size:0.997963056153362 - rollout_corr/rollout_is_seq_mean:0.999801754951477 - rollout_corr/rollout_is_seq_std:0.0029568150639533997 - rollout_corr/rollout_is_seq_max:1.01389741897583 - rollout_corr/rollout_is_seq_min:0.9917296171188354 - rollout_corr/rollout_is_seq_max_deviation:0.013897418975830078 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3477132506668568) - rollout_corr/training_log_ppl:np.float64(0.29560438107000664) - rollout_corr/kl:np.float64(0.0015795158470695014) - rollout_corr/k3_kl:np.float64(0.0011915537743334426) - rollout_corr/rollout_ppl:np.float64(1.3455825243145227) - rollout_corr/rollout_log_ppl:np.float64(0.29402486264007166) - rollout_corr/log_ppl_diff:np.float64(0.0015795184299349785) - rollout_corr/log_ppl_abs_diff:np.float64(0.002752322005107999) - rollout_corr/log_ppl_diff_max:np.float64(0.010404467582702637) - rollout_corr/log_ppl_diff_min:np.float64(-0.006978675723075867) - rollout_corr/ppl_ratio:np.float64(1.0015854325611144) - rollout_corr/chi2_token:np.float64(0.0016510849818587303) - rollout_corr/chi2_seq:np.float64(0.42648625490255654) - actor/pg_loss:np.float64(0.004350315313786268) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001112287538489909) - actor/ppo_kl:np.float64(0.0002975818272203945) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(277881.75766049954) - self_distillation/token_reweight_w_std:np.float64(3405953.521727615) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997533587738872) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09852648590458557) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.5438188463449478) - perf/mfu/actor:np.float64(0.03193194428398073) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.85803985595703) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:0 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00237884558737278 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00237884558737278 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:269.30078125 - response_length/max:549.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:269.30078125 - response_length_non_aborted/max:549.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.15625 - prompt_length/max:500.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013971328735351562 - timing_s/agent_loop/generate_sequences/min:np.float64(1.483020644634962) - timing_s/agent_loop/generate_sequences/max:np.float64(4.714750545099378) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0552332281222334) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.714750545099378) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:338 - timing_s/agent_loop/slowest/response_length:549 - timing_s/gen:10.464355397969484 - timing_s/reward:0.05412936769425869 - timing_s/old_log_prob:2.2855528369545937 - timing_s/adv:0.4730773251503706 - timing_s/update_actor:15.16877150721848 - timing_s/step:28.533766953274608 - timing_s/stop_profile:0.00012415647506713867 - timing_per_token_ms/adv:0.003470624281231398 - timing_per_token_ms/gen:0.15178711358943855 - timing_per_token_ms/update_actor:0.11128224480568767 - perf/total_num_tokens:136309 - perf/time_per_step:28.533766953274608 - perf/throughput:597.1389977321099 (TaskRunner pid=2026458) Training Progress: 38%|███▊ | 38/100 [27:46<32:25, 31.38s/it] (TaskRunner pid=2026458) step:39 - global_seqlen/min:15017 - global_seqlen/max:21775 - global_seqlen/minmax_diff:6758 - global_seqlen/balanced_min:17748 - global_seqlen/balanced_max:17751 - global_seqlen/mean:17750.375 - actor/entropy:0.2934376001358032 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4744115471839905 - training/rollout_probs_diff_mean:0.006076316349208355 - training/rollout_probs_diff_std:0.014462748542428017 - training/rollout_actor_probs_pearson_corr:0.9982326030731201 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:1.0001943111419678 - rollout_corr/rollout_is_ratio_fraction_high:2.8678357921307907e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.735671584261581e-05 - rollout_corr/rollout_is_max:2.3144755363464355 - rollout_corr/rollout_is_min:0.44314810633659363 - rollout_corr/rollout_is_std:0.04461565613746643 - rollout_corr/rollout_is_eff_sample_size:0.9980141101447896 - rollout_corr/rollout_is_seq_mean:1.0003681182861328 - rollout_corr/rollout_is_seq_std:0.0030234723817557096 - rollout_corr/rollout_is_seq_max:1.012935996055603 - rollout_corr/rollout_is_seq_min:0.991753876209259 - rollout_corr/rollout_is_seq_max_deviation:0.012935996055603027 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3561590211465955) - rollout_corr/training_log_ppl:np.float64(0.30050787163781933) - rollout_corr/kl:np.float64(0.0008042539968293738) - rollout_corr/k3_kl:np.float64(0.0011359995904740572) - rollout_corr/rollout_ppl:np.float64(1.355069079902023) - rollout_corr/rollout_log_ppl:np.float64(0.2997036176966503) - rollout_corr/log_ppl_diff:np.float64(0.0008042539411690086) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024447946052532643) - rollout_corr/log_ppl_diff_max:np.float64(0.010187238454818726) - rollout_corr/log_ppl_diff_min:np.float64(-0.010493099689483643) - rollout_corr/ppl_ratio:np.float64(1.0008093006908894) - rollout_corr/chi2_token:np.float64(0.0029845864046365023) - rollout_corr/chi2_seq:np.float64(0.9699735399335623) - actor/pg_loss:np.float64(0.003664600197225809) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006868128334645007) - actor/ppo_kl:np.float64(0.00012619273251779362) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(231536.99509470444) - self_distillation/token_reweight_w_std:np.float64(3103805.8869296443) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012708925642073) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08035396633204073) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.45319733768701553) - perf/mfu/actor:np.float64(0.03298738163573484) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.00569915771484) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:0 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005079654045403004 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005079654045403004 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:272.41796875 - response_length/max:558.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:272.41796875 - response_length_non_aborted/max:558.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:282.28125 - prompt_length/max:452.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.198114275932312e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3789098169654608) - timing_s/agent_loop/generate_sequences/max:np.float64(4.869980722665787) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.020734860765515) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.869980722665787) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:365 - timing_s/agent_loop/slowest/response_length:558 - timing_s/gen:10.739610413089395 - timing_s/reward:0.054600564762949944 - timing_s/old_log_prob:2.247015619650483 - timing_s/adv:0.5021204594522715 - timing_s/update_actor:15.248395329341292 - timing_s/step:28.878361692652106 - timing_s/stop_profile:0.00011930800974369049 - timing_per_token_ms/adv:0.0035359848697018477 - timing_per_token_ms/gen:0.15399719544429077 - timing_per_token_ms/update_actor:0.10738079709119731 - perf/total_num_tokens:142003 - perf/time_per_step:28.878361692652106 - perf/throughput:614.6600416226678 (TaskRunner pid=2026458) Training Progress: 39%|███▉ | 39/100 [28:15<31:09, 30.65s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) To determine the correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to analyze the structure of the product and compare it with the options provided. The product contains a nitrogen atom bonded to a carbon chain, with a substituted aromatic ring (Cc2ccc(Cl)cc2) and a trifluoromethyl group (C(F)(F)F). (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A includes a structure with a ketone group (O=C), an aromatic ring (c2ccccc2), and a nitrogen atom (N) connected to a carbon chain. The nitrogen is bonded to a substituted aromatic ring with a chlorine atom and a trifluoromethyl group, matching the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B includes a structure with a ketone group (O=C), an aromatic ring (c2ccccc2), and a nitrogen atom (N) connected to a carbon chain. However, the substituents on the nitrogen do not match the product's structure, as it lacks the specific chlorine and trifluoromethyl groups. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C includes a structure with a ketone group (O=C), an aromatic ring (c2ccccc2), and a nitrogen atom (N) connected to a carbon chain. However, the substituents on the nitrogen do not match the product's structure, as it includes an oxygen group (Oc2) instead of the chlorine and trifluoromethyl groups. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D includes a structure with a ketone group (O=C), an aromatic ring (c2ccccc2), and a nitrogen atom (N) connected to a carbon chain. However, the substituents on the nitrogen do not match the product's structure, as it includes a double bond (CC=C) and a different trifluoromethyl group. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Based on the structural analysis, Option A is the only one that matches the product's structure, making it the correct reactant. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30/actor (TaskRunner pid=2026458) step:40 - global_seqlen/min:16589 - global_seqlen/max:23076 - global_seqlen/minmax_diff:6487 - global_seqlen/balanced_min:19038 - global_seqlen/balanced_max:19042 - global_seqlen/mean:19040.5 - actor/entropy:0.3083128035068512 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48448076844215393 - training/rollout_probs_diff_mean:0.006092434749007225 - training/rollout_probs_diff_std:0.01408663671463728 - training/rollout_actor_probs_pearson_corr:0.9983260631561279 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.62109375 - self_distillation/correct_sample_fraction:0.54296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.62109375 - rollout_corr/rollout_is_mean:1.0004887580871582 - rollout_corr/rollout_is_ratio_fraction_high:2.767170371953398e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.383585185976699e-05 - rollout_corr/rollout_is_max:2.5501346588134766 - rollout_corr/rollout_is_min:0.2322635054588318 - rollout_corr/rollout_is_std:0.044838204979896545 - rollout_corr/rollout_is_eff_sample_size:0.9979954688959803 - rollout_corr/rollout_is_seq_mean:1.0004291534423828 - rollout_corr/rollout_is_seq_std:0.00280501926317811 - rollout_corr/rollout_is_seq_max:1.0100523233413696 - rollout_corr/rollout_is_seq_min:0.9925862550735474 - rollout_corr/rollout_is_seq_max_deviation:0.010052323341369629 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3748237513937056) - rollout_corr/training_log_ppl:np.float64(0.3138403666380327) - rollout_corr/kl:np.float64(0.0008364543262393553) - rollout_corr/k3_kl:np.float64(0.0011391592736345046) - rollout_corr/rollout_ppl:np.float64(1.3736085868440568) - rollout_corr/rollout_log_ppl:np.float64(0.3130039106181357) - rollout_corr/log_ppl_diff:np.float64(0.0008364560198970139) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023778348113410175) - rollout_corr/log_ppl_diff_max:np.float64(0.009519308805465698) - rollout_corr/log_ppl_diff_min:np.float64(-0.0073702335357666016) - rollout_corr/ppl_ratio:np.float64(1.0008412639144808) - rollout_corr/chi2_token:np.float64(0.0029417627956718206) - rollout_corr/chi2_seq:np.float64(0.8329082692507654) - actor/pg_loss:np.float64(0.0021506232442334294) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00047293603620346403) - actor/ppo_kl:np.float64(0.0002316894797989022) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.62109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.54296875) - self_distillation/token_reweight_w_mean:np.float64(45010.86430421425) - self_distillation/token_reweight_w_std:np.float64(529273.9801185789) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998293393291533) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05143398354994133) - self_distillation/empty_target_batch:np.float64(0.37890625) - actor/grad_norm:np.float64(0.38673160597682) - perf/mfu/actor:np.float64(0.03526754309862706) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.86711502075195) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6357142857142857) - val-aux/sciknoweval/reward/std@16:np.float64(0.16134453805796828) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6995523809523809) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.11925658429648062) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5717714285714286) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15210733801155413) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6355761904761905) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.16083633014996535) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7442000000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.07791528897318264) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5117523809523808) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1301078901202422) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.647747619047619) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.12547152462394415) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7727952380952381) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.04748400982093751) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.45995714285714284) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10409181623779554) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6544619047619048) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09064171674985216) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7905285714285714) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.026333374195131425) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.41540476190476183) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07673490521638164) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6577380952380952) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06656845162631347) - val-aux/sciknoweval/score/mean@16:np.float64(0.6357142857142857) - val-aux/sciknoweval/score/std@16:np.float64(0.16134453805796828) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6995523809523809) - val-aux/sciknoweval/score/best@2/std:np.float64(0.11925658429648062) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5717714285714286) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15210733801155413) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6355761904761905) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.16083633014996535) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7442000000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.07791528897318264) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5117523809523808) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1301078901202422) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.647747619047619) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.12547152462394415) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7727952380952381) - val-aux/sciknoweval/score/best@8/std:np.float64(0.04748400982093751) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.45995714285714284) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10409181623779554) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6544619047619048) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09064171674985216) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7905285714285714) - val-aux/sciknoweval/score/best@16/std:np.float64(0.026333374195131425) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.41540476190476183) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07673490521638164) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6577380952380952) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06656845162631347) - val-core/sciknoweval/acc/mean@16:np.float64(0.6357142857142857) - val-aux/sciknoweval/acc/std@16:np.float64(0.16134453805796828) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6995523809523809) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.11925658429648062) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5717714285714286) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15210733801155413) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6355761904761905) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.16083633014996535) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7442000000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.07791528897318264) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5117523809523808) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1301078901202422) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.647747619047619) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.12547152462394415) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7727952380952381) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.04748400982093751) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.45995714285714284) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10409181623779554) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6544619047619048) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09064171674985216) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7905285714285714) - val-core/sciknoweval/acc/best@16/std:np.float64(0.026333374195131425) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.41540476190476183) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07673490521638164) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6577380952380952) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06656845162631347) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.999702380952381) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0011526736149426837) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999857142857144) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0002604287412166728) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9994380952380952) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0015362291495737217) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9997095238095238) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0011396681698700574) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9989476190476191) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.001975811782419187) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9999333333333333) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0005594782149609339) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9980619047619048) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0023394041530661654) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.999995238095238) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.00015050933932646775) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9969285714285715) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.002278634869993062) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:0 - critic/score/mean:0.54296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.54296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0034849049989134073 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0034849049989134073 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:282.328125 - response_length/max:680.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:282.328125 - response_length_non_aborted/max:680.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.6875 - prompt_length/max:501.0 - prompt_length/min:159.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018454715609550476 - timing_s/agent_loop/generate_sequences/min:np.float64(1.287922067567706) - timing_s/agent_loop/generate_sequences/max:np.float64(5.481446422636509) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0874510004068725) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.481446422636509) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:272 - timing_s/agent_loop/slowest/response_length:680 - timing_s/gen:11.202703416347504 - timing_s/reward:0.05725381709635258 - timing_s/old_log_prob:2.3498482443392277 - timing_s/adv:0.48736315965652466 - timing_s/update_actor:15.279724219813943 - timing_s/step:29.463446816429496 - timing_s/testing:112.37399195507169 - timing_s/save_checkpoint:6.7144703436642885 - timing_s/stop_profile:0.00011608190834522247 - timing_per_token_ms/adv:0.003199516554558209 - timing_per_token_ms/gen:0.15499894039995993 - timing_per_token_ms/update_actor:0.10031068130966848 - perf/total_num_tokens:152324 - perf/time_per_step:29.463446816429496 - perf/throughput:646.2414298853378 (TaskRunner pid=2026458) Training Progress: 40%|████ | 40/100 [30:44<1:06:02, 66.04s/it] (TaskRunner pid=2026458) step:41 - global_seqlen/min:14161 - global_seqlen/max:22430 - global_seqlen/minmax_diff:8269 - global_seqlen/balanced_min:17529 - global_seqlen/balanced_max:17569 - global_seqlen/mean:17540.0 - actor/entropy:0.28421929478645325 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3167548179626465 - training/rollout_probs_diff_mean:0.005681131035089493 - training/rollout_probs_diff_std:0.013473561964929104 - training/rollout_actor_probs_pearson_corr:0.9984502792358398 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.59375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0002758502960205 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.967482328414917 - rollout_corr/rollout_is_min:0.5226783752441406 - rollout_corr/rollout_is_std:0.04252113774418831 - rollout_corr/rollout_is_eff_sample_size:0.998196166076475 - rollout_corr/rollout_is_seq_mean:1.000309944152832 - rollout_corr/rollout_is_seq_std:0.002816120395436883 - rollout_corr/rollout_is_seq_max:1.0100390911102295 - rollout_corr/rollout_is_seq_min:0.9916365146636963 - rollout_corr/rollout_is_seq_max_deviation:0.010039091110229492 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3460121774114668) - rollout_corr/training_log_ppl:np.float64(0.29434393605333753) - rollout_corr/kl:np.float64(0.0006344841518632904) - rollout_corr/k3_kl:np.float64(0.0010341189710061371) - rollout_corr/rollout_ppl:np.float64(1.3451492232270539) - rollout_corr/rollout_log_ppl:np.float64(0.2937094499648083) - rollout_corr/log_ppl_diff:np.float64(0.0006344860885292292) - rollout_corr/log_ppl_abs_diff:np.float64(0.002157465787604451) - rollout_corr/log_ppl_diff_max:np.float64(0.00896558165550232) - rollout_corr/log_ppl_diff_min:np.float64(-0.008629709482192993) - rollout_corr/ppl_ratio:np.float64(1.000638237921521) - rollout_corr/chi2_token:np.float64(0.0028806456830352545) - rollout_corr/chi2_seq:np.float64(0.6779269557446241) - actor/pg_loss:np.float64(0.004367844783701003) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004992729859623068) - actor/ppo_kl:np.float64(5.57020750591164e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59375) - self_distillation/token_reweight_w_mean:np.float64(156417.83956303308) - self_distillation/token_reweight_w_std:np.float64(2357416.5479677515) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000363286351785) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0830252695013769) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.500577874481678) - perf/mfu/actor:np.float64(0.032587394156940674) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.82418060302734) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0022402419708669186 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0022402419708669186 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:284.21875 - response_length/max:788.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:284.21875 - response_length_non_aborted/max:788.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.90625 - prompt_length/max:631.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017307698726654053 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2802286799997091) - timing_s/agent_loop/generate_sequences/max:np.float64(5.627637119963765) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.104791782410757) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.627637119963765) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:423 - timing_s/agent_loop/slowest/response_length:788 - timing_s/gen:11.781217468902469 - timing_s/reward:0.05423154681921005 - timing_s/old_log_prob:2.465151973068714 - timing_s/adv:0.5479183085262775 - timing_s/update_actor:15.487478408962488 - timing_s/step:30.435431083664298 - timing_s/stop_profile:0.00011926889419555664 - timing_per_token_ms/adv:0.0039047769991895492 - timing_per_token_ms/gen:0.16191887670289265 - timing_per_token_ms/update_actor:0.11037256562829595 - perf/total_num_tokens:140320 - perf/time_per_step:30.435431083664298 - perf/throughput:576.3020064274463 (TaskRunner pid=2026458) Training Progress: 41%|████ | 41/100 [31:14<54:26, 55.37s/it] (TaskRunner pid=2026458) step:42 - global_seqlen/min:15278 - global_seqlen/max:20664 - global_seqlen/minmax_diff:5386 - global_seqlen/balanced_min:18018 - global_seqlen/balanced_max:18114 - global_seqlen/mean:18031.0 - actor/entropy:0.2842845022678375 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.289776086807251 - training/rollout_probs_diff_mean:0.005445326212793589 - training/rollout_probs_diff_std:0.013068844564259052 - training/rollout_actor_probs_pearson_corr:0.9985550045967102 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9997981786727905 - rollout_corr/rollout_is_ratio_fraction_high:1.2730420166917611e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.5460840333835222e-05 - rollout_corr/rollout_is_max:2.036485433578491 - rollout_corr/rollout_is_min:0.43032172322273254 - rollout_corr/rollout_is_std:0.04114258661866188 - rollout_corr/rollout_is_eff_sample_size:0.998309494689457 - rollout_corr/rollout_is_seq_mean:0.999789297580719 - rollout_corr/rollout_is_seq_std:0.0025704929139465094 - rollout_corr/rollout_is_seq_max:1.008382797241211 - rollout_corr/rollout_is_seq_min:0.9924810528755188 - rollout_corr/rollout_is_seq_max_deviation:0.008382797241210938 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3495313432067633) - rollout_corr/training_log_ppl:np.float64(0.2967612352222204) - rollout_corr/kl:np.float64(0.0012985346789022856) - rollout_corr/k3_kl:np.float64(0.0010939320847001) - rollout_corr/rollout_ppl:np.float64(1.3477259748615324) - rollout_corr/rollout_log_ppl:np.float64(0.29546269873389974) - rollout_corr/log_ppl_diff:np.float64(0.0012985364883206785) - rollout_corr/log_ppl_abs_diff:np.float64(0.002426193736027926) - rollout_corr/log_ppl_diff_max:np.float64(0.00859573483467102) - rollout_corr/log_ppl_diff_min:np.float64(-0.006498366594314575) - rollout_corr/ppl_ratio:np.float64(1.0013033242430538) - rollout_corr/chi2_token:np.float64(0.001749418443068862) - rollout_corr/chi2_seq:np.float64(0.668122704140842) - actor/pg_loss:np.float64(0.0077214669436216354) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001590086207670538) - actor/ppo_kl:np.float64(0.0002024844293835315) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(176662.73121298593) - self_distillation/token_reweight_w_std:np.float64(2621435.3882032083) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9981021112762392) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14360627194400877) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.6128042787313461) - perf/mfu/actor:np.float64(0.03360615388418303) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.27114486694336) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003445170121267438 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003445170121267438 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:306.84375 - response_length/max:965.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:306.84375 - response_length_non_aborted/max:965.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:256.625 - prompt_length/max:557.0 - prompt_length/min:175.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013544969260692596 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3969923835247755) - timing_s/agent_loop/generate_sequences/max:np.float64(6.683027235791087) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.339469947764883) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.683027235791087) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:965 - timing_s/gen:13.290308959782124 - timing_s/reward:0.05642969720065594 - timing_s/old_log_prob:2.2229634914547205 - timing_s/adv:0.47765903919935226 - timing_s/update_actor:15.349972175434232 - timing_s/step:31.484555166214705 - timing_s/stop_profile:3.08305025100708e-05 - timing_per_token_ms/adv:0.003311373739666077 - timing_per_token_ms/gen:0.16919122313603885 - timing_per_token_ms/update_actor:0.10641376085238084 - perf/total_num_tokens:144248 - perf/time_per_step:31.484555166214705 - perf/throughput:572.6934970117862 (TaskRunner pid=2026458) Training Progress: 42%|████▏ | 42/100 [31:46<46:36, 48.21s/it] (TaskRunner pid=2026458) step:43 - global_seqlen/min:17746 - global_seqlen/max:21005 - global_seqlen/minmax_diff:3259 - global_seqlen/balanced_min:19023 - global_seqlen/balanced_max:19027 - global_seqlen/mean:19025.375 - actor/entropy:0.29123467206954956 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.277299165725708 - training/rollout_probs_diff_mean:0.0057848598808050156 - training/rollout_probs_diff_std:0.013805463910102844 - training/rollout_actor_probs_pearson_corr:0.9984029531478882 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:0.9998417496681213 - rollout_corr/rollout_is_ratio_fraction_high:1.3395668247540016e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.697834032820538e-05 - rollout_corr/rollout_is_max:2.0824544429779053 - rollout_corr/rollout_is_min:0.3074188232421875 - rollout_corr/rollout_is_std:0.0433402918279171 - rollout_corr/rollout_is_eff_sample_size:0.9981245471106103 - rollout_corr/rollout_is_seq_mean:0.9999098181724548 - rollout_corr/rollout_is_seq_std:0.0025561333168298006 - rollout_corr/rollout_is_seq_max:1.0072154998779297 - rollout_corr/rollout_is_seq_min:0.9910159111022949 - rollout_corr/rollout_is_seq_max_deviation:0.008984088897705078 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3661411092616618) - rollout_corr/training_log_ppl:np.float64(0.30844287818763405) - rollout_corr/kl:np.float64(0.001555773359804391) - rollout_corr/k3_kl:np.float64(0.0014595502659631165) - rollout_corr/rollout_ppl:np.float64(1.3639800301752985) - rollout_corr/rollout_log_ppl:np.float64(0.30688710472895764) - rollout_corr/log_ppl_diff:np.float64(0.0015557734586764127) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026391229184810072) - rollout_corr/log_ppl_diff_max:np.float64(0.011147260665893555) - rollout_corr/log_ppl_diff_min:np.float64(-0.006583303213119507) - rollout_corr/ppl_ratio:np.float64(1.0015615709125996) - rollout_corr/chi2_token:np.float64(0.0026983728166669607) - rollout_corr/chi2_seq:np.float64(0.5271497289650142) - actor/pg_loss:np.float64(0.0038334736600518227) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013588511892521637) - actor/ppo_kl:np.float64(0.00048035153239256445) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(120698.68936810177) - self_distillation/token_reweight_w_std:np.float64(1773706.5877801427) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000403418438509) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07702912960667163) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.46645358949899673) - perf/mfu/actor:np.float64(0.03541705134193752) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.77776336669922) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:0 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00016074800805654377 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00016074800805654377 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:291.60546875 - response_length/max:660.0 - response_length/min:133.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:291.60546875 - response_length_non_aborted/max:660.0 - response_length_non_aborted/min:133.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.9375 - prompt_length/max:813.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.972890019416809e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.429830538108945) - timing_s/agent_loop/generate_sequences/max:np.float64(5.387127343565226) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1587736944129574) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.387127343565226) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:197 - timing_s/agent_loop/slowest/response_length:660 - timing_s/gen:11.481718624010682 - timing_s/reward:0.056908562779426575 - timing_s/old_log_prob:2.2525762263685465 - timing_s/adv:0.5500077605247498 - timing_s/update_actor:15.172130836173892 - timing_s/step:29.60062885284424 - timing_s/stop_profile:0.00011467933654785156 - timing_per_token_ms/adv:0.0036136459894006673 - timing_per_token_ms/gen:0.15380528893130277 - timing_per_token_ms/update_actor:0.09968352027341046 - perf/total_num_tokens:152203 - perf/time_per_step:29.60062885284424 - perf/throughput:642.7355004713661 (TaskRunner pid=2026458) Training Progress: 43%|████▎ | 43/100 [32:15<40:30, 42.64s/it] (TaskRunner pid=2026458) step:44 - global_seqlen/min:16682 - global_seqlen/max:21465 - global_seqlen/minmax_diff:4783 - global_seqlen/balanced_min:19287 - global_seqlen/balanced_max:19289 - global_seqlen/mean:19287.75 - actor/entropy:0.2704760432243347 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26803696155548096 - training/rollout_probs_diff_mean:0.005588939879089594 - training/rollout_probs_diff_std:0.013731644488871098 - training/rollout_actor_probs_pearson_corr:0.9983340501785278 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:1.0000927448272705 - rollout_corr/rollout_is_ratio_fraction_high:2.68305120698642e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.049153257161379e-05 - rollout_corr/rollout_is_max:2.0968687534332275 - rollout_corr/rollout_is_min:0.22089464962482452 - rollout_corr/rollout_is_std:0.04294377565383911 - rollout_corr/rollout_is_eff_sample_size:0.9981595832973885 - rollout_corr/rollout_is_seq_mean:1.0000972747802734 - rollout_corr/rollout_is_seq_std:0.00284771085716784 - rollout_corr/rollout_is_seq_max:1.0088731050491333 - rollout_corr/rollout_is_seq_min:0.9906178116798401 - rollout_corr/rollout_is_seq_max_deviation:0.009382188320159912 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3369123949669302) - rollout_corr/training_log_ppl:np.float64(0.2868449309025891) - rollout_corr/kl:np.float64(0.0009355866875218055) - rollout_corr/k3_kl:np.float64(0.0010745994904937106) - rollout_corr/rollout_ppl:np.float64(1.335645436309278) - rollout_corr/rollout_log_ppl:np.float64(0.28590934313251637) - rollout_corr/log_ppl_diff:np.float64(0.0009355877700727433) - rollout_corr/log_ppl_abs_diff:np.float64(0.002383651997661218) - rollout_corr/log_ppl_diff_max:np.float64(0.011576682329177856) - rollout_corr/log_ppl_diff_min:np.float64(-0.006330162286758423) - rollout_corr/ppl_ratio:np.float64(1.00094054476358) - rollout_corr/chi2_token:np.float64(0.00241070962511003) - rollout_corr/chi2_seq:np.float64(0.8048715838231146) - actor/pg_loss:np.float64(0.0025291290367022157) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006139230463304557) - actor/ppo_kl:np.float64(5.7461318307527876e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(186553.62624621042) - self_distillation/token_reweight_w_std:np.float64(2401896.473150722) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0013129995204508) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05587577508413233) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.40642886608839035) - perf/mfu/actor:np.float64(0.03559055533852201) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.5995101928711) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:0 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0029178180266171694 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0029178180266171694 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:291.1796875 - response_length/max:674.0 - response_length/min:128.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:291.1796875 - response_length_non_aborted/max:674.0 - response_length_non_aborted/min:128.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:311.5625 - prompt_length/max:701.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001512337476015091 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5085251331329346) - timing_s/agent_loop/generate_sequences/max:np.float64(5.416358392685652) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1938710649119457) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.416358392685652) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:634 - timing_s/gen:11.832684468477964 - timing_s/reward:0.057033905759453773 - timing_s/old_log_prob:2.2766003496944904 - timing_s/adv:0.5443154312670231 - timing_s/update_actor:15.36143240518868 - timing_s/step:30.159771636128426 - timing_s/stop_profile:0.0001326482743024826 - timing_per_token_ms/adv:0.003527598030271954 - timing_per_token_ms/gen:0.1587384892876226 - timing_per_token_ms/update_actor:0.0995543311505274 - perf/total_num_tokens:154302 - perf/time_per_step:30.159771636128426 - perf/throughput:639.5190995708728 (TaskRunner pid=2026458) Training Progress: 44%|████▍ | 44/100 [32:46<36:19, 38.91s/it] (TaskRunner pid=2026458) step:45 - global_seqlen/min:15729 - global_seqlen/max:24872 - global_seqlen/minmax_diff:9143 - global_seqlen/balanced_min:18453 - global_seqlen/balanced_max:18459 - global_seqlen/mean:18456.25 - actor/entropy:0.28802311420440674 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29810237884521484 - training/rollout_probs_diff_mean:0.005760597065091133 - training/rollout_probs_diff_std:0.013836598955094814 - training/rollout_actor_probs_pearson_corr:0.9983675479888916 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.6484375 - self_distillation/correct_sample_fraction:0.53515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6484375 - rollout_corr/rollout_is_mean:1.000093698501587 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.471605279832147e-05 - rollout_corr/rollout_is_max:1.7223904132843018 - rollout_corr/rollout_is_min:0.3712316155433655 - rollout_corr/rollout_is_std:0.04281449690461159 - rollout_corr/rollout_is_eff_sample_size:0.9981707478872216 - rollout_corr/rollout_is_seq_mean:1.000179648399353 - rollout_corr/rollout_is_seq_std:0.002977126045152545 - rollout_corr/rollout_is_seq_max:1.008954644203186 - rollout_corr/rollout_is_seq_min:0.9924071431159973 - rollout_corr/rollout_is_seq_max_deviation:0.008954644203186035 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3517238958738744) - rollout_corr/training_log_ppl:np.float64(0.2982309421640821) - rollout_corr/kl:np.float64(0.0008551144521504739) - rollout_corr/k3_kl:np.float64(0.0011088182236562716) - rollout_corr/rollout_ppl:np.float64(1.350550597999245) - rollout_corr/rollout_log_ppl:np.float64(0.29737582663074136) - rollout_corr/log_ppl_diff:np.float64(0.0008551155333407223) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025770837091840804) - rollout_corr/log_ppl_diff_max:np.float64(0.009607642889022827) - rollout_corr/log_ppl_diff_min:np.float64(-0.007143139839172363) - rollout_corr/ppl_ratio:np.float64(1.000860691536218) - rollout_corr/chi2_token:np.float64(0.002733499277383089) - rollout_corr/chi2_seq:np.float64(0.8130777420010418) - actor/pg_loss:np.float64(0.002860508277080953) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007193945029939641) - actor/ppo_kl:np.float64(8.01470097968604e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53515625) - self_distillation/token_reweight_w_mean:np.float64(108369.78586033895) - self_distillation/token_reweight_w_std:np.float64(1441947.6739233742) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9984894776716828) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07410955597879365) - self_distillation/empty_target_batch:np.float64(0.3515625) - actor/grad_norm:np.float64(0.417113721370697) - perf/mfu/actor:np.float64(0.03440527561058198) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.48589324951172) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:0 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00041734983096830547 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00041734983096830547 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:262.0703125 - response_length/max:648.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:262.0703125 - response_length_non_aborted/max:648.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.6875 - prompt_length/max:845.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.201839566230774e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3249220605939627) - timing_s/agent_loop/generate_sequences/max:np.float64(5.273081086575985) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.846716286490846) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.273081086575985) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:418 - timing_s/agent_loop/slowest/response_length:648 - timing_s/gen:11.049067402258515 - timing_s/reward:0.0605863519012928 - timing_s/old_log_prob:2.241894433274865 - timing_s/adv:0.5028043333441019 - timing_s/update_actor:15.405420746654272 - timing_s/step:29.348894773051143 - timing_s/stop_profile:0.00011529400944709778 - timing_per_token_ms/adv:0.0034053798397839616 - timing_per_token_ms/gen:0.16469022808553457 - timing_per_token_ms/update_actor:0.1043374246302355 - perf/total_num_tokens:147650 - perf/time_per_step:29.348894773051143 - perf/throughput:628.8567301330532 (TaskRunner pid=2026458) Training Progress: 45%|████▌ | 45/100 [33:15<33:03, 36.06s/it] (TaskRunner pid=2026458) step:46 - global_seqlen/min:16485 - global_seqlen/max:20444 - global_seqlen/minmax_diff:3959 - global_seqlen/balanced_min:17819 - global_seqlen/balanced_max:17823 - global_seqlen/mean:17821.125 - actor/entropy:0.29167261719703674 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5032017827033997 - training/rollout_probs_diff_mean:0.005424427799880505 - training/rollout_probs_diff_std:0.013135196641087532 - training/rollout_actor_probs_pearson_corr:0.9985377788543701 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.54296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:0.9998041391372681 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.7731173759093508e-05 - rollout_corr/rollout_is_max:1.7920777797698975 - rollout_corr/rollout_is_min:0.19150957465171814 - rollout_corr/rollout_is_std:0.03981738165020943 - rollout_corr/rollout_is_eff_sample_size:0.9984166697314456 - rollout_corr/rollout_is_seq_mean:0.9997422695159912 - rollout_corr/rollout_is_seq_std:0.0028411736711859703 - rollout_corr/rollout_is_seq_max:1.0074349641799927 - rollout_corr/rollout_is_seq_min:0.9918615221977234 - rollout_corr/rollout_is_seq_max_deviation:0.008138477802276611 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3577020564116538) - rollout_corr/training_log_ppl:np.float64(0.3013367626699619) - rollout_corr/kl:np.float64(0.0011281410026526828) - rollout_corr/k3_kl:np.float64(0.000916888057986398) - rollout_corr/rollout_ppl:np.float64(1.3561205281876028) - rollout_corr/rollout_log_ppl:np.float64(0.3002086222113576) - rollout_corr/log_ppl_diff:np.float64(0.0011281404586043209) - rollout_corr/log_ppl_abs_diff:np.float64(0.00256365115637891) - rollout_corr/log_ppl_diff_max:np.float64(0.008949041366577148) - rollout_corr/log_ppl_diff_min:np.float64(-0.009114667773246765) - rollout_corr/ppl_ratio:np.float64(1.0011336989700794) - rollout_corr/chi2_token:np.float64(0.0014092018827795982) - rollout_corr/chi2_seq:np.float64(0.8297181797679514) - actor/pg_loss:np.float64(0.005181095330044627) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006262393026190693) - actor/ppo_kl:np.float64(4.9855951789634645e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.54296875) - self_distillation/token_reweight_w_mean:np.float64(97563.27732459153) - self_distillation/token_reweight_w_std:np.float64(1470502.0369112268) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995444938540459) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10749436303740367) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.4946809858083725) - perf/mfu/actor:np.float64(0.03312182529882031) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.53104400634766) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:0 - critic/score/mean:0.54296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.54296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002026108792051673 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002026108792051673 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:281.72265625 - response_length/max:593.0 - response_length/min:109.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:281.72265625 - response_length_non_aborted/max:593.0 - response_length_non_aborted/min:109.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:275.1875 - prompt_length/max:595.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.71662050485611e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3297640141099691) - timing_s/agent_loop/generate_sequences/max:np.float64(5.09482192248106) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.099804258738004) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.09482192248106) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:304 - timing_s/agent_loop/slowest/response_length:593 - timing_s/gen:11.312629146501422 - timing_s/reward:0.05594059266149998 - timing_s/old_log_prob:2.290945826098323 - timing_s/adv:0.4917367957532406 - timing_s/update_actor:15.34849851578474 - timing_s/step:29.586808905005455 - timing_s/stop_profile:0.00011978484690189362 - timing_per_token_ms/adv:0.0034491144340862362 - timing_per_token_ms/gen:0.15685624362531608 - timing_per_token_ms/update_actor:0.10765663303933352 - perf/total_num_tokens:142569 - perf/time_per_step:29.586808905005455 - perf/throughput:602.3334607398315 (TaskRunner pid=2026458) Training Progress: 46%|████▌ | 46/100 [33:45<30:43, 34.13s/it] (TaskRunner pid=2026458) step:47 - global_seqlen/min:14094 - global_seqlen/max:21506 - global_seqlen/minmax_diff:7412 - global_seqlen/balanced_min:18641 - global_seqlen/balanced_max:18644 - global_seqlen/mean:18642.625 - actor/entropy:0.27096572518348694 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7320154905319214 - training/rollout_probs_diff_mean:0.005300942808389664 - training/rollout_probs_diff_std:0.015477053821086884 - training/rollout_actor_probs_pearson_corr:0.9979209303855896 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0002624988555908 - rollout_corr/rollout_is_ratio_fraction_high:1.350238289887784e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00017553097859490663 - rollout_corr/rollout_is_max:5.250345230102539 - rollout_corr/rollout_is_min:0.1398765593767166 - rollout_corr/rollout_is_std:0.041101276874542236 - rollout_corr/rollout_is_eff_sample_size:0.9983143657831011 - rollout_corr/rollout_is_seq_mean:1.0002628564834595 - rollout_corr/rollout_is_seq_std:0.0028906052466481924 - rollout_corr/rollout_is_seq_max:1.0110986232757568 - rollout_corr/rollout_is_seq_min:0.9913266897201538 - rollout_corr/rollout_is_seq_max_deviation:0.011098623275756836 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.33038782607764) - rollout_corr/training_log_ppl:np.float64(0.2808810631977394) - rollout_corr/kl:np.float64(0.0008357813613741882) - rollout_corr/k3_kl:np.float64(0.0010391420327664491) - rollout_corr/rollout_ppl:np.float64(1.329236305784434) - rollout_corr/rollout_log_ppl:np.float64(0.28004528186284006) - rollout_corr/log_ppl_diff:np.float64(0.0008357813348993659) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023171580396592617) - rollout_corr/log_ppl_diff_max:np.float64(0.011047214269638062) - rollout_corr/log_ppl_diff_min:np.float64(-0.006683439016342163) - rollout_corr/ppl_ratio:np.float64(1.0008402362000197) - rollout_corr/chi2_token:np.float64(0.002344296546652913) - rollout_corr/chi2_seq:np.float64(0.7856939071789384) - actor/pg_loss:np.float64(0.0035682206507772207) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00080771961438586) - actor/ppo_kl:np.float64(0.000137977912149978) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(180196.61803952185) - self_distillation/token_reweight_w_std:np.float64(1850028.0704001258) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998111962340772) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09769563036388718) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.4677669107913971) - perf/mfu/actor:np.float64(0.03410809941893227) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.31404495239258) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:0 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003762101521715522 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003762101521715522 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:289.30078125 - response_length/max:738.0 - response_length/min:131.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:289.30078125 - response_length_non_aborted/max:738.0 - response_length_non_aborted/min:131.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.28125 - prompt_length/max:950.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.999237954616547e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5701810363680124) - timing_s/agent_loop/generate_sequences/max:np.float64(5.741097932681441) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.086432929841976) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.741097932681441) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:208 - timing_s/agent_loop/slowest/response_length:738 - timing_s/gen:12.44257121346891 - timing_s/reward:0.11020655557513237 - timing_s/old_log_prob:3.6139515712857246 - timing_s/adv:0.5418413802981377 - timing_s/update_actor:15.522931521758437 - timing_s/step:32.423666927963495 - timing_s/stop_profile:0.00011558458209037781 - timing_per_token_ms/adv:0.0036330813143142238 - timing_per_token_ms/gen:0.16800436415210313 - timing_per_token_ms/update_actor:0.10408225452262247 - perf/total_num_tokens:149141 - perf/time_per_step:32.423666927963495 - perf/throughput:574.9696677250851 (TaskRunner pid=2026458) Training Progress: 47%|████▋ | 47/100 [34:17<29:42, 33.63s/it] (TaskRunner pid=2026458) step:48 - global_seqlen/min:14172 - global_seqlen/max:18922 - global_seqlen/minmax_diff:4750 - global_seqlen/balanced_min:17273 - global_seqlen/balanced_max:17276 - global_seqlen/mean:17275.125 - actor/entropy:0.28335925936698914 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2923179566860199 - training/rollout_probs_diff_mean:0.005517815239727497 - training/rollout_probs_diff_std:0.0134055744856596 - training/rollout_actor_probs_pearson_corr:0.9984236359596252 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9999392032623291 - rollout_corr/rollout_is_ratio_fraction_high:1.4249682863010094e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.8499365726020187e-05 - rollout_corr/rollout_is_max:2.0768961906433105 - rollout_corr/rollout_is_min:0.30299994349479675 - rollout_corr/rollout_is_std:0.04144066944718361 - rollout_corr/rollout_is_eff_sample_size:0.9982854963367315 - rollout_corr/rollout_is_seq_mean:0.9998867511749268 - rollout_corr/rollout_is_seq_std:0.002759289462119341 - rollout_corr/rollout_is_seq_max:1.0119271278381348 - rollout_corr/rollout_is_seq_min:0.9927634000778198 - rollout_corr/rollout_is_seq_max_deviation:0.011927127838134766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3371913330629468) - rollout_corr/training_log_ppl:np.float64(0.28653379750903696) - rollout_corr/kl:np.float64(0.0009971100477166317) - rollout_corr/k3_kl:np.float64(0.0009379864269476457) - rollout_corr/rollout_ppl:np.float64(1.3358645238913596) - rollout_corr/rollout_log_ppl:np.float64(0.28553668738459237) - rollout_corr/log_ppl_diff:np.float64(0.000997110124444589) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023510434839408845) - rollout_corr/log_ppl_diff_max:np.float64(0.009961530566215515) - rollout_corr/log_ppl_diff_min:np.float64(-0.00725024938583374) - rollout_corr/ppl_ratio:np.float64(1.0010016579180956) - rollout_corr/chi2_token:np.float64(0.0017369601409882307) - rollout_corr/chi2_seq:np.float64(0.5906375367194414) - actor/pg_loss:np.float64(0.0036190334940329194) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005808295049973822) - actor/ppo_kl:np.float64(-3.3437734998642554e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(104527.51239224058) - self_distillation/token_reweight_w_std:np.float64(1541202.450584725) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000157124362886) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09064709232188761) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.42223525047302246) - perf/mfu/actor:np.float64(0.031950463769025786) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.0163230895996) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019717998802661896 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0019717998802661896 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:274.12890625 - response_length/max:573.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:274.12890625 - response_length_non_aborted/max:573.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.71875 - prompt_length/max:563.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012047216296195984 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2815109174698591) - timing_s/agent_loop/generate_sequences/max:np.float64(4.910025550052524) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0799834246499813) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.910025550052524) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:200 - timing_s/agent_loop/slowest/response_length:573 - timing_s/gen:10.97082038037479 - timing_s/reward:0.05485686659812927 - timing_s/old_log_prob:2.286929327994585 - timing_s/adv:0.5000464376062155 - timing_s/update_actor:15.382348308339715 - timing_s/step:29.284351360052824 - timing_s/stop_profile:0.0001140497624874115 - timing_per_token_ms/adv:0.0036182548433529096 - timing_per_token_ms/gen:0.15633071206199736 - timing_per_token_ms/update_actor:0.11130417513867277 - perf/total_num_tokens:138201 - perf/time_per_step:29.284351360052824 - perf/throughput:589.9097708397677 (TaskRunner pid=2026458) Training Progress: 48%|████▊ | 48/100 [34:46<28:01, 32.34s/it] (TaskRunner pid=2026458) step:49 - global_seqlen/min:17224 - global_seqlen/max:19716 - global_seqlen/minmax_diff:2492 - global_seqlen/balanced_min:18546 - global_seqlen/balanced_max:18550 - global_seqlen/mean:18548.75 - actor/entropy:0.27977773547172546 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3583575189113617 - training/rollout_probs_diff_mean:0.005447280127555132 - training/rollout_probs_diff_std:0.013272756710648537 - training/rollout_actor_probs_pearson_corr:0.9984596967697144 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:0.9998903274536133 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.59693042607978e-05 - rollout_corr/rollout_is_max:1.740144968032837 - rollout_corr/rollout_is_min:0.2820626199245453 - rollout_corr/rollout_is_std:0.04080290347337723 - rollout_corr/rollout_is_eff_sample_size:0.9983375338303951 - rollout_corr/rollout_is_seq_mean:0.9998434782028198 - rollout_corr/rollout_is_seq_std:0.0024030529893934727 - rollout_corr/rollout_is_seq_max:1.0073314905166626 - rollout_corr/rollout_is_seq_min:0.9888448715209961 - rollout_corr/rollout_is_seq_max_deviation:0.011155128479003906 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3396195233799517) - rollout_corr/training_log_ppl:np.float64(0.28772373546962626) - rollout_corr/kl:np.float64(0.0007274425354566461) - rollout_corr/k3_kl:np.float64(0.0009657581270516857) - rollout_corr/rollout_ppl:np.float64(1.338642387650907) - rollout_corr/rollout_log_ppl:np.float64(0.2869962909608148) - rollout_corr/log_ppl_diff:np.float64(0.0007274445088114589) - rollout_corr/log_ppl_abs_diff:np.float64(0.002191261184634641) - rollout_corr/log_ppl_diff_max:np.float64(0.011016637086868286) - rollout_corr/log_ppl_diff_min:np.float64(-0.009674102067947388) - rollout_corr/ppl_ratio:np.float64(1.0007315212860703) - rollout_corr/chi2_token:np.float64(0.002396710915490985) - rollout_corr/chi2_seq:np.float64(1.0418477742932737) - actor/pg_loss:np.float64(0.003264832077547908) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005003044952900382) - actor/ppo_kl:np.float64(-0.0002956932246789279) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(133482.32340499433) - self_distillation/token_reweight_w_std:np.float64(1837865.312099801) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993252530694008) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07909147051395848) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.44828861951828003) - perf/mfu/actor:np.float64(0.03315053423820138) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(151.27535438537598) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:0 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005617484916001558 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005617484916001558 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:300.8359375 - response_length/max:722.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:300.8359375 - response_length_non_aborted/max:722.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.8125 - prompt_length/max:418.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001618172973394394 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2382317576557398) - timing_s/agent_loop/generate_sequences/max:np.float64(5.747685985639691) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2742665582336485) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.747685985639691) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:358 - timing_s/agent_loop/slowest/response_length:722 - timing_s/gen:11.949256183579564 - timing_s/reward:0.0566436555236578 - timing_s/old_log_prob:2.331390691921115 - timing_s/adv:0.5100956819951534 - timing_s/update_actor:16.06329415179789 - timing_s/step:30.9992211535573 - timing_s/stop_profile:0.00011705979704856873 - timing_per_token_ms/adv:0.003437534079083182 - timing_per_token_ms/gen:0.15515693488949495 - timing_per_token_ms/update_actor:0.1082505165563575 - perf/total_num_tokens:148390 - perf/time_per_step:30.9992211535573 - perf/throughput:598.3618074827485 (TaskRunner pid=2026458) Training Progress: 49%|████▉ | 49/100 [35:17<27:09, 31.95s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) To determine the correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to identify which of the given options matches the structure of the reactant that would form this product through a chemical reaction. (TaskRunner pid=2026458) (TaskRunner pid=2026458) The product contains a structure with a substituted benzene ring (Cc2ccc(Cl)cc2) and a trifluoromethyl group (C(F)(F)F). The correct reactant should include these key functional groups and match the overall structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A contains a structure with a substituted benzene ring (c2ccccc2) and a nitro group (N), which is consistent with the product's structure. It also includes a chlorine atom and a trifluoromethyl group, making it a strong candidate for the correct reactant. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B includes a chlorine atom and a substituted benzene ring, but it lacks the trifluoromethyl group and does not match the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C includes a substituted benzene ring and a trifluoromethyl group, but it has a different functional group (an oxygen atom) that is not present in the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D includes a substituted benzene ring and a trifluoromethyl group, but it has a different functional group (a double bond) that is not present in the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the correct answer is Option A, as it contains the necessary functional groups and matches the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40/actor (TaskRunner pid=2026458) step:50 - global_seqlen/min:15714 - global_seqlen/max:24076 - global_seqlen/minmax_diff:8362 - global_seqlen/balanced_min:18055 - global_seqlen/balanced_max:18058 - global_seqlen/mean:18056.5 - actor/entropy:0.24377241730690002 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.36303967237472534 - training/rollout_probs_diff_mean:0.005148114170879126 - training/rollout_probs_diff_std:0.013693224638700485 - training/rollout_actor_probs_pearson_corr:0.9982551336288452 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.61328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:1.0000401735305786 - rollout_corr/rollout_is_ratio_fraction_high:2.7458949261927046e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.864737224532291e-05 - rollout_corr/rollout_is_max:2.6678342819213867 - rollout_corr/rollout_is_min:0.17930372059345245 - rollout_corr/rollout_is_std:0.04035785421729088 - rollout_corr/rollout_is_eff_sample_size:0.9983741296547397 - rollout_corr/rollout_is_seq_mean:1.0000826120376587 - rollout_corr/rollout_is_seq_std:0.002955564297735691 - rollout_corr/rollout_is_seq_max:1.0125819444656372 - rollout_corr/rollout_is_seq_min:0.9909312725067139 - rollout_corr/rollout_is_seq_max_deviation:0.012581944465637207 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3021241263486445) - rollout_corr/training_log_ppl:np.float64(0.26025468575244304) - rollout_corr/kl:np.float64(0.0007726539755310569) - rollout_corr/k3_kl:np.float64(0.0010959870580791176) - rollout_corr/rollout_ppl:np.float64(1.3010970023460686) - rollout_corr/rollout_log_ppl:np.float64(0.2594820303056622) - rollout_corr/log_ppl_diff:np.float64(0.0007726554467808455) - rollout_corr/log_ppl_abs_diff:np.float64(0.00248320066020824) - rollout_corr/log_ppl_diff_max:np.float64(0.010783776640892029) - rollout_corr/log_ppl_diff_min:np.float64(-0.014668971300125122) - rollout_corr/ppl_ratio:np.float64(1.0007780094165355) - rollout_corr/chi2_token:np.float64(0.002904584165662527) - rollout_corr/chi2_seq:np.float64(1.7037486750632524) - actor/pg_loss:np.float64(0.00390949088614434) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012861561140198319) - actor/ppo_kl:np.float64(-2.874969411337247e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.61328125) - self_distillation/token_reweight_w_mean:np.float64(303182.6307455059) - self_distillation/token_reweight_w_std:np.float64(3853282.430746559) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001452625496313) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08799264291883446) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.9146713092923164) - perf/mfu/actor:np.float64(0.03308314844340515) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.47300720214844) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6586309523809524) - val-aux/sciknoweval/reward/std@16:np.float64(0.1466147179951147) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7180000000000002) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.10968372523717904) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5997904761904762) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1360982101557332) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6585333333333333) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.146035780800308) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7595095238095239) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.07337309476992995) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5448142857142857) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11407729981266174) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.669457142857143) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.11545652344653062) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7867285714285714) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.04348485587197246) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4990380952380952) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08739882833894748) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.674152380952381) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.08550843930681744) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8024761904761903) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.02280799651519509) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4621714285714286) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05941417072300563) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6757285714285716) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06458122232163772) - val-aux/sciknoweval/score/mean@16:np.float64(0.6586309523809524) - val-aux/sciknoweval/score/std@16:np.float64(0.1466147179951147) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7180000000000002) - val-aux/sciknoweval/score/best@2/std:np.float64(0.10968372523717904) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5997904761904762) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1360982101557332) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6585333333333333) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.146035780800308) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7595095238095239) - val-aux/sciknoweval/score/best@4/std:np.float64(0.07337309476992995) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5448142857142857) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11407729981266174) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.669457142857143) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.11545652344653062) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7867285714285714) - val-aux/sciknoweval/score/best@8/std:np.float64(0.04348485587197246) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4990380952380952) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08739882833894748) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.674152380952381) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.08550843930681744) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8024761904761903) - val-aux/sciknoweval/score/best@16/std:np.float64(0.02280799651519509) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4621714285714286) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05941417072300563) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6757285714285716) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06458122232163772) - val-core/sciknoweval/acc/mean@16:np.float64(0.6586309523809524) - val-aux/sciknoweval/acc/std@16:np.float64(0.1466147179951147) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7180000000000002) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.10968372523717904) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5997904761904762) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1360982101557332) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6585333333333333) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.146035780800308) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7595095238095239) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.07337309476992995) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5448142857142857) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11407729981266174) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.669457142857143) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.11545652344653062) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7867285714285714) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.04348485587197246) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4990380952380952) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08739882833894748) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.674152380952381) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.08550843930681744) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8024761904761903) - val-core/sciknoweval/acc/best@16/std:np.float64(0.02280799651519509) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4621714285714286) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05941417072300563) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6757285714285716) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06458122232163772) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:0 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00038099291850812733 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00038099291850812733 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:284.515625 - response_length/max:754.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:284.515625 - response_length_non_aborted/max:754.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.75 - prompt_length/max:900.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001567751169204712 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4153082128614187) - timing_s/agent_loop/generate_sequences/max:np.float64(6.037689724937081) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1098132879196783) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.037689724937081) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:216 - timing_s/agent_loop/slowest/response_length:754 - timing_s/gen:12.157338604331017 - timing_s/reward:0.07574324123561382 - timing_s/old_log_prob:2.3888504561036825 - timing_s/adv:0.52120690792799 - timing_s/update_actor:15.507932955399156 - timing_s/step:30.738427244126797 - timing_s/testing:64.3304407633841 - timing_s/save_checkpoint:6.778321286663413 - timing_s/stop_profile:0.00014533661305904388 - timing_per_token_ms/adv:0.0036081667815467417 - timing_per_token_ms/gen:0.16691386957453755 - timing_per_token_ms/update_actor:0.10735699717137288 - perf/total_num_tokens:144452 - perf/time_per_step:30.738427244126797 - perf/throughput:587.4243290521658 (TaskRunner pid=2026458) Training Progress: 50%|█████ | 50/100 [36:59<44:06, 52.94s/it] (TaskRunner pid=2026458) step:51 - global_seqlen/min:17642 - global_seqlen/max:21320 - global_seqlen/minmax_diff:3678 - global_seqlen/balanced_min:19266 - global_seqlen/balanced_max:19274 - global_seqlen/mean:19272.125 - actor/entropy:0.27418410778045654 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3186328411102295 - training/rollout_probs_diff_mean:0.005429690238088369 - training/rollout_probs_diff_std:0.013169948011636734 - training/rollout_actor_probs_pearson_corr:0.9984568953514099 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.5390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9999269247055054 - rollout_corr/rollout_is_ratio_fraction_high:2.6965442884829827e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.6965442884829827e-05 - rollout_corr/rollout_is_max:2.9393632411956787 - rollout_corr/rollout_is_min:0.18107180297374725 - rollout_corr/rollout_is_std:0.04137301445007324 - rollout_corr/rollout_is_eff_sample_size:0.9982909612066878 - rollout_corr/rollout_is_seq_mean:0.9999714493751526 - rollout_corr/rollout_is_seq_std:0.0026941560208797455 - rollout_corr/rollout_is_seq_max:1.0119987726211548 - rollout_corr/rollout_is_seq_min:0.9906357526779175 - rollout_corr/rollout_is_seq_max_deviation:0.011998772621154785 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3326939661055803) - rollout_corr/training_log_ppl:np.float64(0.283328754856484) - rollout_corr/kl:np.float64(0.001354399174752885) - rollout_corr/k3_kl:np.float64(0.0011438059663078093) - rollout_corr/rollout_ppl:np.float64(1.3308947938494384) - rollout_corr/rollout_log_ppl:np.float64(0.2819743549771374) - rollout_corr/log_ppl_diff:np.float64(0.0013543998793466017) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025673647614894435) - rollout_corr/log_ppl_diff_max:np.float64(0.017022624611854553) - rollout_corr/log_ppl_diff_min:np.float64(-0.007628902792930603) - rollout_corr/ppl_ratio:np.float64(1.001360428519547) - rollout_corr/chi2_token:np.float64(0.0018329326994717121) - rollout_corr/chi2_seq:np.float64(1.2063880120404065) - actor/pg_loss:np.float64(0.006517422734759748) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015835570143281075) - actor/ppo_kl:np.float64(0.0004230728660630234) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5390625) - self_distillation/token_reweight_w_mean:np.float64(238186.0888677421) - self_distillation/token_reweight_w_std:np.float64(3422829.743828913) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0022471062839031) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1062716192973312) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.5796995311975479) - perf/mfu/actor:np.float64(0.035632098241036) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.68563079833984) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:0 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.010413717478513718 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.010413717478513718 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:289.72265625 - response_length/max:773.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:289.72265625 - response_length_non_aborted/max:773.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.53125 - prompt_length/max:608.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014895200729370117 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6900891289114952) - timing_s/agent_loop/generate_sequences/max:np.float64(5.7452072985470295) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1825576704068226) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.7452072985470295) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:321 - timing_s/agent_loop/slowest/response_length:773 - timing_s/gen:11.57777032442391 - timing_s/reward:0.05630043335258961 - timing_s/old_log_prob:2.3727036882191896 - timing_s/adv:0.5643414072692394 - timing_s/update_actor:15.28370295651257 - timing_s/step:29.95019925571978 - timing_s/stop_profile:0.00013649649918079376 - timing_per_token_ms/adv:0.003660347569801199 - timing_per_token_ms/gen:0.15609985741244872 - timing_per_token_ms/update_actor:0.09913088824216693 - perf/total_num_tokens:154177 - perf/time_per_step:29.95019925571978 - perf/throughput:643.472346726357 (TaskRunner pid=2026458) Training Progress: 51%|█████ | 51/100 [37:29<37:36, 46.06s/it] (TaskRunner pid=2026458) step:52 - global_seqlen/min:14188 - global_seqlen/max:21038 - global_seqlen/minmax_diff:6850 - global_seqlen/balanced_min:18334 - global_seqlen/balanced_max:18335 - global_seqlen/mean:18334.5 - actor/entropy:0.2573181688785553 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27730220556259155 - training/rollout_probs_diff_mean:0.005135765764862299 - training/rollout_probs_diff_std:0.012896839529275894 - training/rollout_actor_probs_pearson_corr:0.9984732270240784 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.59375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:0.9999938011169434 - rollout_corr/rollout_is_ratio_fraction_high:2.519780355214607e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.519780355214607e-05 - rollout_corr/rollout_is_max:2.1215898990631104 - rollout_corr/rollout_is_min:0.3974515199661255 - rollout_corr/rollout_is_std:0.040721017867326736 - rollout_corr/rollout_is_eff_sample_size:0.9983446626766718 - rollout_corr/rollout_is_seq_mean:0.9999977946281433 - rollout_corr/rollout_is_seq_std:0.0026824194937944412 - rollout_corr/rollout_is_seq_max:1.0067678689956665 - rollout_corr/rollout_is_seq_min:0.9906572103500366 - rollout_corr/rollout_is_seq_max_deviation:0.009342789649963379 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3137918766587973) - rollout_corr/training_log_ppl:np.float64(0.2694710589275928) - rollout_corr/kl:np.float64(0.0010667287329408914) - rollout_corr/k3_kl:np.float64(0.0010078331736167456) - rollout_corr/rollout_ppl:np.float64(1.3123429850675166) - rollout_corr/rollout_log_ppl:np.float64(0.26840433149482124) - rollout_corr/log_ppl_diff:np.float64(0.0010667274327715859) - rollout_corr/log_ppl_abs_diff:np.float64(0.002380369245656766) - rollout_corr/log_ppl_diff_max:np.float64(0.010203629732131958) - rollout_corr/log_ppl_diff_min:np.float64(-0.006238669157028198) - rollout_corr/ppl_ratio:np.float64(1.0010714137461036) - rollout_corr/chi2_token:np.float64(0.001948290504515171) - rollout_corr/chi2_seq:np.float64(0.5412429641000926) - actor/pg_loss:np.float64(0.003885703510604799) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005619293583549734) - actor/ppo_kl:np.float64(0.00019392038652732424) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59375) - self_distillation/token_reweight_w_mean:np.float64(186537.93477050052) - self_distillation/token_reweight_w_std:np.float64(2011255.6988170468) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005899087991565) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06515720163588412) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.4466311186552048) - perf/mfu/actor:np.float64(0.03402000847749656) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.35942840576172) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01099569071084261 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01099569071084261 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:310.046875 - response_length/max:700.0 - response_length/min:123.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:310.046875 - response_length_non_aborted/max:700.0 - response_length_non_aborted/min:123.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.90625 - prompt_length/max:453.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014230050146579742 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4891746453940868) - timing_s/agent_loop/generate_sequences/max:np.float64(5.560187539085746) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.207101171356044) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.560187539085746) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:211 - timing_s/agent_loop/slowest/response_length:700 - timing_s/gen:11.716578045859933 - timing_s/reward:0.05753312073647976 - timing_s/old_log_prob:2.249606389552355 - timing_s/adv:0.5425826162099838 - timing_s/update_actor:15.384762125089765 - timing_s/step:30.04255287349224 - timing_s/stop_profile:0.00013405084609985352 - timing_per_token_ms/adv:0.003699191525607351 - timing_per_token_ms/gen:0.14761601126165314 - timing_per_token_ms/update_actor:0.10488943061639099 - perf/total_num_tokens:146676 - perf/time_per_step:30.04255287349224 - perf/throughput:610.2843549017191 (TaskRunner pid=2026458) Training Progress: 52%|█████▏ | 52/100 [37:59<33:00, 41.27s/it] (TaskRunner pid=2026458) step:53 - global_seqlen/min:17379 - global_seqlen/max:20313 - global_seqlen/minmax_diff:2934 - global_seqlen/balanced_min:18481 - global_seqlen/balanced_max:18482 - global_seqlen/mean:18481.75 - actor/entropy:0.278646856546402 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18462908267974854 - training/rollout_probs_diff_mean:0.005563129670917988 - training/rollout_probs_diff_std:0.013384878635406494 - training/rollout_actor_probs_pearson_corr:0.9984221458435059 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6875 - self_distillation/correct_sample_fraction:0.57421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6875 - rollout_corr/rollout_is_mean:0.9999546408653259 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.5409334891010076e-05 - rollout_corr/rollout_is_max:1.8313579559326172 - rollout_corr/rollout_is_min:0.0737028419971466 - rollout_corr/rollout_is_std:0.04174019396305084 - rollout_corr/rollout_is_eff_sample_size:0.9982605487774039 - rollout_corr/rollout_is_seq_mean:0.9999428391456604 - rollout_corr/rollout_is_seq_std:0.0028358809649944305 - rollout_corr/rollout_is_seq_max:1.0075714588165283 - rollout_corr/rollout_is_seq_min:0.9910898804664612 - rollout_corr/rollout_is_seq_max_deviation:0.008910119533538818 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.338077249005437) - rollout_corr/training_log_ppl:np.float64(0.2873227519157808) - rollout_corr/kl:np.float64(0.0011784572657376202) - rollout_corr/k3_kl:np.float64(0.0013956419594478575) - rollout_corr/rollout_ppl:np.float64(1.3364087981171906) - rollout_corr/rollout_log_ppl:np.float64(0.2861442923604045) - rollout_corr/log_ppl_diff:np.float64(0.001178459555376321) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027842955314554274) - rollout_corr/log_ppl_diff_max:np.float64(0.0132656991481781) - rollout_corr/log_ppl_diff_min:np.float64(-0.010441526770591736) - rollout_corr/ppl_ratio:np.float64(1.0011851615272462) - rollout_corr/chi2_token:np.float64(0.003180387197062373) - rollout_corr/chi2_seq:np.float64(1.2029992926400155) - actor/pg_loss:np.float64(0.003771678893826902) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009931258632605022) - actor/ppo_kl:np.float64(0.00019185108615538127) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.57421875) - self_distillation/token_reweight_w_mean:np.float64(52868.786943790736) - self_distillation/token_reweight_w_std:np.float64(637934.6799204362) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9990235811565071) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07278861297527328) - self_distillation/empty_target_batch:np.float64(0.3125) - actor/grad_norm:np.float64(0.44415684044361115) - perf/mfu/actor:np.float64(0.0346355208538403) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.01051712036133) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:0 - critic/score/mean:0.57421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.57421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006060396321117878 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006060396321117878 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:281.9921875 - response_length/max:616.0 - response_length/min:128.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:281.9921875 - response_length_non_aborted/max:616.0 - response_length_non_aborted/min:128.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.5625 - prompt_length/max:532.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001376233994960785 - timing_s/agent_loop/generate_sequences/min:np.float64(1.366797188296914) - timing_s/agent_loop/generate_sequences/max:np.float64(5.209049364551902) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1154251298939926) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.209049364551902) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:227 - timing_s/agent_loop/slowest/response_length:616 - timing_s/gen:11.11843347735703 - timing_s/reward:0.05367938429117203 - timing_s/old_log_prob:2.252691153436899 - timing_s/adv:0.4951089806854725 - timing_s/update_actor:15.301375834271312 - timing_s/step:29.318861059844494 - timing_s/stop_profile:0.00011683069169521332 - timing_per_token_ms/adv:0.0033486343330953 - timing_per_token_ms/gen:0.15401625540042985 - timing_per_token_ms/update_actor:0.10348976581134979 - perf/total_num_tokens:147854 - perf/time_per_step:29.318861059844494 - perf/throughput:630.3706669326542 (TaskRunner pid=2026458) Training Progress: 53%|█████▎ | 53/100 [38:29<29:31, 37.70s/it] (TaskRunner pid=2026458) step:54 - global_seqlen/min:15111 - global_seqlen/max:18863 - global_seqlen/minmax_diff:3752 - global_seqlen/balanced_min:17356 - global_seqlen/balanced_max:17358 - global_seqlen/mean:17357.625 - actor/entropy:0.2270389199256897 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2413564920425415 - training/rollout_probs_diff_mean:0.005126654170453548 - training/rollout_probs_diff_std:0.01343945786356926 - training/rollout_actor_probs_pearson_corr:0.9981903433799744 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.50390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:1.0001782178878784 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.876001235563308e-05 - rollout_corr/rollout_is_max:1.8594493865966797 - rollout_corr/rollout_is_min:0.35453474521636963 - rollout_corr/rollout_is_std:0.04036819189786911 - rollout_corr/rollout_is_eff_sample_size:0.998373654367098 - rollout_corr/rollout_is_seq_mean:1.000128984451294 - rollout_corr/rollout_is_seq_std:0.002654197160154581 - rollout_corr/rollout_is_seq_max:1.0092252492904663 - rollout_corr/rollout_is_seq_min:0.9911713600158691 - rollout_corr/rollout_is_seq_max_deviation:0.009225249290466309 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2745554209686816) - rollout_corr/training_log_ppl:np.float64(0.23950671087368391) - rollout_corr/kl:np.float64(0.0011859710301269644) - rollout_corr/k3_kl:np.float64(0.0012516252674004136) - rollout_corr/rollout_ppl:np.float64(1.2729601715691388) - rollout_corr/rollout_log_ppl:np.float64(0.23832074020174332) - rollout_corr/log_ppl_diff:np.float64(0.001185970671940595) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027243185904808342) - rollout_corr/log_ppl_diff_max:np.float64(0.01128050684928894) - rollout_corr/log_ppl_diff_min:np.float64(-0.011040359735488892) - rollout_corr/ppl_ratio:np.float64(1.0011925832368433) - rollout_corr/chi2_token:np.float64(0.0025079476181417704) - rollout_corr/chi2_seq:np.float64(1.487264727940783) - actor/pg_loss:np.float64(0.0043368348851799965) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017240041906916304) - actor/ppo_kl:np.float64(0.0004469881809043841) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.50390625) - self_distillation/token_reweight_w_mean:np.float64(275413.10884484975) - self_distillation/token_reweight_w_std:np.float64(3763164.204352789) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0020550324115902) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08829297052579932) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.5274116396903992) - perf/mfu/actor:np.float64(0.03254539451835227) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.84357070922852) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:0 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0047957319766283035 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0047957319766283035 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:271.64453125 - response_length/max:650.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:271.64453125 - response_length_non_aborted/max:650.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.78125 - prompt_length/max:597.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.452683687210083e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3436731528490782) - timing_s/agent_loop/generate_sequences/max:np.float64(5.206967985257506) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.978192554124689) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.206967985257506) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:650 - timing_s/gen:10.970427636057138 - timing_s/reward:0.05443461053073406 - timing_s/old_log_prob:2.2816072031855583 - timing_s/adv:0.47508152201771736 - timing_s/update_actor:15.217902828007936 - timing_s/step:29.08791142515838 - timing_s/stop_profile:0.0001310836523771286 - timing_per_token_ms/adv:0.0034212739503367926 - timing_per_token_ms/gen:0.15775481566352423 - timing_per_token_ms/update_actor:0.10959090621562524 - perf/total_num_tokens:138861 - perf/time_per_step:29.08791142515838 - perf/throughput:596.7298492592096 (TaskRunner pid=2026458) Training Progress: 54%|█████▍ | 54/100 [38:58<26:55, 35.13s/it] (TaskRunner pid=2026458) step:55 - global_seqlen/min:15554 - global_seqlen/max:21932 - global_seqlen/minmax_diff:6378 - global_seqlen/balanced_min:18814 - global_seqlen/balanced_max:18817 - global_seqlen/mean:18816.625 - actor/entropy:0.22956356406211853 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9232618808746338 - training/rollout_probs_diff_mean:0.005393934901803732 - training/rollout_probs_diff_std:0.014924020506441593 - training/rollout_actor_probs_pearson_corr:0.9978342056274414 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.000108242034912 - rollout_corr/rollout_is_ratio_fraction_high:4.133313268539496e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.133313268539496e-05 - rollout_corr/rollout_is_max:2.1670196056365967 - rollout_corr/rollout_is_min:0.04885798692703247 - rollout_corr/rollout_is_std:0.042956262826919556 - rollout_corr/rollout_is_eff_sample_size:0.9981586331310626 - rollout_corr/rollout_is_seq_mean:1.0001986026763916 - rollout_corr/rollout_is_seq_std:0.002816676627844572 - rollout_corr/rollout_is_seq_max:1.0112348794937134 - rollout_corr/rollout_is_seq_min:0.9917329549789429 - rollout_corr/rollout_is_seq_max_deviation:0.011234879493713379 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2765053128823638) - rollout_corr/training_log_ppl:np.float64(0.24006979388650507) - rollout_corr/kl:np.float64(0.0010645344639215182) - rollout_corr/k3_kl:np.float64(0.0011318304883047858) - rollout_corr/rollout_ppl:np.float64(1.275088603142649) - rollout_corr/rollout_log_ppl:np.float64(0.23900525833596475) - rollout_corr/log_ppl_diff:np.float64(0.0010645355505403131) - rollout_corr/log_ppl_abs_diff:np.float64(0.002453350374707952) - rollout_corr/log_ppl_diff_max:np.float64(0.011057406663894653) - rollout_corr/log_ppl_diff_min:np.float64(-0.007540911436080933) - rollout_corr/ppl_ratio:np.float64(1.0010695066303015) - rollout_corr/chi2_token:np.float64(0.0023642820306122303) - rollout_corr/chi2_seq:np.float64(0.9924367545172572) - actor/pg_loss:np.float64(0.005413581035099924) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001406622541253455) - actor/ppo_kl:np.float64(0.0002698558086962066) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(295807.0609528662) - self_distillation/token_reweight_w_std:np.float64(3781227.560846627) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014416894409806) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09695331199327484) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.5869030505418777) - perf/mfu/actor:np.float64(0.03432748799407685) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.77687454223633) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:0 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0014518261887133121 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0014518261887133121 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:283.51953125 - response_length/max:687.0 - response_length/min:138.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:283.51953125 - response_length_non_aborted/max:687.0 - response_length_non_aborted/min:138.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:304.5 - prompt_length/max:625.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010005757212638855 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9696572478860617) - timing_s/agent_loop/generate_sequences/max:np.float64(5.428103374317288) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.78926639315614) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.428103374317288) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:329 - timing_s/agent_loop/slowest/response_length:687 - timing_s/gen:11.274114403873682 - timing_s/reward:0.0568949319422245 - timing_s/old_log_prob:2.307572791352868 - timing_s/adv:0.5059547368437052 - timing_s/update_actor:15.750575561076403 - timing_s/step:29.98292907886207 - timing_s/stop_profile:0.00012236833572387695 - timing_per_token_ms/adv:0.003361088511115205 - timing_per_token_ms/gen:0.1553314834994514 - timing_per_token_ms/update_actor:0.10463204454223594 - perf/total_num_tokens:150533 - perf/time_per_step:29.98292907886207 - perf/throughput:627.5779444532556 (TaskRunner pid=2026458) Training Progress: 55%|█████▌ | 55/100 [39:28<25:11, 33.60s/it] (TaskRunner pid=2026458) step:56 - global_seqlen/min:15183 - global_seqlen/max:19968 - global_seqlen/minmax_diff:4785 - global_seqlen/balanced_min:16851 - global_seqlen/balanced_max:16856 - global_seqlen/mean:16853.75 - actor/entropy:0.23840856552124023 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5032567977905273 - training/rollout_probs_diff_mean:0.0051898774690926075 - training/rollout_probs_diff_std:0.013416245579719543 - training/rollout_actor_probs_pearson_corr:0.9982264637947083 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0000948905944824 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.9873039238736965e-05 - rollout_corr/rollout_is_max:1.868481159210205 - rollout_corr/rollout_is_min:0.42863455414772034 - rollout_corr/rollout_is_std:0.03991232439875603 - rollout_corr/rollout_is_eff_sample_size:0.9984097775176272 - rollout_corr/rollout_is_seq_mean:1.0001156330108643 - rollout_corr/rollout_is_seq_std:0.002804977586492896 - rollout_corr/rollout_is_seq_max:1.008453607559204 - rollout_corr/rollout_is_seq_min:0.9917566776275635 - rollout_corr/rollout_is_seq_max_deviation:0.008453607559204102 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.288645532913506) - rollout_corr/training_log_ppl:np.float64(0.25009057341958396) - rollout_corr/kl:np.float64(0.001021610375019577) - rollout_corr/k3_kl:np.float64(0.0012247983307815957) - rollout_corr/rollout_ppl:np.float64(1.2872521225363016) - rollout_corr/rollout_log_ppl:np.float64(0.2490689625556115) - rollout_corr/log_ppl_diff:np.float64(0.0010216108639724553) - rollout_corr/log_ppl_abs_diff:np.float64(0.002677871671039611) - rollout_corr/log_ppl_diff_max:np.float64(0.011799484491348267) - rollout_corr/log_ppl_diff_min:np.float64(-0.013348877429962158) - rollout_corr/ppl_ratio:np.float64(1.001028069993481) - rollout_corr/chi2_token:np.float64(0.0027951118536293507) - rollout_corr/chi2_seq:np.float64(0.911775927990675) - actor/pg_loss:np.float64(0.0038659967249259353) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001200501193125092) - actor/ppo_kl:np.float64(0.0002928259359418206) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(173579.36585247447) - self_distillation/token_reweight_w_std:np.float64(2190372.348648581) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999903631862253) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08299089543288574) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.5205514430999756) - perf/mfu/actor:np.float64(0.03136809590233023) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.65859985351562) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00015309933223761618 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00015309933223761618 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:261.5234375 - response_length/max:725.0 - response_length/min:131.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:261.5234375 - response_length_non_aborted/max:725.0 - response_length_non_aborted/min:131.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.15625 - prompt_length/max:458.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001024492084980011 - timing_s/agent_loop/generate_sequences/min:np.float64(1.564612640067935) - timing_s/agent_loop/generate_sequences/max:np.float64(5.375423099845648) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8240807638067054) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.375423099845648) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:298 - timing_s/agent_loop/slowest/response_length:725 - timing_s/gen:11.348324349150062 - timing_s/reward:0.054198144003748894 - timing_s/old_log_prob:2.395893217995763 - timing_s/adv:0.4817402958869934 - timing_s/update_actor:15.269863871857524 - timing_s/step:29.636497838422656 - timing_s/stop_profile:0.00012066960334777832 - timing_per_token_ms/adv:0.003572945901409133 - timing_per_token_ms/gen:0.1695044712345043 - timing_per_token_ms/update_actor:0.11325271728738058 - perf/total_num_tokens:134830 - perf/time_per_step:29.636497838422656 - perf/throughput:568.6822407926255 (TaskRunner pid=2026458) Training Progress: 56%|█████▌ | 56/100 [39:58<23:46, 32.42s/it] (TaskRunner pid=2026458) step:57 - global_seqlen/min:15145 - global_seqlen/max:21004 - global_seqlen/minmax_diff:5859 - global_seqlen/balanced_min:17327 - global_seqlen/balanced_max:17330 - global_seqlen/mean:17329.125 - actor/entropy:0.2190825492143631 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4744122624397278 - training/rollout_probs_diff_mean:0.005394987761974335 - training/rollout_probs_diff_std:0.014672977849841118 - training/rollout_actor_probs_pearson_corr:0.9977869987487793 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:1.0002177953720093 - rollout_corr/rollout_is_ratio_fraction_high:9.200055501423776e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013800083252135664 - rollout_corr/rollout_is_max:3.8171095848083496 - rollout_corr/rollout_is_min:0.15978100895881653 - rollout_corr/rollout_is_std:0.04392360523343086 - rollout_corr/rollout_is_eff_sample_size:0.9980752630793054 - rollout_corr/rollout_is_seq_mean:1.0002235174179077 - rollout_corr/rollout_is_seq_std:0.0027234358713030815 - rollout_corr/rollout_is_seq_max:1.0101075172424316 - rollout_corr/rollout_is_seq_min:0.9909296035766602 - rollout_corr/rollout_is_seq_max_deviation:0.01010751724243164 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2523039323277771) - rollout_corr/training_log_ppl:np.float64(0.22247063313261606) - rollout_corr/kl:np.float64(0.0011533432969415003) - rollout_corr/k3_kl:np.float64(0.0011055539546305226) - rollout_corr/rollout_ppl:np.float64(1.2508012815378606) - rollout_corr/rollout_log_ppl:np.float64(0.22131728671956807) - rollout_corr/log_ppl_diff:np.float64(0.0011533464130479842) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025634898629505187) - rollout_corr/log_ppl_diff_max:np.float64(0.016222700476646423) - rollout_corr/log_ppl_diff_min:np.float64(-0.007622018456459045) - rollout_corr/ppl_ratio:np.float64(1.0011591671500355) - rollout_corr/chi2_token:np.float64(0.0020452477037906647) - rollout_corr/chi2_seq:np.float64(0.5157537555787712) - actor/pg_loss:np.float64(0.003759522340260446) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008975071282293356) - actor/ppo_kl:np.float64(0.00035579913428307464) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.578125) - self_distillation/token_reweight_w_mean:np.float64(236077.81382388272) - self_distillation/token_reweight_w_std:np.float64(2308889.815033672) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994992383290082) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0630278438620735) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.43273352086544037) - perf/mfu/actor:np.float64(0.031510354454649704) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.82762956619263) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:0 - critic/score/mean:0.578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0027676832396537066 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0027676832396537066 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:254.75390625 - response_length/max:747.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:254.75390625 - response_length_non_aborted/max:747.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.78125 - prompt_length/max:660.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014020688831806183 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4702318627387285) - timing_s/agent_loop/generate_sequences/max:np.float64(5.407115560024977) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.896707163978135) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.407115560024977) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:747 - timing_s/gen:11.385951856151223 - timing_s/reward:0.05382969416677952 - timing_s/old_log_prob:2.539514096453786 - timing_s/adv:0.49977708980441093 - timing_s/update_actor:15.644645780324936 - timing_s/step:30.209736336022615 - timing_s/stop_profile:0.00012044794857501984 - timing_per_token_ms/adv:0.0036050369666992053 - timing_per_token_ms/gen:0.17458564264150794 - timing_per_token_ms/update_actor:0.1128493632852563 - perf/total_num_tokens:138633 - perf/time_per_step:30.209736336022615 - perf/throughput:573.6271514338392 (TaskRunner pid=2026458) Training Progress: 57%|█████▋ | 57/100 [40:28<22:46, 31.77s/it] (TaskRunner pid=2026458) step:58 - global_seqlen/min:14734 - global_seqlen/max:19599 - global_seqlen/minmax_diff:4865 - global_seqlen/balanced_min:17102 - global_seqlen/balanced_max:17103 - global_seqlen/mean:17102.75 - actor/entropy:0.23608146607875824 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5141049027442932 - training/rollout_probs_diff_mean:0.005427653901278973 - training/rollout_probs_diff_std:0.013945525512099266 - training/rollout_actor_probs_pearson_corr:0.9981228709220886 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9999080896377563 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.5163001990003977e-05 - rollout_corr/rollout_is_max:1.724382996559143 - rollout_corr/rollout_is_min:0.16556662321090698 - rollout_corr/rollout_is_std:0.04180582985281944 - rollout_corr/rollout_is_eff_sample_size:0.9982550842404107 - rollout_corr/rollout_is_seq_mean:0.9999476671218872 - rollout_corr/rollout_is_seq_std:0.002901083789765835 - rollout_corr/rollout_is_seq_max:1.00826895236969 - rollout_corr/rollout_is_seq_min:0.9917892217636108 - rollout_corr/rollout_is_seq_max_deviation:0.008268952369689941 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2862667385488749) - rollout_corr/training_log_ppl:np.float64(0.24756408171379007) - rollout_corr/kl:np.float64(0.0008958796237372724) - rollout_corr/k3_kl:np.float64(0.0010313347952433105) - rollout_corr/rollout_ppl:np.float64(1.2851230534724891) - rollout_corr/rollout_log_ppl:np.float64(0.24666820294805802) - rollout_corr/log_ppl_diff:np.float64(0.0008958787657320499) - rollout_corr/log_ppl_abs_diff:np.float64(0.002455878769978881) - rollout_corr/log_ppl_diff_max:np.float64(0.014661163091659546) - rollout_corr/log_ppl_diff_min:np.float64(-0.006324887275695801) - rollout_corr/ppl_ratio:np.float64(1.00090107973665) - rollout_corr/chi2_token:np.float64(0.0023643041495233774) - rollout_corr/chi2_seq:np.float64(0.6059110292699188) - actor/pg_loss:np.float64(0.003710375865921378) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011193532754987245) - actor/ppo_kl:np.float64(-8.923532409355062e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(286496.5454768897) - self_distillation/token_reweight_w_std:np.float64(3628499.957205932) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005135682877153) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07967197702964768) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.5009380057454109) - perf/mfu/actor:np.float64(0.03152191694270064) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.63263320922852) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0016660349210724235 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0016660349210724235 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:257.6171875 - response_length/max:583.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:257.6171875 - response_length_non_aborted/max:583.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.84375 - prompt_length/max:598.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001420155167579651 - timing_s/agent_loop/generate_sequences/min:np.float64(1.489874416962266) - timing_s/agent_loop/generate_sequences/max:np.float64(4.795747706666589) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8081408347570687) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.795747706666589) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:583 - timing_s/gen:10.590774795040488 - timing_s/reward:0.05500875972211361 - timing_s/old_log_prob:2.199430391192436 - timing_s/adv:0.4823594633489847 - timing_s/update_actor:15.483584174886346 - timing_s/step:28.898630160838366 - timing_s/stop_profile:0.0001272875815629959 - timing_per_token_ms/adv:0.0035254525101883084 - timing_per_token_ms/gen:0.16058794230539028 - timing_per_token_ms/update_actor:0.11316589565191523 - perf/total_num_tokens:136822 - perf/time_per_step:28.898630160838366 - perf/throughput:591.818709219532 (TaskRunner pid=2026458) Training Progress: 58%|█████▊ | 58/100 [40:57<21:38, 30.92s/it] (TaskRunner pid=2026458) step:59 - global_seqlen/min:15244 - global_seqlen/max:19485 - global_seqlen/minmax_diff:4241 - global_seqlen/balanced_min:16769 - global_seqlen/balanced_max:16771 - global_seqlen/mean:16770.625 - actor/entropy:0.2315026819705963 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5046374797821045 - training/rollout_probs_diff_mean:0.0052477153949439526 - training/rollout_probs_diff_std:0.013823322020471096 - training/rollout_actor_probs_pearson_corr:0.9981176853179932 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65625 - self_distillation/correct_sample_fraction:0.58203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65625 - rollout_corr/rollout_is_mean:1.0000110864639282 - rollout_corr/rollout_is_ratio_fraction_high:1.5287481801351532e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.114992720540613e-05 - rollout_corr/rollout_is_max:2.5501413345336914 - rollout_corr/rollout_is_min:0.3053041398525238 - rollout_corr/rollout_is_std:0.04093707725405693 - rollout_corr/rollout_is_eff_sample_size:0.998326959562596 - rollout_corr/rollout_is_seq_mean:1.000074863433838 - rollout_corr/rollout_is_seq_std:0.0029152550268918276 - rollout_corr/rollout_is_seq_max:1.0087932348251343 - rollout_corr/rollout_is_seq_min:0.9920890927314758 - rollout_corr/rollout_is_seq_max_deviation:0.008793234825134277 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2814294979907572) - rollout_corr/training_log_ppl:np.float64(0.24554033184540458) - rollout_corr/kl:np.float64(0.0010128226648049576) - rollout_corr/k3_kl:np.float64(0.0010760613429283694) - rollout_corr/rollout_ppl:np.float64(1.280145457945764) - rollout_corr/rollout_log_ppl:np.float64(0.2445275079808198) - rollout_corr/log_ppl_diff:np.float64(0.0010128238645847887) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026435373874846846) - rollout_corr/log_ppl_diff_max:np.float64(0.01640312373638153) - rollout_corr/log_ppl_diff_min:np.float64(-0.01078614592552185) - rollout_corr/ppl_ratio:np.float64(1.001019265735522) - rollout_corr/chi2_token:np.float64(0.0022960123606026173) - rollout_corr/chi2_seq:np.float64(0.9392846249975264) - actor/pg_loss:np.float64(0.002857705345377326) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00044364196128299227) - actor/ppo_kl:np.float64(0.0001824798405234418) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58203125) - self_distillation/token_reweight_w_mean:np.float64(71359.55967221712) - self_distillation/token_reweight_w_std:np.float64(989988.4983619121) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995234010275453) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04141386490664445) - self_distillation/empty_target_batch:np.float64(0.34375) - actor/grad_norm:np.float64(0.3792598769068718) - perf/mfu/actor:np.float64(0.03147454843432276) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.65855026245117) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:0 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003848623251542449 - critic/advantages/max:0.625 - critic/advantages/min:-0.75 - critic/returns/mean:0.003848623251542449 - critic/returns/max:0.625 - critic/returns/min:-0.75 - response_length/mean:255.51953125 - response_length/max:621.0 - response_length/min:117.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:255.51953125 - response_length_non_aborted/max:621.0 - response_length_non_aborted/min:117.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.5625 - prompt_length/max:615.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.882924258708954e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.939049456268549) - timing_s/agent_loop/generate_sequences/max:np.float64(5.183893235400319) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.658139636150736) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.183893235400319) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:217 - timing_s/agent_loop/slowest/response_length:621 - timing_s/gen:10.904466819018126 - timing_s/reward:0.052795736119151115 - timing_s/old_log_prob:2.23296419903636 - timing_s/adv:0.47304221242666245 - timing_s/update_actor:15.1854842081666 - timing_s/step:28.936049316078424 - timing_s/stop_profile:5.451589822769165e-05 - timing_per_token_ms/adv:0.0035258242643510787 - timing_per_token_ms/gen:0.16670183020222473 - timing_per_token_ms/update_actor:0.11318513925514552 - perf/total_num_tokens:134165 - perf/time_per_step:28.936049316078424 - perf/throughput:579.575491346752 (TaskRunner pid=2026458) Training Progress: 59%|█████▉ | 59/100 [41:26<20:43, 30.33s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) To determine the correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to identify which of the given options contains the correct functional groups and structure that match the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) The product contains a nitrogen-containing group (n1) connected to a cyclohexane ring (Cc2ccc(Cl)cc2), with a Cl group and a trifluoromethyl group (C(F)(F)F). The correct reactant must include these key features. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A contains a ketone group (O=C1), an aromatic ring (c2ccccc2), and a nitrogen (N1) connected to a cyclohexane ring with a Cl group and a trifluoromethyl group. This matches the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B contains a ketone group (O=C1), an aromatic ring (c2ccccc2), and a nitrogen (N1), but it does not include the Cl group or the trifluoromethyl group in the correct position. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C contains a ketone group (O=C1), an aromatic ring (c2ccccc2), and a nitrogen (N1), but it includes a different functional group (Oc2ccc(Cl)c(C(F)(F)F)c2) that does not match the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D contains a ketone group (O=C1), an aromatic ring (c2ccccc2), and a nitrogen (N1), but it includes a different functional group (CC=C(Cc1ccccc1)C(F)(F)F) that does not match the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the correct answer is Option A. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50/actor (TaskRunner pid=2026458) step:60 - global_seqlen/min:14552 - global_seqlen/max:17730 - global_seqlen/minmax_diff:3178 - global_seqlen/balanced_min:16889 - global_seqlen/balanced_max:16891 - global_seqlen/mean:16890.25 - actor/entropy:0.258317232131958 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5851317644119263 - training/rollout_probs_diff_mean:0.005422248970717192 - training/rollout_probs_diff_std:0.013785075396299362 - training/rollout_actor_probs_pearson_corr:0.9982817769050598 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:1.0001024007797241 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4916022337274626e-05 - rollout_corr/rollout_is_max:1.976409912109375 - rollout_corr/rollout_is_min:0.3921801745891571 - rollout_corr/rollout_is_std:0.041325442492961884 - rollout_corr/rollout_is_eff_sample_size:0.9982955945128663 - rollout_corr/rollout_is_seq_mean:1.000079870223999 - rollout_corr/rollout_is_seq_std:0.002677295822650194 - rollout_corr/rollout_is_seq_max:1.0086641311645508 - rollout_corr/rollout_is_seq_min:0.9929971694946289 - rollout_corr/rollout_is_seq_max_deviation:0.008664131164550781 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3128179563209414) - rollout_corr/training_log_ppl:np.float64(0.2678291550837457) - rollout_corr/kl:np.float64(0.001152293861238718) - rollout_corr/k3_kl:np.float64(0.0010784027733734547) - rollout_corr/rollout_ppl:np.float64(1.3112971791997552) - rollout_corr/rollout_log_ppl:np.float64(0.2666768589988351) - rollout_corr/log_ppl_diff:np.float64(0.0011522960849106312) - rollout_corr/log_ppl_abs_diff:np.float64(0.002510662190616131) - rollout_corr/log_ppl_diff_max:np.float64(0.009669288992881775) - rollout_corr/log_ppl_diff_min:np.float64(-0.007052525877952576) - rollout_corr/ppl_ratio:np.float64(1.0011574435047805) - rollout_corr/chi2_token:np.float64(0.002051107119768858) - rollout_corr/chi2_seq:np.float64(0.5519429405685514) - actor/pg_loss:np.float64(0.003786680055782199) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008521524032403249) - actor/ppo_kl:np.float64(0.0003417033792416646) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(148988.88128135214) - self_distillation/token_reweight_w_std:np.float64(1866554.9818183868) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014604937750846) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08368598294327967) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.4603109806776047) - perf/mfu/actor:np.float64(0.03142972200463806) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.4005994796753) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6735119047619048) - val-aux/sciknoweval/reward/std@16:np.float64(0.14333415818798717) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7287000000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.10654141009401605) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6172761904761904) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.13505886263851702) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6726047619047619) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1435861836456534) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7666619047619049) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.0728423418058757) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5634142857142856) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11842617318668035) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6849761904761906) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.10836775791233498) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7947571428571428) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.05056443439799785) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5141142857142857) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09350537912002374) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6924571428571429) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07287959815596419) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8152857142857142) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03269928922744704) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.47582857142857143) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.0635348044707675) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6967476190476191) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.04617062732616158) - val-aux/sciknoweval/score/mean@16:np.float64(0.6735119047619048) - val-aux/sciknoweval/score/std@16:np.float64(0.14333415818798717) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7287000000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.10654141009401605) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6172761904761904) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.13505886263851702) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6726047619047619) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1435861836456534) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7666619047619049) - val-aux/sciknoweval/score/best@4/std:np.float64(0.0728423418058757) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5634142857142856) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11842617318668035) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6849761904761906) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.10836775791233498) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7947571428571428) - val-aux/sciknoweval/score/best@8/std:np.float64(0.05056443439799785) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5141142857142857) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09350537912002374) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6924571428571429) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07287959815596419) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8152857142857142) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03269928922744704) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.47582857142857143) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.0635348044707675) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6967476190476191) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.04617062732616158) - val-core/sciknoweval/acc/mean@16:np.float64(0.6735119047619048) - val-aux/sciknoweval/acc/std@16:np.float64(0.14333415818798717) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7287000000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.10654141009401605) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6172761904761904) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.13505886263851702) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6726047619047619) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1435861836456534) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7666619047619049) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.0728423418058757) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5634142857142856) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11842617318668035) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6849761904761906) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.10836775791233498) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7947571428571428) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.05056443439799785) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5141142857142857) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09350537912002374) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6924571428571429) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07287959815596419) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8152857142857142) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03269928922744704) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.47582857142857143) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.0635348044707675) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6967476190476191) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.04617062732616158) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:1 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.000639524485450238 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.000639524485450238 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:261.8828125 - response_length/max:611.0 - response_length/min:121.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:261.8828125 - response_length_non_aborted/max:611.0 - response_length_non_aborted/min:121.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.9375 - prompt_length/max:451.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00021154806017875671 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3727056793868542) - timing_s/agent_loop/generate_sequences/max:np.float64(4.906513184309006) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9096029985012137) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.906513184309006) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:563 - timing_s/gen:11.531464690342546 - timing_s/reward:0.05282927677035332 - timing_s/old_log_prob:2.459550989791751 - timing_s/adv:0.5421040039509535 - timing_s/update_actor:15.288631148636341 - timing_s/step:29.9776940792799 - timing_s/testing:64.2218036558479 - timing_s/save_checkpoint:6.659373983740807 - timing_s/stop_profile:5.859695374965668e-05 - timing_per_token_ms/adv:0.00401195959170937 - timing_per_token_ms/gen:0.17200359014263514 - timing_per_token_ms/update_actor:0.11314686837551502 - perf/total_num_tokens:135122 - perf/time_per_step:29.9776940792799 - perf/throughput:563.4272587922054 (TaskRunner pid=2026458) Training Progress: 60%|██████ | 60/100 [43:07<34:24, 51.61s/it] (TaskRunner pid=2026458) step:61 - global_seqlen/min:14736 - global_seqlen/max:19497 - global_seqlen/minmax_diff:4761 - global_seqlen/balanced_min:17532 - global_seqlen/balanced_max:17536 - global_seqlen/mean:17534.5 - actor/entropy:0.2423284649848938 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.49669769406318665 - training/rollout_probs_diff_mean:0.005022233817726374 - training/rollout_probs_diff_std:0.013324090279638767 - training/rollout_actor_probs_pearson_corr:0.9983177781105042 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.0002062320709229 - rollout_corr/rollout_is_ratio_fraction_high:5.464779314934276e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.0985847590491176e-05 - rollout_corr/rollout_is_max:7.0302934646606445 - rollout_corr/rollout_is_min:0.35777419805526733 - rollout_corr/rollout_is_std:0.03978368639945984 - rollout_corr/rollout_is_eff_sample_size:0.998420353539922 - rollout_corr/rollout_is_seq_mean:1.0001554489135742 - rollout_corr/rollout_is_seq_std:0.002474896376952529 - rollout_corr/rollout_is_seq_max:1.0129971504211426 - rollout_corr/rollout_is_seq_min:0.9919136762619019 - rollout_corr/rollout_is_seq_max_deviation:0.012997150421142578 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2963423300534487) - rollout_corr/training_log_ppl:np.float64(0.25556481117382646) - rollout_corr/kl:np.float64(0.0009956922927543133) - rollout_corr/k3_kl:np.float64(0.0010460353189500893) - rollout_corr/rollout_ppl:np.float64(1.2950277719646692) - rollout_corr/rollout_log_ppl:np.float64(0.25456911721266806) - rollout_corr/log_ppl_diff:np.float64(0.0009956939611583948) - rollout_corr/log_ppl_abs_diff:np.float64(0.002202652278356254) - rollout_corr/log_ppl_diff_max:np.float64(0.010044902563095093) - rollout_corr/log_ppl_diff_min:np.float64(-0.009525060653686523) - rollout_corr/ppl_ratio:np.float64(1.0009999251924455) - rollout_corr/chi2_token:np.float64(0.0022548860870301723) - rollout_corr/chi2_seq:np.float64(0.8442793434951454) - actor/pg_loss:np.float64(0.0042004447896033525) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007894714767644473) - actor/ppo_kl:np.float64(0.0002806229531095994) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(132276.08868701942) - self_distillation/token_reweight_w_std:np.float64(1859101.6540320972) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012411118950695) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08409801000379957) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.4940867945551872) - perf/mfu/actor:np.float64(0.03230972888329671) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(146.0987777709961) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:1 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0024950134102255106 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0024950134102255106 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:285.921875 - response_length/max:685.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:285.921875 - response_length_non_aborted/max:685.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.03125 - prompt_length/max:462.0 - prompt_length/min:177.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.605738937854767e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.467216296121478) - timing_s/agent_loop/generate_sequences/max:np.float64(5.52247816324234) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0488981239686836) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.52247816324234) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:202 - timing_s/agent_loop/slowest/response_length:652 - timing_s/gen:11.610483139753342 - timing_s/reward:0.05246041156351566 - timing_s/old_log_prob:2.416056863963604 - timing_s/adv:0.5235678162425756 - timing_s/update_actor:15.423095140606165 - timing_s/step:30.12737883068621 - timing_s/stop_profile:0.00011511333286762238 - timing_per_token_ms/adv:0.003732411932494337 - timing_per_token_ms/gen:0.15862182550622086 - timing_per_token_ms/update_actor:0.10994821024698569 - perf/total_num_tokens:140276 - perf/time_per_step:30.12737883068621 - perf/throughput:582.0121325038822 (TaskRunner pid=2026458) Training Progress: 61%|██████ | 61/100 [43:37<29:22, 45.18s/it] (TaskRunner pid=2026458) step:62 - global_seqlen/min:15907 - global_seqlen/max:20488 - global_seqlen/minmax_diff:4581 - global_seqlen/balanced_min:18196 - global_seqlen/balanced_max:18198 - global_seqlen/mean:18197.0 - actor/entropy:0.26120486855506897 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35323476791381836 - training/rollout_probs_diff_mean:0.005538288038223982 - training/rollout_probs_diff_std:0.014037296175956726 - training/rollout_actor_probs_pearson_corr:0.9981968998908997 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:0.9998036623001099 - rollout_corr/rollout_is_ratio_fraction_high:2.8998956622672267e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014499477401841432 - rollout_corr/rollout_is_max:2.314438819885254 - rollout_corr/rollout_is_min:0.32504311203956604 - rollout_corr/rollout_is_std:0.04294585436582565 - rollout_corr/rollout_is_eff_sample_size:0.9981582768191568 - rollout_corr/rollout_is_seq_mean:0.9998564720153809 - rollout_corr/rollout_is_seq_std:0.0027946694754064083 - rollout_corr/rollout_is_seq_max:1.0084697008132935 - rollout_corr/rollout_is_seq_min:0.9910843372344971 - rollout_corr/rollout_is_seq_max_deviation:0.00891566276550293 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.316816312726587) - rollout_corr/training_log_ppl:np.float64(0.27135412095231004) - rollout_corr/kl:np.float64(0.0013785573038145316) - rollout_corr/k3_kl:np.float64(0.0011328787384172756) - rollout_corr/rollout_ppl:np.float64(1.3149180496111512) - rollout_corr/rollout_log_ppl:np.float64(0.2699755621724762) - rollout_corr/log_ppl_diff:np.float64(0.0013785587798338383) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027030363271478564) - rollout_corr/log_ppl_diff_max:np.float64(0.010782003402709961) - rollout_corr/log_ppl_diff_min:np.float64(-0.008555859327316284) - rollout_corr/ppl_ratio:np.float64(1.0013843621127307) - rollout_corr/chi2_token:np.float64(0.0016965712420642376) - rollout_corr/chi2_seq:np.float64(0.6767998114228249) - actor/pg_loss:np.float64(0.003519455436617136) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008139853748616588) - actor/ppo_kl:np.float64(0.0002313825499626887) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(140065.4251762007) - self_distillation/token_reweight_w_std:np.float64(1931525.736894751) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999061315320432) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07281633446109481) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.4214628040790558) - perf/mfu/actor:np.float64(0.033835202551722315) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(146.0380401611328) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:1 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003894922323524952 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003894922323524952 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:269.40625 - response_length/max:578.0 - response_length/min:124.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:269.40625 - response_length_non_aborted/max:578.0 - response_length_non_aborted/min:124.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.25 - prompt_length/max:662.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001499168574810028 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3069061245769262) - timing_s/agent_loop/generate_sequences/max:np.float64(4.653931180015206) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8548159639394726) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.653931180015206) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:578 - timing_s/gen:10.786067547276616 - timing_s/reward:0.05438114143908024 - timing_s/old_log_prob:2.2739491928368807 - timing_s/adv:0.471931716427207 - timing_s/update_actor:15.285212887451053 - timing_s/step:28.96465684659779 - timing_s/stop_profile:0.00012806802988052368 - timing_per_token_ms/adv:0.003241823627707912 - timing_per_token_ms/gen:0.15639234931093574 - timing_per_token_ms/update_actor:0.10499816513333965 - perf/total_num_tokens:145576 - perf/time_per_step:28.96465684659779 - perf/throughput:628.2484234622456 (TaskRunner pid=2026458) Training Progress: 62%|██████▏ | 62/100 [44:06<25:32, 40.33s/it] (TaskRunner pid=2026458) step:63 - global_seqlen/min:15346 - global_seqlen/max:20074 - global_seqlen/minmax_diff:4728 - global_seqlen/balanced_min:17670 - global_seqlen/balanced_max:17672 - global_seqlen/mean:17671.0 - actor/entropy:0.2494240403175354 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44295963644981384 - training/rollout_probs_diff_mean:0.005514265038073063 - training/rollout_probs_diff_std:0.013980090618133545 - training/rollout_actor_probs_pearson_corr:0.9981726408004761 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.48828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:0.9999319911003113 - rollout_corr/rollout_is_ratio_fraction_high:1.4685150745208375e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.811090083327144e-05 - rollout_corr/rollout_is_max:2.1109066009521484 - rollout_corr/rollout_is_min:0.21203994750976562 - rollout_corr/rollout_is_std:0.04199644550681114 - rollout_corr/rollout_is_eff_sample_size:0.9982390473577023 - rollout_corr/rollout_is_seq_mean:0.9999494552612305 - rollout_corr/rollout_is_seq_std:0.003092808648943901 - rollout_corr/rollout_is_seq_max:1.0130763053894043 - rollout_corr/rollout_is_seq_min:0.9914883375167847 - rollout_corr/rollout_is_seq_max_deviation:0.013076305389404297 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.302847915329039) - rollout_corr/training_log_ppl:np.float64(0.26041782830725424) - rollout_corr/kl:np.float64(0.001262190604840896) - rollout_corr/k3_kl:np.float64(0.0011403532549252304) - rollout_corr/rollout_ppl:np.float64(1.3011768972501159) - rollout_corr/rollout_log_ppl:np.float64(0.2591556366533041) - rollout_corr/log_ppl_diff:np.float64(0.0012621916539501399) - rollout_corr/log_ppl_abs_diff:np.float64(0.002628726389957592) - rollout_corr/log_ppl_diff_max:np.float64(0.010536238551139832) - rollout_corr/log_ppl_diff_min:np.float64(-0.009842857718467712) - rollout_corr/ppl_ratio:np.float64(1.0012680946383625) - rollout_corr/chi2_token:np.float64(0.0019653229974210262) - rollout_corr/chi2_seq:np.float64(0.7706008821260184) - actor/pg_loss:np.float64(0.0034944345243275166) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000709521887529263) - actor/ppo_kl:np.float64(0.00027179806370725146) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48828125) - self_distillation/token_reweight_w_mean:np.float64(168796.3436012969) - self_distillation/token_reweight_w_std:np.float64(2124951.2078802614) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015958338044584) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0751116150058806) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.4171854108572006) - perf/mfu/actor:np.float64(0.03263190662040404) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.89733123779297) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:1 - critic/score/mean:0.48828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005943814758211374 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005943814758211374 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:266.0 - response_length/max:610.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:266.0 - response_length_non_aborted/max:610.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.21875 - prompt_length/max:532.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011231563985347748 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2275551408529282) - timing_s/agent_loop/generate_sequences/max:np.float64(4.99046435020864) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8069054595180205) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.99046435020864) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:231 - timing_s/agent_loop/slowest/response_length:607 - timing_s/gen:11.166345795616508 - timing_s/reward:0.05529453977942467 - timing_s/old_log_prob:2.2847698368132114 - timing_s/adv:0.49184093065559864 - timing_s/update_actor:15.462059672921896 - timing_s/step:29.548018461093307 - timing_s/stop_profile:0.00011456944048404694 - timing_per_token_ms/adv:0.0034791532076254785 - timing_per_token_ms/gen:0.1639794671583721 - timing_per_token_ms/update_actor:0.10937453789345464 - perf/total_num_tokens:141368 - perf/time_per_step:29.548018461093307 - perf/throughput:598.0434871890951 (TaskRunner pid=2026458) Training Progress: 63%|██████▎ | 63/100 [44:36<22:53, 37.11s/it] (TaskRunner pid=2026458) step:64 - global_seqlen/min:14067 - global_seqlen/max:24197 - global_seqlen/minmax_diff:10130 - global_seqlen/balanced_min:18752 - global_seqlen/balanced_max:25893 - global_seqlen/mean:19646.75 - actor/entropy:0.21532860398292542 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8697976469993591 - training/rollout_probs_diff_mean:0.004864232614636421 - training/rollout_probs_diff_std:0.013850908726453781 - training/rollout_actor_probs_pearson_corr:0.9980413317680359 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.70703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:1.0002894401550293 - rollout_corr/rollout_is_ratio_fraction_high:6.0805057728430256e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.4322023818967864e-05 - rollout_corr/rollout_is_max:8.297144889831543 - rollout_corr/rollout_is_min:0.23644955456256866 - rollout_corr/rollout_is_std:0.04067561775445938 - rollout_corr/rollout_is_eff_sample_size:0.9983489400333069 - rollout_corr/rollout_is_seq_mean:1.000409722328186 - rollout_corr/rollout_is_seq_std:0.0031241707038134336 - rollout_corr/rollout_is_seq_max:1.0219839811325073 - rollout_corr/rollout_is_seq_min:0.9937897324562073 - rollout_corr/rollout_is_seq_max_deviation:0.021983981132507324 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2946171956136823) - rollout_corr/training_log_ppl:np.float64(0.25466993783356884) - rollout_corr/kl:np.float64(0.0008532866019912433) - rollout_corr/k3_kl:np.float64(0.0010788948769118178) - rollout_corr/rollout_ppl:np.float64(1.2935234676115215) - rollout_corr/rollout_log_ppl:np.float64(0.25381665135137155) - rollout_corr/log_ppl_diff:np.float64(0.0008532864821972908) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022936501272852183) - rollout_corr/log_ppl_diff_max:np.float64(0.00881144404411316) - rollout_corr/log_ppl_diff_min:np.float64(-0.008881300687789917) - rollout_corr/ppl_ratio:np.float64(1.0008577948901802) - rollout_corr/chi2_token:np.float64(0.003156282240524888) - rollout_corr/chi2_seq:np.float64(3.035388365620747) - actor/pg_loss:np.float64(0.0014677918516099453) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004414845861901995) - actor/ppo_kl:np.float64(0.00026685018261352056) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_w_mean:np.float64(81162.51946377661) - self_distillation/token_reweight_w_std:np.float64(1170164.887981981) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010531947482377) - self_distillation/token_reweight_w_clip_frac:np.float64(0.049339942750521004) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.263648372143507) - perf/mfu/actor:np.float64(0.03462126720848994) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.74031829833984) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:1 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.08103185892105103 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.08103185892105103 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:321.2109375 - response_length/max:8192.0 - response_length/min:110.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:321.2109375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:292.75 - prompt_length/max:495.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010729953646659851 - timing_s/agent_loop/generate_sequences/min:np.float64(1.288361655548215) - timing_s/agent_loop/generate_sequences/max:np.float64(50.12881679646671) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2828551806887845) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.12881679646671) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:172 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.03219731338322 - timing_s/reward:0.05905804969370365 - timing_s/old_log_prob:2.5028711035847664 - timing_s/adv:0.508072393015027 - timing_s/update_actor:16.261700792238116 - timing_s/step:75.45720112323761 - timing_s/stop_profile:4.878826439380646e-05 - timing_per_token_ms/adv:0.0032325473234442533 - timing_per_token_ms/gen:0.6814082124940195 - timing_per_token_ms/update_actor:0.10346304600148953 - perf/total_num_tokens:157174 - perf/time_per_step:75.45720112323761 - perf/throughput:260.3694505964075 (TaskRunner pid=2026458) Training Progress: 64%|██████▍ | 64/100 [45:51<29:10, 48.62s/it] (TaskRunner pid=2026458) step:65 - global_seqlen/min:15781 - global_seqlen/max:21398 - global_seqlen/minmax_diff:5617 - global_seqlen/balanced_min:18710 - global_seqlen/balanced_max:18715 - global_seqlen/mean:18712.625 - actor/entropy:0.2451825588941574 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43764662742614746 - training/rollout_probs_diff_mean:0.005050553474575281 - training/rollout_probs_diff_std:0.013090502470731735 - training/rollout_actor_probs_pearson_corr:0.9983762502670288 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.74609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999321103096008 - rollout_corr/rollout_is_ratio_fraction_high:1.2970000170753337e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.8910002331249416e-05 - rollout_corr/rollout_is_max:2.924093246459961 - rollout_corr/rollout_is_min:0.4824545979499817 - rollout_corr/rollout_is_std:0.040282461792230606 - rollout_corr/rollout_is_eff_sample_size:0.9983795954951445 - rollout_corr/rollout_is_seq_mean:0.9999394416809082 - rollout_corr/rollout_is_seq_std:0.002558288164436817 - rollout_corr/rollout_is_seq_max:1.0071020126342773 - rollout_corr/rollout_is_seq_min:0.9918232560157776 - rollout_corr/rollout_is_seq_max_deviation:0.008176743984222412 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2973304083570838) - rollout_corr/training_log_ppl:np.float64(0.2564336106006522) - rollout_corr/kl:np.float64(0.0012040652124341022) - rollout_corr/k3_kl:np.float64(0.000990912636439134) - rollout_corr/rollout_ppl:np.float64(1.2957068742252886) - rollout_corr/rollout_log_ppl:np.float64(0.2552295460482128) - rollout_corr/log_ppl_diff:np.float64(0.0012040645524393767) - rollout_corr/log_ppl_abs_diff:np.float64(0.002526016061892733) - rollout_corr/log_ppl_diff_max:np.float64(0.011661797761917114) - rollout_corr/log_ppl_diff_min:np.float64(-0.006277412176132202) - rollout_corr/ppl_ratio:np.float64(1.001209291163832) - rollout_corr/chi2_token:np.float64(0.0015423318836838007) - rollout_corr/chi2_seq:np.float64(0.9701050808653235) - actor/pg_loss:np.float64(0.0032792859710752964) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007688776604481973) - actor/ppo_kl:np.float64(0.0002768948086071532) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_w_mean:np.float64(86791.39557419321) - self_distillation/token_reweight_w_std:np.float64(1084418.8798323106) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9982076121959835) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08052845095517114) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.3758349269628525) - perf/mfu/actor:np.float64(0.03501136292112507) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.70735931396484) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:1 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0052852751687169075 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0052852751687169075 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:301.17578125 - response_length/max:840.0 - response_length/min:129.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:301.17578125 - response_length_non_aborted/max:840.0 - response_length_non_aborted/min:129.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.59375 - prompt_length/max:818.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.040077328681946e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2666476387530565) - timing_s/agent_loop/generate_sequences/max:np.float64(6.11080476641655) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.148740480595734) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.11080476641655) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:228 - timing_s/agent_loop/slowest/response_length:840 - timing_s/gen:12.000718062743545 - timing_s/reward:0.056409286335110664 - timing_s/old_log_prob:2.2715286798775196 - timing_s/adv:0.546351995319128 - timing_s/update_actor:15.278582841157913 - timing_s/step:30.242278227582574 - timing_s/stop_profile:0.0001621972769498825 - timing_per_token_ms/adv:0.0036496215477460274 - timing_per_token_ms/gen:0.1556493179432633 - timing_per_token_ms/update_actor:0.10206065985636645 - perf/total_num_tokens:149701 - perf/time_per_step:30.242278227582574 - perf/throughput:618.757120716292 (TaskRunner pid=2026458) Training Progress: 65%|██████▌ | 65/100 [46:22<25:09, 43.12s/it] (TaskRunner pid=2026458) step:66 - global_seqlen/min:16986 - global_seqlen/max:21356 - global_seqlen/minmax_diff:4370 - global_seqlen/balanced_min:19034 - global_seqlen/balanced_max:19038 - global_seqlen/mean:19037.375 - actor/entropy:0.2407594919204712 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26579251885414124 - training/rollout_probs_diff_mean:0.005038884002715349 - training/rollout_probs_diff_std:0.013142738491296768 - training/rollout_actor_probs_pearson_corr:0.9983550906181335 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:1.000045657157898 - rollout_corr/rollout_is_ratio_fraction_high:2.5642670152592473e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.974934462457895e-05 - rollout_corr/rollout_is_max:2.355177402496338 - rollout_corr/rollout_is_min:0.19801293313503265 - rollout_corr/rollout_is_std:0.04063016548752785 - rollout_corr/rollout_is_eff_sample_size:0.9983520292581156 - rollout_corr/rollout_is_seq_mean:1.0001461505889893 - rollout_corr/rollout_is_seq_std:0.0026480352971702814 - rollout_corr/rollout_is_seq_max:1.0091423988342285 - rollout_corr/rollout_is_seq_min:0.9913557171821594 - rollout_corr/rollout_is_seq_max_deviation:0.009142398834228516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3078779838979244) - rollout_corr/training_log_ppl:np.float64(0.26354357780655846) - rollout_corr/kl:np.float64(0.0010749662497779866) - rollout_corr/k3_kl:np.float64(0.0014432408553943787) - rollout_corr/rollout_ppl:np.float64(1.3064235104247928) - rollout_corr/rollout_log_ppl:np.float64(0.2624686111230403) - rollout_corr/log_ppl_diff:np.float64(0.0010749666835181415) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029108942835591733) - rollout_corr/log_ppl_diff_max:np.float64(0.013920098543167114) - rollout_corr/log_ppl_diff_min:np.float64(-0.009019121527671814) - rollout_corr/ppl_ratio:np.float64(1.0010822340846062) - rollout_corr/chi2_token:np.float64(0.003545147832483053) - rollout_corr/chi2_seq:np.float64(1.548627549316734) - actor/pg_loss:np.float64(0.003003176534548402) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009145650715254305) - actor/ppo_kl:np.float64(0.0003188287759954278) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.546875) - self_distillation/token_reweight_w_mean:np.float64(126959.81397781824) - self_distillation/token_reweight_w_std:np.float64(1629415.2238851967) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008254635613412) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06369021107093431) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.4391012266278267) - perf/mfu/actor:np.float64(0.03551272812953314) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.41056060791016) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:1 - critic/score/mean:0.546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004540354944765568 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004540354944765568 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:304.66796875 - response_length/max:841.0 - response_length/min:130.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:304.66796875 - response_length_non_aborted/max:841.0 - response_length_non_aborted/min:130.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.25 - prompt_length/max:617.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014583580195903778 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5362531188875437) - timing_s/agent_loop/generate_sequences/max:np.float64(6.417147822678089) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.267450157953135) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.417147822678089) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:841 - timing_s/gen:12.298093724995852 - timing_s/reward:0.05658415146172047 - timing_s/old_log_prob:2.3751799557358027 - timing_s/adv:0.4913473054766655 - timing_s/update_actor:15.358580730855465 - timing_s/step:30.66549047641456 - timing_s/stop_profile:0.0001288391649723053 - timing_per_token_ms/adv:0.0032262017838374873 - timing_per_token_ms/gen:0.1576779758317309 - timing_per_token_ms/update_actor:0.10084492170569383 - perf/total_num_tokens:152299 - perf/time_per_step:30.66549047641456 - perf/throughput:620.8077778714162 (TaskRunner pid=2026458) Training Progress: 66%|██████▌ | 66/100 [46:52<22:19, 39.40s/it] (TaskRunner pid=2026458) step:67 - global_seqlen/min:15159 - global_seqlen/max:19915 - global_seqlen/minmax_diff:4756 - global_seqlen/balanced_min:18046 - global_seqlen/balanced_max:18050 - global_seqlen/mean:18048.5 - actor/entropy:0.22801370918750763 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9852267503738403 - training/rollout_probs_diff_mean:0.005018836818635464 - training/rollout_probs_diff_std:0.014277998358011246 - training/rollout_actor_probs_pearson_corr:0.9979830384254456 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:1.0002268552780151 - rollout_corr/rollout_is_ratio_fraction_high:2.8704287615255453e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001004650112008676 - rollout_corr/rollout_is_max:2.3428878784179688 - rollout_corr/rollout_is_min:0.0007197936065495014 - rollout_corr/rollout_is_std:0.040458161383867264 - rollout_corr/rollout_is_eff_sample_size:0.9983666439269478 - rollout_corr/rollout_is_seq_mean:1.0003077983856201 - rollout_corr/rollout_is_seq_std:0.0026670496445149183 - rollout_corr/rollout_is_seq_max:1.0164320468902588 - rollout_corr/rollout_is_seq_min:0.9921717047691345 - rollout_corr/rollout_is_seq_max_deviation:0.01643204689025879 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2767656035721302) - rollout_corr/training_log_ppl:np.float64(0.24004710461304057) - rollout_corr/kl:np.float64(0.0011609104617287258) - rollout_corr/k3_kl:np.float64(0.001655576513314827) - rollout_corr/rollout_ppl:np.float64(1.2751760962419212) - rollout_corr/rollout_log_ppl:np.float64(0.23888619328499772) - rollout_corr/log_ppl_diff:np.float64(0.0011609113280428573) - rollout_corr/log_ppl_abs_diff:np.float64(0.003236807693610899) - rollout_corr/log_ppl_diff_max:np.float64(0.017100274562835693) - rollout_corr/log_ppl_diff_min:np.float64(-0.02350728213787079) - rollout_corr/ppl_ratio:np.float64(1.0011714671272784) - rollout_corr/chi2_token:np.float64(0.004160782555118203) - rollout_corr/chi2_seq:np.float64(5.24587120115757) - actor/pg_loss:np.float64(0.0036287190159782767) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001126493662923167) - actor/ppo_kl:np.float64(0.0005246336532138685) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(249329.88290784857) - self_distillation/token_reweight_w_std:np.float64(2866482.3437989554) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994784640148282) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06014655635226518) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.471420306712389) - perf/mfu/actor:np.float64(0.033855896624554245) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.65926361083984) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:1 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004662652965635061 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004662652965635061 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:272.171875 - response_length/max:628.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:272.171875 - response_length_non_aborted/max:628.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.84375 - prompt_length/max:576.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.801262497901917e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2853057831525803) - timing_s/agent_loop/generate_sequences/max:np.float64(5.174439303576946) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.863578474461974) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.174439303576946) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:351 - timing_s/agent_loop/slowest/response_length:628 - timing_s/gen:10.778497187420726 - timing_s/reward:0.052504874765872955 - timing_s/old_log_prob:2.3092593122273684 - timing_s/adv:0.5003343336284161 - timing_s/update_actor:15.202262729406357 - timing_s/step:28.936630526557565 - timing_s/stop_profile:0.00012007169425487518 - timing_per_token_ms/adv:0.0034652071753083088 - timing_per_token_ms/gen:0.1546945460046605 - timing_per_token_ms/update_actor:0.10528757742614592 - perf/total_num_tokens:144388 - perf/time_per_step:28.936630526557565 - perf/throughput:623.7250043136634 (TaskRunner pid=2026458) Training Progress: 67%|██████▋ | 67/100 [47:21<19:57, 36.28s/it] (TaskRunner pid=2026458) step:68 - global_seqlen/min:14735 - global_seqlen/max:19627 - global_seqlen/minmax_diff:4892 - global_seqlen/balanced_min:17371 - global_seqlen/balanced_max:17375 - global_seqlen/mean:17373.75 - actor/entropy:0.26004546880722046 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23114943504333496 - training/rollout_probs_diff_mean:0.005509829614311457 - training/rollout_probs_diff_std:0.013461560010910034 - training/rollout_actor_probs_pearson_corr:0.9983240365982056 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.58984375 - self_distillation/correct_sample_fraction:0.4765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.58984375 - rollout_corr/rollout_is_mean:1.0001424551010132 - rollout_corr/rollout_is_ratio_fraction_high:3.430884680710733e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.14632738486398e-05 - rollout_corr/rollout_is_max:2.094249725341797 - rollout_corr/rollout_is_min:0.4616723358631134 - rollout_corr/rollout_is_std:0.04154840111732483 - rollout_corr/rollout_is_eff_sample_size:0.998277299143981 - rollout_corr/rollout_is_seq_mean:1.0000828504562378 - rollout_corr/rollout_is_seq_std:0.0031394551042467356 - rollout_corr/rollout_is_seq_max:1.0125868320465088 - rollout_corr/rollout_is_seq_min:0.9910873770713806 - rollout_corr/rollout_is_seq_max_deviation:0.012586832046508789 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3149044853635132) - rollout_corr/training_log_ppl:np.float64(0.26928867847891524) - rollout_corr/kl:np.float64(0.0007745910923624422) - rollout_corr/k3_kl:np.float64(0.0011792316492460486) - rollout_corr/rollout_ppl:np.float64(1.3139065788127482) - rollout_corr/rollout_log_ppl:np.float64(0.26851408526999876) - rollout_corr/log_ppl_diff:np.float64(0.0007745932089164853) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025513150612823665) - rollout_corr/log_ppl_diff_max:np.float64(0.009900659322738647) - rollout_corr/log_ppl_diff_min:np.float64(-0.015614360570907593) - rollout_corr/ppl_ratio:np.float64(1.0007805349305272) - rollout_corr/chi2_token:np.float64(0.0031827117782086134) - rollout_corr/chi2_seq:np.float64(0.9155827462673187) - actor/pg_loss:np.float64(0.0037549110129475594) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007232226507767336) - actor/ppo_kl:np.float64(-4.5092836657545377e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.58984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4765625) - self_distillation/token_reweight_w_mean:np.float64(174603.2610843731) - self_distillation/token_reweight_w_std:np.float64(2132750.0757954237) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9986913346219808) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0688194558606483) - self_distillation/empty_target_batch:np.float64(0.41015625) - actor/grad_norm:np.float64(0.4323424845933914) - perf/mfu/actor:np.float64(0.03227397909516918) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.99407958984375) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:1 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006370724178850651 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006370724178850651 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:227.7109375 - response_length/max:542.0 - response_length/min:102.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:227.7109375 - response_length_non_aborted/max:542.0 - response_length_non_aborted/min:102.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.21875 - prompt_length/max:704.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010382384061813354 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2069221679121256) - timing_s/agent_loop/generate_sequences/max:np.float64(4.399640990421176) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.518257544754306) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.399640990421176) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:508 - timing_s/agent_loop/slowest/response_length:542 - timing_s/gen:10.388320125639439 - timing_s/reward:0.053288890048861504 - timing_s/old_log_prob:2.246592290699482 - timing_s/adv:0.4770530089735985 - timing_s/update_actor:15.537012735381722 - timing_s/step:28.787579828873277 - timing_s/stop_profile:0.00013300031423568726 - timing_per_token_ms/adv:0.003432282962613127 - timing_per_token_ms/gen:0.1782056493916945 - timing_per_token_ms/update_actor:0.1117851121331155 - perf/total_num_tokens:138990 - perf/time_per_step:28.787579828873277 - perf/throughput:603.5154779692363 (TaskRunner pid=2026458) Training Progress: 68%|██████▊ | 68/100 [47:50<18:09, 34.04s/it] (TaskRunner pid=2026458) step:69 - global_seqlen/min:15072 - global_seqlen/max:24554 - global_seqlen/minmax_diff:9482 - global_seqlen/balanced_min:18606 - global_seqlen/balanced_max:18614 - global_seqlen/mean:18608.375 - actor/entropy:0.24838018417358398 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21835267543792725 - training/rollout_probs_diff_mean:0.004963765386492014 - training/rollout_probs_diff_std:0.012902619317173958 - training/rollout_actor_probs_pearson_corr:0.9984732270240784 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.56640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9996421337127686 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.750844502472319e-05 - rollout_corr/rollout_is_max:1.9659780263900757 - rollout_corr/rollout_is_min:0.26981014013290405 - rollout_corr/rollout_is_std:0.039976488798856735 - rollout_corr/rollout_is_eff_sample_size:0.9984033607144375 - rollout_corr/rollout_is_seq_mean:0.9996398687362671 - rollout_corr/rollout_is_seq_std:0.0026584318839013577 - rollout_corr/rollout_is_seq_max:1.0071197748184204 - rollout_corr/rollout_is_seq_min:0.9907388687133789 - rollout_corr/rollout_is_seq_max_deviation:0.009261131286621094 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3142020958475769) - rollout_corr/training_log_ppl:np.float64(0.2681919914903119) - rollout_corr/kl:np.float64(0.0011005508938133346) - rollout_corr/k3_kl:np.float64(0.0010078800825397138) - rollout_corr/rollout_ppl:np.float64(1.3127118106931448) - rollout_corr/rollout_log_ppl:np.float64(0.2670914397749584) - rollout_corr/log_ppl_diff:np.float64(0.0011005517153535038) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023552981147076935) - rollout_corr/log_ppl_diff_max:np.float64(0.013139069080352783) - rollout_corr/log_ppl_diff_min:np.float64(-0.007561713457107544) - rollout_corr/ppl_ratio:np.float64(1.001105590723455) - rollout_corr/chi2_token:np.float64(0.0017845977563410997) - rollout_corr/chi2_seq:np.float64(0.37655540159903467) - actor/pg_loss:np.float64(0.004595633479766548) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005241842873147107) - actor/ppo_kl:np.float64(-0.0001359343073588093) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.56640625) - self_distillation/token_reweight_w_mean:np.float64(242376.49935866683) - self_distillation/token_reweight_w_std:np.float64(3279716.472702818) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019234952051193) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08817204824299552) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.47684382647275925) - perf/mfu/actor:np.float64(0.03472723156176391) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.89558792114258) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:1 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0034019839949905872 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0034019839949905872 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:271.69921875 - response_length/max:979.0 - response_length/min:109.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:271.69921875 - response_length_non_aborted/max:979.0 - response_length_non_aborted/min:109.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:309.8125 - prompt_length/max:597.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001641698181629181 - timing_s/agent_loop/generate_sequences/min:np.float64(1.224152859300375) - timing_s/agent_loop/generate_sequences/max:np.float64(6.517763238400221) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8646460080635734) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.517763238400221) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:979 - timing_s/gen:12.344549676403403 - timing_s/reward:0.0557593759149313 - timing_s/old_log_prob:2.2437479849904776 - timing_s/adv:0.49268065579235554 - timing_s/update_actor:14.758547224104404 - timing_s/step:29.983076663687825 - timing_s/stop_profile:0.00012305378913879395 - timing_per_token_ms/adv:0.003309535731843562 - timing_per_token_ms/gen:0.17747896882184463 - timing_per_token_ms/update_actor:0.09913914584229147 - perf/total_num_tokens:148867 - perf/time_per_step:29.983076663687825 - perf/throughput:620.6292705957158 (TaskRunner pid=2026458) Training Progress: 69%|██████▉ | 69/100 [48:20<16:58, 32.84s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which of the given options is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". To determine the correct answer, we need to analyze the structure of the product and compare it with the options. (TaskRunner pid=2026458) (TaskRunner pid=2026458) The product contains a nitrile group (–n) attached to a cyclohexyl ring with a chlorine atom and a trifluoromethyl group (–C(F)(F)F). The structure also includes a benzene ring with a chlorine atom and a trifluoromethyl group. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Looking at the options: (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option A: Contains a nitrile group and a cyclohexyl ring with a chlorine atom and a trifluoromethyl group, matching the product's structure. (TaskRunner pid=2026458) - Option B: Contains a nitrile group but does not have the correct trifluoromethyl group or the correct chlorine atom placement. (TaskRunner pid=2026458) - Option C: Contains a nitrile group but does not match the correct structure of the product, especially the trifluoromethyl group. (TaskRunner pid=2026458) - Option D: Contains a nitrile group but does not have the correct structure of the product, especially the trifluoromethyl group. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the correct answer is Option A, as it matches the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60/actor (TaskRunner pid=2026458) step:70 - global_seqlen/min:12837 - global_seqlen/max:19669 - global_seqlen/minmax_diff:6832 - global_seqlen/balanced_min:17821 - global_seqlen/balanced_max:17831 - global_seqlen/mean:17825.125 - actor/entropy:0.2367510199546814 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.33400535583496094 - training/rollout_probs_diff_mean:0.005260511301457882 - training/rollout_probs_diff_std:0.013495014049112797 - training/rollout_actor_probs_pearson_corr:0.9982122778892517 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.55859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:0.9999286532402039 - rollout_corr/rollout_is_ratio_fraction_high:4.527823330136016e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.5092745343281422e-05 - rollout_corr/rollout_is_max:2.0651345252990723 - rollout_corr/rollout_is_min:0.28940898180007935 - rollout_corr/rollout_is_std:0.04225678741931915 - rollout_corr/rollout_is_eff_sample_size:0.9982174280795298 - rollout_corr/rollout_is_seq_mean:0.9999866485595703 - rollout_corr/rollout_is_seq_std:0.00303856679238379 - rollout_corr/rollout_is_seq_max:1.0101226568222046 - rollout_corr/rollout_is_seq_min:0.9901509284973145 - rollout_corr/rollout_is_seq_max_deviation:0.01012265682220459 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2921742992475629) - rollout_corr/training_log_ppl:np.float64(0.2528939112526132) - rollout_corr/kl:np.float64(0.0014884298633148774) - rollout_corr/k3_kl:np.float64(0.0010952503159273874) - rollout_corr/rollout_ppl:np.float64(1.2902372791431844) - rollout_corr/rollout_log_ppl:np.float64(0.2514054805797059) - rollout_corr/log_ppl_diff:np.float64(0.0014884306729072705) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028520045889308676) - rollout_corr/log_ppl_diff_max:np.float64(0.0136488676071167) - rollout_corr/log_ppl_diff_min:np.float64(-0.008753955364227295) - rollout_corr/ppl_ratio:np.float64(1.0014950232580304) - rollout_corr/chi2_token:np.float64(0.0014098831452429295) - rollout_corr/chi2_seq:np.float64(0.360071137547493) - actor/pg_loss:np.float64(0.0027717185439541936) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003434370364630013) - actor/ppo_kl:np.float64(0.0005119445677905787) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.55859375) - self_distillation/token_reweight_w_mean:np.float64(69071.25273821992) - self_distillation/token_reweight_w_std:np.float64(1045658.8316577639) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004556903149933) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06493389664683491) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.3887019380927086) - perf/mfu/actor:np.float64(0.0325070599234879) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.87537002563477) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6851190476190476) - val-aux/sciknoweval/reward/std@16:np.float64(0.1354593314121246) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7372809523809524) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.09687814765870191) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6334190476190477) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1304254368473761) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6852190476190475) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.13554201698632168) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7717095238095236) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.0626549022097885) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5802904761904761) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11571033595984151) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.697847619047619) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.10316342544885265) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7940761904761904) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.040787555686648956) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5327809523809524) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09421852437209087) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7060047619047619) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07272307414099477) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8114142857142858) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.02829417180722469) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4923904761904762) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06774926114873739) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.712047619047619) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.0541181009825183) - val-aux/sciknoweval/score/mean@16:np.float64(0.6851190476190476) - val-aux/sciknoweval/score/std@16:np.float64(0.1354593314121246) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7372809523809524) - val-aux/sciknoweval/score/best@2/std:np.float64(0.09687814765870191) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6334190476190477) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1304254368473761) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6852190476190475) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.13554201698632168) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7717095238095236) - val-aux/sciknoweval/score/best@4/std:np.float64(0.0626549022097885) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5802904761904761) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11571033595984151) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.697847619047619) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.10316342544885265) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7940761904761904) - val-aux/sciknoweval/score/best@8/std:np.float64(0.040787555686648956) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5327809523809524) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09421852437209087) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7060047619047619) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07272307414099477) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8114142857142858) - val-aux/sciknoweval/score/best@16/std:np.float64(0.02829417180722469) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4923904761904762) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06774926114873739) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.712047619047619) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.0541181009825183) - val-core/sciknoweval/acc/mean@16:np.float64(0.6851190476190476) - val-aux/sciknoweval/acc/std@16:np.float64(0.1354593314121246) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7372809523809524) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.09687814765870191) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6334190476190477) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1304254368473761) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6852190476190475) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.13554201698632168) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7717095238095236) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.0626549022097885) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5802904761904761) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11571033595984151) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.697847619047619) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.10316342544885265) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7940761904761904) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.040787555686648956) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5327809523809524) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09421852437209087) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7060047619047619) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07272307414099477) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8114142857142858) - val-core/sciknoweval/acc/best@16/std:np.float64(0.02829417180722469) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4923904761904762) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06774926114873739) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.712047619047619) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.0541181009825183) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:1 - critic/score/mean:0.55859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036373515613377094 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0036373515613377094 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:258.81640625 - response_length/max:938.0 - response_length/min:113.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:258.81640625 - response_length_non_aborted/max:938.0 - response_length_non_aborted/min:113.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.21875 - prompt_length/max:701.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016323477029800415 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2699729204177856) - timing_s/agent_loop/generate_sequences/max:np.float64(6.416786439716816) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.761028841654479) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.416786439716816) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:938 - timing_s/gen:12.333267584443092 - timing_s/reward:0.05591795593500137 - timing_s/old_log_prob:2.2401132248342037 - timing_s/adv:0.506872920319438 - timing_s/update_actor:15.221702938899398 - timing_s/step:30.44651348516345 - timing_s/testing:81.57863584533334 - timing_s/save_checkpoint:6.6424451023340225 - timing_s/stop_profile:0.00010693445801734924 - timing_per_token_ms/adv:0.003554483631387143 - timing_per_token_ms/gen:0.18614286165149482 - timing_per_token_ms/update_actor:0.1067433113295096 - perf/total_num_tokens:142601 - perf/time_per_step:30.44651348516345 - perf/throughput:585.4570182128132 (TaskRunner pid=2026458) Training Progress: 70%|███████ | 70/100 [50:19<29:18, 58.61s/it] (TaskRunner pid=2026458) step:71 - global_seqlen/min:15483 - global_seqlen/max:28966 - global_seqlen/minmax_diff:13483 - global_seqlen/balanced_min:18380 - global_seqlen/balanced_max:25891 - global_seqlen/mean:19333.5 - actor/entropy:0.21173985302448273 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4046512842178345 - training/rollout_probs_diff_mean:0.0045140874572098255 - training/rollout_probs_diff_std:0.012615382671356201 - training/rollout_actor_probs_pearson_corr:0.9983184337615967 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:0.9998964667320251 - rollout_corr/rollout_is_ratio_fraction_high:1.2943308320245706e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.471654342021793e-05 - rollout_corr/rollout_is_max:2.0498809814453125 - rollout_corr/rollout_is_min:0.38866347074508667 - rollout_corr/rollout_is_std:0.03843024745583534 - rollout_corr/rollout_is_eff_sample_size:0.9985250563653204 - rollout_corr/rollout_is_seq_mean:0.9998904466629028 - rollout_corr/rollout_is_seq_std:0.0028965510427951813 - rollout_corr/rollout_is_seq_max:1.0107746124267578 - rollout_corr/rollout_is_seq_min:0.9922298192977905 - rollout_corr/rollout_is_seq_max_deviation:0.010774612426757812 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2945193266496062) - rollout_corr/training_log_ppl:np.float64(0.2542406297643538) - rollout_corr/kl:np.float64(0.0010824492369110317) - rollout_corr/k3_kl:np.float64(0.0011051825894128342) - rollout_corr/rollout_ppl:np.float64(1.293056899216026) - rollout_corr/rollout_log_ppl:np.float64(0.2531581802359142) - rollout_corr/log_ppl_diff:np.float64(0.0010824495284396107) - rollout_corr/log_ppl_abs_diff:np.float64(0.00271785432141769) - rollout_corr/log_ppl_diff_max:np.float64(0.014131128787994385) - rollout_corr/log_ppl_diff_min:np.float64(-0.009531468152999878) - rollout_corr/ppl_ratio:np.float64(1.0010888520628214) - rollout_corr/chi2_token:np.float64(0.002212358172982931) - rollout_corr/chi2_seq:np.float64(0.6793141439557076) - actor/pg_loss:np.float64(0.002975177369080484) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000972424295014207) - actor/ppo_kl:np.float64(7.461081707615946e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(134136.12361396686) - self_distillation/token_reweight_w_std:np.float64(1624645.7259825736) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988281566184014) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07092063533491455) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.4537533223628998) - perf/mfu/actor:np.float64(0.03374089352858298) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.65079879760742) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:1 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.06832286715507507 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.06832286715507507 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:301.796875 - response_length/max:8192.0 - response_length/min:93.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:301.796875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:302.375 - prompt_length/max:666.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.677132427692413e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1272850260138512) - timing_s/agent_loop/generate_sequences/max:np.float64(47.67028555460274) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.04078216401831) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(47.67028555460274) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:666 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:53.34413158334792 - timing_s/reward:0.05393144115805626 - timing_s/old_log_prob:2.626548172906041 - timing_s/adv:0.5594039708375931 - timing_s/update_actor:16.17577089741826 - timing_s/step:72.8568678945303 - timing_s/stop_profile:2.8194859623908997e-05 - timing_per_token_ms/adv:0.0036168048389944468 - timing_per_token_ms/gen:0.6904495415913527 - timing_per_token_ms/update_actor:0.1045838240451694 - perf/total_num_tokens:154668 - perf/time_per_step:72.8568678945303 - perf/throughput:265.3627661840711 (TaskRunner pid=2026458) Training Progress: 71%|███████ | 71/100 [51:32<30:23, 62.89s/it] (TaskRunner pid=2026458) step:72 - global_seqlen/min:14180 - global_seqlen/max:19404 - global_seqlen/minmax_diff:5224 - global_seqlen/balanced_min:16784 - global_seqlen/balanced_max:16786 - global_seqlen/mean:16785.25 - actor/entropy:0.2532234191894531 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37658625841140747 - training/rollout_probs_diff_mean:0.005358405876904726 - training/rollout_probs_diff_std:0.01324445940554142 - training/rollout_actor_probs_pearson_corr:0.9983224272727966 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.71484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.000357747077942 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6467605829238892 - rollout_corr/rollout_is_min:0.5746954679489136 - rollout_corr/rollout_is_std:0.04072979837656021 - rollout_corr/rollout_is_eff_sample_size:0.9983450191216581 - rollout_corr/rollout_is_seq_mean:1.0003376007080078 - rollout_corr/rollout_is_seq_std:0.0029273421969264746 - rollout_corr/rollout_is_seq_max:1.0102252960205078 - rollout_corr/rollout_is_seq_min:0.9919271469116211 - rollout_corr/rollout_is_seq_max_deviation:0.010225296020507812 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3001699862070382) - rollout_corr/training_log_ppl:np.float64(0.25781238308991306) - rollout_corr/kl:np.float64(0.0006664992815785808) - rollout_corr/k3_kl:np.float64(0.0009609586412011595) - rollout_corr/rollout_ppl:np.float64(1.2992715733125806) - rollout_corr/rollout_log_ppl:np.float64(0.25714588271512184) - rollout_corr/log_ppl_diff:np.float64(0.0006665003747912124) - rollout_corr/log_ppl_abs_diff:np.float64(0.002382488935836591) - rollout_corr/log_ppl_diff_max:np.float64(0.009035348892211914) - rollout_corr/log_ppl_diff_min:np.float64(-0.009852290153503418) - rollout_corr/ppl_ratio:np.float64(1.0006713322363794) - rollout_corr/chi2_token:np.float64(0.0025634136982262135) - rollout_corr/chi2_seq:np.float64(1.15624994575046) - actor/pg_loss:np.float64(0.003654895001091063) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009709688329166966) - actor/ppo_kl:np.float64(0.00014433233151223135) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_w_mean:np.float64(213152.86371882842) - self_distillation/token_reweight_w_std:np.float64(2646677.052918058) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9974161058198661) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09049051211331971) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.43892450630664825) - perf/mfu/actor:np.float64(0.03149296183920366) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.40920639038086) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:1 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009050944820046425 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009050944820046425 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:237.6953125 - response_length/max:467.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:237.6953125 - response_length_non_aborted/max:467.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.84375 - prompt_length/max:620.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.147497773170471e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3375506270676851) - timing_s/agent_loop/generate_sequences/max:np.float64(4.0367792677134275) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.622172743664123) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.0367792677134275) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:326 - timing_s/agent_loop/slowest/response_length:467 - timing_s/gen:10.345969874411821 - timing_s/reward:0.05066956579685211 - timing_s/old_log_prob:2.2474361546337605 - timing_s/adv:0.4828864671289921 - timing_s/update_actor:14.702434957027435 - timing_s/step:27.925694411620498 - timing_s/stop_profile:3.0016526579856873e-05 - timing_per_token_ms/adv:0.0035960625186472655 - timing_per_token_ms/gen:0.17002415570109813 - timing_per_token_ms/update_actor:0.1094892461910564 - perf/total_num_tokens:134282 - perf/time_per_step:27.925694411620498 - perf/throughput:601.0683119491305 (TaskRunner pid=2026458) Training Progress: 72%|███████▏ | 72/100 [52:00<24:27, 52.41s/it] (TaskRunner pid=2026458) step:73 - global_seqlen/min:15559 - global_seqlen/max:20190 - global_seqlen/minmax_diff:4631 - global_seqlen/balanced_min:17173 - global_seqlen/balanced_max:17174 - global_seqlen/mean:17173.5 - actor/entropy:0.24391600489616394 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135395884513855 - training/rollout_probs_diff_mean:0.0049339784309268 - training/rollout_probs_diff_std:0.012776896357536316 - training/rollout_actor_probs_pearson_corr:0.998493492603302 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:1.000167965888977 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.0463656003121287e-05 - rollout_corr/rollout_is_max:1.767087697982788 - rollout_corr/rollout_is_min:0.47380420565605164 - rollout_corr/rollout_is_std:0.03922998905181885 - rollout_corr/rollout_is_eff_sample_size:0.9984639671449654 - rollout_corr/rollout_is_seq_mean:1.0002386569976807 - rollout_corr/rollout_is_seq_std:0.0029904802795499563 - rollout_corr/rollout_is_seq_max:1.0164623260498047 - rollout_corr/rollout_is_seq_min:0.9890998005867004 - rollout_corr/rollout_is_seq_max_deviation:0.016462326049804688 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2968735643662512) - rollout_corr/training_log_ppl:np.float64(0.25511286556138657) - rollout_corr/kl:np.float64(0.0009459122673263209) - rollout_corr/k3_kl:np.float64(0.0009600993125502555) - rollout_corr/rollout_ppl:np.float64(1.2955679851584136) - rollout_corr/rollout_log_ppl:np.float64(0.2541669542551972) - rollout_corr/log_ppl_diff:np.float64(0.0009459113061893731) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024720369547139853) - rollout_corr/log_ppl_diff_max:np.float64(0.012520670890808105) - rollout_corr/log_ppl_diff_min:np.float64(-0.0084361732006073) - rollout_corr/ppl_ratio:np.float64(1.0009514002595097) - rollout_corr/chi2_token:np.float64(0.001954789971932769) - rollout_corr/chi2_seq:np.float64(0.6583140406291932) - actor/pg_loss:np.float64(0.0032611776841804385) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000914203752472531) - actor/ppo_kl:np.float64(0.00038008014968227144) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.546875) - self_distillation/token_reweight_w_mean:np.float64(311191.56539016124) - self_distillation/token_reweight_w_std:np.float64(2860058.778589467) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015116163995117) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06564340484328568) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.42411575466394424) - perf/mfu/actor:np.float64(0.031904336867600726) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.16930770874023) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:1 - critic/score/mean:0.546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001247106003575027 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001247106003575027 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:256.453125 - response_length/max:556.0 - response_length/min:109.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:256.453125 - response_length_non_aborted/max:556.0 - response_length_non_aborted/min:109.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.21875 - prompt_length/max:557.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.804398536682129e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1476398799568415) - timing_s/agent_loop/generate_sequences/max:np.float64(4.690207960084081) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.7287142683780985) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.690207960084081) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:229 - timing_s/agent_loop/slowest/response_length:556 - timing_s/gen:10.320297587662935 - timing_s/reward:0.0441241730004549 - timing_s/old_log_prob:2.164985027164221 - timing_s/adv:0.4551666658371687 - timing_s/update_actor:14.973258456215262 - timing_s/step:27.992081409320235 - timing_s/stop_profile:2.880394458770752e-05 - timing_per_token_ms/adv:0.003313001614676454 - timing_per_token_ms/gen:0.15719700218824917 - timing_per_token_ms/update_actor:0.10898519853418975 - perf/total_num_tokens:137388 - perf/time_per_step:27.992081409320235 - perf/throughput:613.5127913096851 (TaskRunner pid=2026458) Training Progress: 73%|███████▎ | 73/100 [52:28<20:17, 45.09s/it] (TaskRunner pid=2026458) step:74 - global_seqlen/min:13447 - global_seqlen/max:21573 - global_seqlen/minmax_diff:8126 - global_seqlen/balanced_min:16831 - global_seqlen/balanced_max:16917 - global_seqlen/mean:16843.875 - actor/entropy:0.24996380507946014 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3424726128578186 - training/rollout_probs_diff_mean:0.004949183203279972 - training/rollout_probs_diff_std:0.012561521492898464 - training/rollout_actor_probs_pearson_corr:0.9985408782958984 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.62109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:0.9998319149017334 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.6325607399921864e-05 - rollout_corr/rollout_is_max:1.7145717144012451 - rollout_corr/rollout_is_min:0.17540410161018372 - rollout_corr/rollout_is_std:0.03882066160440445 - rollout_corr/rollout_is_eff_sample_size:0.9984948673727597 - rollout_corr/rollout_is_seq_mean:0.9999223947525024 - rollout_corr/rollout_is_seq_std:0.0029323359485715628 - rollout_corr/rollout_is_seq_max:1.009851336479187 - rollout_corr/rollout_is_seq_min:0.9908139109611511 - rollout_corr/rollout_is_seq_max_deviation:0.009851336479187012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3159835655242205) - rollout_corr/training_log_ppl:np.float64(0.26891483918007) - rollout_corr/kl:np.float64(0.0010735450956096404) - rollout_corr/k3_kl:np.float64(0.0011378318484958072) - rollout_corr/rollout_ppl:np.float64(1.3145075496286154) - rollout_corr/rollout_log_ppl:np.float64(0.26784129165753257) - rollout_corr/log_ppl_diff:np.float64(0.0010735475225374103) - rollout_corr/log_ppl_abs_diff:np.float64(0.0030432986677624285) - rollout_corr/log_ppl_diff_max:np.float64(0.012433648109436035) - rollout_corr/log_ppl_diff_min:np.float64(-0.009890466928482056) - rollout_corr/ppl_ratio:np.float64(1.0010810112580657) - rollout_corr/chi2_token:np.float64(0.0024086034391075373) - rollout_corr/chi2_seq:np.float64(1.130532833514735) - actor/pg_loss:np.float64(0.0028035463765263557) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007331344913836801) - actor/ppo_kl:np.float64(0.00016281594710676472) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_w_mean:np.float64(183593.82668150146) - self_distillation/token_reweight_w_std:np.float64(2093931.331328835) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001840919721872) - self_distillation/token_reweight_w_clip_frac:np.float64(0.060318807751173154) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.3842148035764694) - perf/mfu/actor:np.float64(0.031244242735268143) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.2487654685974) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:1 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0016619311645627022 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0016619311645627022 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:252.96484375 - response_length/max:993.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:252.96484375 - response_length_non_aborted/max:993.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.40625 - prompt_length/max:665.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7175992727279663e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.240340730175376) - timing_s/agent_loop/generate_sequences/max:np.float64(6.539421858265996) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.619629321692628) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.539421858265996) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:291 - timing_s/agent_loop/slowest/response_length:993 - timing_s/gen:12.270534712821245 - timing_s/reward:0.04380444623529911 - timing_s/old_log_prob:2.2239825148135424 - timing_s/adv:0.5528004374355078 - timing_s/update_actor:15.180137295275927 - timing_s/step:30.30796923674643 - timing_s/stop_profile:5.2906572818756104e-05 - timing_per_token_ms/adv:0.004102384675701908 - timing_per_token_ms/gen:0.18947999062402515 - timing_per_token_ms/update_actor:0.11265324409671117 - perf/total_num_tokens:134751 - perf/time_per_step:30.30796923674643 - perf/throughput:555.7572950014052 (TaskRunner pid=2026458) Training Progress: 74%|███████▍ | 74/100 [52:58<17:37, 40.66s/it] (TaskRunner pid=2026458) step:75 - global_seqlen/min:14038 - global_seqlen/max:20039 - global_seqlen/minmax_diff:6001 - global_seqlen/balanced_min:18453 - global_seqlen/balanced_max:18459 - global_seqlen/mean:18457.625 - actor/entropy:0.24194937944412231 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4665491580963135 - training/rollout_probs_diff_mean:0.005109612364321947 - training/rollout_probs_diff_std:0.013375498354434967 - training/rollout_actor_probs_pearson_corr:0.9982732534408569 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.73046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:0.9997668862342834 - rollout_corr/rollout_is_ratio_fraction_high:2.9664348403457552e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.449652260518633e-05 - rollout_corr/rollout_is_max:2.278925657272339 - rollout_corr/rollout_is_min:0.3029129207134247 - rollout_corr/rollout_is_std:0.040616225451231 - rollout_corr/rollout_is_eff_sample_size:0.9983523857083623 - rollout_corr/rollout_is_seq_mean:0.9997424483299255 - rollout_corr/rollout_is_seq_std:0.002785229589790106 - rollout_corr/rollout_is_seq_max:1.00969660282135 - rollout_corr/rollout_is_seq_min:0.9920895099639893 - rollout_corr/rollout_is_seq_max_deviation:0.009696602821350098 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2928350591100752) - rollout_corr/training_log_ppl:np.float64(0.2525325442838948) - rollout_corr/kl:np.float64(0.0011320576819571215) - rollout_corr/k3_kl:np.float64(0.0010110275231340893) - rollout_corr/rollout_ppl:np.float64(1.2913245973177254) - rollout_corr/rollout_log_ppl:np.float64(0.2514004855765961) - rollout_corr/log_ppl_diff:np.float64(0.001132058707298711) - rollout_corr/log_ppl_abs_diff:np.float64(0.002472092892276123) - rollout_corr/log_ppl_diff_max:np.float64(0.01102399080991745) - rollout_corr/log_ppl_diff_min:np.float64(-0.008037805557250977) - rollout_corr/ppl_ratio:np.float64(1.0011374710593373) - rollout_corr/chi2_token:np.float64(0.0017729024402797222) - rollout_corr/chi2_seq:np.float64(0.6597908593248576) - actor/pg_loss:np.float64(0.0018322557443752885) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036231086232874077) - actor/ppo_kl:np.float64(-3.7132614664869834e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_w_mean:np.float64(138490.71220888244) - self_distillation/token_reweight_w_std:np.float64(1204680.9713628318) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000044918153435) - self_distillation/token_reweight_w_clip_frac:np.float64(0.041466833936283365) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.29594044387340546) - perf/mfu/actor:np.float64(0.03412545800281451) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.2256622314453) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:1 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:6.489075894933194e-05 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:6.489075894933194e-05 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:263.36328125 - response_length/max:680.0 - response_length/min:127.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:263.36328125 - response_length_non_aborted/max:680.0 - response_length_non_aborted/min:127.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.4375 - prompt_length/max:608.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.105005741119385e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.53872056491673) - timing_s/agent_loop/generate_sequences/max:np.float64(5.3331572357565165) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.911040708415385) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.3331572357565165) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:367 - timing_s/agent_loop/slowest/response_length:680 - timing_s/gen:11.2033476177603 - timing_s/reward:0.055844465270638466 - timing_s/old_log_prob:2.219141924753785 - timing_s/adv:0.4940295107662678 - timing_s/update_actor:15.452317954972386 - timing_s/step:29.51427774503827 - timing_s/stop_profile:0.0001329611986875534 - timing_per_token_ms/adv:0.003345700697992481 - timing_per_token_ms/gen:0.16617000070838908 - timing_per_token_ms/update_actor:0.10464725252417623 - perf/total_num_tokens:147661 - perf/time_per_step:29.51427774503827 - perf/throughput:625.3795251046915 (TaskRunner pid=2026458) Training Progress: 75%|███████▌ | 75/100 [53:28<15:33, 37.33s/it] (TaskRunner pid=2026458) step:76 - global_seqlen/min:15400 - global_seqlen/max:20933 - global_seqlen/minmax_diff:5533 - global_seqlen/balanced_min:18318 - global_seqlen/balanced_max:18443 - global_seqlen/mean:18349.25 - actor/entropy:0.23565568029880524 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6839720010757446 - training/rollout_probs_diff_mean:0.00476441252976656 - training/rollout_probs_diff_std:0.01295106578618288 - training/rollout_actor_probs_pearson_corr:0.9984387159347534 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65625 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65625 - rollout_corr/rollout_is_mean:0.999829113483429 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.072877345606685e-05 - rollout_corr/rollout_is_max:1.8591279983520508 - rollout_corr/rollout_is_min:0.14236801862716675 - rollout_corr/rollout_is_std:0.038981545716524124 - rollout_corr/rollout_is_eff_sample_size:0.9984822693566601 - rollout_corr/rollout_is_seq_mean:0.9998883008956909 - rollout_corr/rollout_is_seq_std:0.0026900693774223328 - rollout_corr/rollout_is_seq_max:1.0127232074737549 - rollout_corr/rollout_is_seq_min:0.9910237193107605 - rollout_corr/rollout_is_seq_max_deviation:0.012723207473754883 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3050027494318783) - rollout_corr/training_log_ppl:np.float64(0.26096004332066514) - rollout_corr/kl:np.float64(0.001120272666678801) - rollout_corr/k3_kl:np.float64(0.000998899616035942) - rollout_corr/rollout_ppl:np.float64(1.3034690767526627) - rollout_corr/rollout_log_ppl:np.float64(0.25983976959832944) - rollout_corr/log_ppl_diff:np.float64(0.0011202737223356962) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024523677420802414) - rollout_corr/log_ppl_diff_max:np.float64(0.01079338788986206) - rollout_corr/log_ppl_diff_min:np.float64(-0.011696979403495789) - rollout_corr/ppl_ratio:np.float64(1.001125503797084) - rollout_corr/chi2_token:np.float64(0.0016837881412357092) - rollout_corr/chi2_seq:np.float64(0.4817654227372259) - actor/pg_loss:np.float64(0.0022594364127144217) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005095393812553084) - actor/ppo_kl:np.float64(0.000131376795138749) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(86914.96509032976) - self_distillation/token_reweight_w_std:np.float64(1369863.7220702441) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999393313191831) - self_distillation/token_reweight_w_clip_frac:np.float64(0.041579058073693886) - self_distillation/empty_target_batch:np.float64(0.34375) - actor/grad_norm:np.float64(0.3407774642109871) - perf/mfu/actor:np.float64(0.034170829781458016) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.09953689575195) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:1 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002358874771744013 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002358874771744013 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:287.7265625 - response_length/max:956.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:287.7265625 - response_length_non_aborted/max:956.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:285.6875 - prompt_length/max:526.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015220046043395996 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3281499948352575) - timing_s/agent_loop/generate_sequences/max:np.float64(6.804745152592659) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0793025807652157) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.804745152592659) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:306 - timing_s/agent_loop/slowest/response_length:956 - timing_s/gen:12.672567792236805 - timing_s/reward:0.05679433047771454 - timing_s/old_log_prob:2.200064066797495 - timing_s/adv:0.5124469976872206 - timing_s/update_actor:15.322908757254481 - timing_s/step:30.852899638935924 - timing_s/stop_profile:0.00013393722474575043 - timing_per_token_ms/adv:0.0034909260438929423 - timing_per_token_ms/gen:0.17204604784594754 - timing_per_token_ms/update_actor:0.10438375381319728 - perf/total_num_tokens:146794 - perf/time_per_step:30.852899638935924 - perf/throughput:594.7334031723717 (TaskRunner pid=2026458) Training Progress: 76%|███████▌ | 76/100 [53:59<14:09, 35.40s/it] (TaskRunner pid=2026458) step:77 - global_seqlen/min:16068 - global_seqlen/max:21136 - global_seqlen/minmax_diff:5068 - global_seqlen/balanced_min:18347 - global_seqlen/balanced_max:18355 - global_seqlen/mean:18353.125 - actor/entropy:0.24358664453029633 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3763711750507355 - training/rollout_probs_diff_mean:0.005085653159767389 - training/rollout_probs_diff_std:0.012961719185113907 - training/rollout_actor_probs_pearson_corr:0.9984139204025269 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.7265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:0.9999815821647644 - rollout_corr/rollout_is_ratio_fraction_high:1.5232524674502201e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.61626215535216e-05 - rollout_corr/rollout_is_max:2.443542003631592 - rollout_corr/rollout_is_min:0.0030977046117186546 - rollout_corr/rollout_is_std:0.04062869772315025 - rollout_corr/rollout_is_eff_sample_size:0.9983521480748362 - rollout_corr/rollout_is_seq_mean:1.0000320672988892 - rollout_corr/rollout_is_seq_std:0.0030800357926636934 - rollout_corr/rollout_is_seq_max:1.008837342262268 - rollout_corr/rollout_is_seq_min:0.9924508333206177 - rollout_corr/rollout_is_seq_max_deviation:0.008837342262268066 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.30547040887177) - rollout_corr/training_log_ppl:np.float64(0.262247261358425) - rollout_corr/kl:np.float64(0.001232846368517393) - rollout_corr/k3_kl:np.float64(0.0012621798805412254) - rollout_corr/rollout_ppl:np.float64(1.303758448921144) - rollout_corr/rollout_log_ppl:np.float64(0.2610144141508499) - rollout_corr/log_ppl_diff:np.float64(0.0012328472075751051) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028479194588726386) - rollout_corr/log_ppl_diff_max:np.float64(0.01525259017944336) - rollout_corr/log_ppl_diff_min:np.float64(-0.007772982120513916) - rollout_corr/ppl_ratio:np.float64(1.0012400422710925) - rollout_corr/chi2_token:np.float64(0.0023893536999821663) - rollout_corr/chi2_seq:np.float64(0.8159184374380857) - actor/pg_loss:np.float64(0.002790838829241693) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009325358823843999) - actor/ppo_kl:np.float64(0.0003359223030457992) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_w_mean:np.float64(180959.5167653456) - self_distillation/token_reweight_w_std:np.float64(2185127.907001757) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988750158809125) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06124208416440524) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.4322587177157402) - perf/mfu/actor:np.float64(0.034241717008689325) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.1767692565918) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:1 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0021534981206059456 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0021534981206059456 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:256.44140625 - response_length/max:782.0 - response_length/min:126.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:256.44140625 - response_length_non_aborted/max:782.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:317.09375 - prompt_length/max:716.0 - prompt_length/min:173.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013564899563789368 - timing_s/agent_loop/generate_sequences/min:np.float64(1.442766698077321) - timing_s/agent_loop/generate_sequences/max:np.float64(5.666013702750206) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.7960805580951273) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.666013702750206) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:368 - timing_s/agent_loop/slowest/response_length:782 - timing_s/gen:11.443953750655055 - timing_s/reward:0.054556334391236305 - timing_s/old_log_prob:2.218324951827526 - timing_s/adv:0.4792825896292925 - timing_s/update_actor:15.45035495236516 - timing_s/step:29.73319257237017 - timing_s/stop_profile:2.9521062970161438e-05 - timing_per_token_ms/adv:0.0032643118653450875 - timing_per_token_ms/gen:0.1743203057267446 - timing_per_token_ms/update_actor:0.10522972894510581 - perf/total_num_tokens:146825 - perf/time_per_step:29.73319257237017 - perf/throughput:617.2604894455499 (TaskRunner pid=2026458) Training Progress: 77%|███████▋ | 77/100 [54:28<12:55, 33.71s/it] (TaskRunner pid=2026458) step:78 - global_seqlen/min:15356 - global_seqlen/max:19696 - global_seqlen/minmax_diff:4340 - global_seqlen/balanced_min:17649 - global_seqlen/balanced_max:17655 - global_seqlen/mean:17652.75 - actor/entropy:0.23699037730693817 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39975830912590027 - training/rollout_probs_diff_mean:0.005081570707261562 - training/rollout_probs_diff_std:0.013243689201772213 - training/rollout_actor_probs_pearson_corr:0.9983371496200562 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9999217987060547 - rollout_corr/rollout_is_ratio_fraction_high:4.307003109715879e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.307003109715879e-05 - rollout_corr/rollout_is_max:2.9357402324676514 - rollout_corr/rollout_is_min:0.4458056390285492 - rollout_corr/rollout_is_std:0.04097927734255791 - rollout_corr/rollout_is_eff_sample_size:0.9983230388235971 - rollout_corr/rollout_is_seq_mean:1.0000085830688477 - rollout_corr/rollout_is_seq_std:0.002981689525768161 - rollout_corr/rollout_is_seq_max:1.0086852312088013 - rollout_corr/rollout_is_seq_min:0.9888802170753479 - rollout_corr/rollout_is_seq_max_deviation:0.0111197829246521 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3004110623151064) - rollout_corr/training_log_ppl:np.float64(0.257169630160206) - rollout_corr/kl:np.float64(0.0012380461494814199) - rollout_corr/k3_kl:np.float64(0.001083872083654569) - rollout_corr/rollout_ppl:np.float64(1.2987679266370833) - rollout_corr/rollout_log_ppl:np.float64(0.25593158273841254) - rollout_corr/log_ppl_diff:np.float64(0.0012380474217934534) - rollout_corr/log_ppl_abs_diff:np.float64(0.002574279482359998) - rollout_corr/log_ppl_diff_max:np.float64(0.015157654881477356) - rollout_corr/log_ppl_diff_min:np.float64(-0.0076891034841537476) - rollout_corr/ppl_ratio:np.float64(1.0012440220452845) - rollout_corr/chi2_token:np.float64(0.001788182882592082) - rollout_corr/chi2_seq:np.float64(1.9581220541149378) - actor/pg_loss:np.float64(0.0033810948953032494) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009038109546963824) - actor/ppo_kl:np.float64(0.00033419562218028886) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(189436.75045557343) - self_distillation/token_reweight_w_std:np.float64(2503367.794994272) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009928282815963) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07160566429956816) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.3780764937400818) - perf/mfu/actor:np.float64(0.03291894268049457) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.0838165283203) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:1 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019830160308629274 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0019830160308629274 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:272.0859375 - response_length/max:624.0 - response_length/min:113.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:272.0859375 - response_length_non_aborted/max:624.0 - response_length_non_aborted/min:113.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.5625 - prompt_length/max:449.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.438022971153259e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3432003632187843) - timing_s/agent_loop/generate_sequences/max:np.float64(5.082683224231005) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9419547133074957) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.082683224231005) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:190 - timing_s/agent_loop/slowest/response_length:624 - timing_s/gen:10.891948658972979 - timing_s/reward:0.054630380123853683 - timing_s/old_log_prob:2.192325107753277 - timing_s/adv:0.4803696293383837 - timing_s/update_actor:15.322288809344172 - timing_s/step:29.02985128760338 - timing_s/stop_profile:0.00014253705739974976 - timing_per_token_ms/adv:0.003401521217220997 - timing_per_token_ms/gen:0.15637219196274413 - timing_per_token_ms/update_actor:0.10849788849714756 - perf/total_num_tokens:141222 - perf/time_per_step:29.02985128760338 - perf/throughput:608.0895773495835 (TaskRunner pid=2026458) Training Progress: 78%|███████▊ | 78/100 [54:57<11:51, 32.32s/it] (TaskRunner pid=2026458) step:79 - global_seqlen/min:13318 - global_seqlen/max:19723 - global_seqlen/minmax_diff:6405 - global_seqlen/balanced_min:16851 - global_seqlen/balanced_max:16855 - global_seqlen/mean:16852.75 - actor/entropy:0.2380763292312622 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24131831526756287 - training/rollout_probs_diff_mean:0.00509747164323926 - training/rollout_probs_diff_std:0.01309633906930685 - training/rollout_actor_probs_pearson_corr:0.998324453830719 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:0.999850869178772 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4865467164781876e-05 - rollout_corr/rollout_is_max:1.6486144065856934 - rollout_corr/rollout_is_min:0.48596420884132385 - rollout_corr/rollout_is_std:0.04019506648182869 - rollout_corr/rollout_is_eff_sample_size:0.998386606117186 - rollout_corr/rollout_is_seq_mean:0.9999116659164429 - rollout_corr/rollout_is_seq_std:0.0028780419379472733 - rollout_corr/rollout_is_seq_max:1.0085704326629639 - rollout_corr/rollout_is_seq_min:0.9908472299575806 - rollout_corr/rollout_is_seq_max_deviation:0.009152770042419434 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2909451411105692) - rollout_corr/training_log_ppl:np.float64(0.250693125941325) - rollout_corr/kl:np.float64(0.0009612457183649781) - rollout_corr/k3_kl:np.float64(0.0010165603632685816) - rollout_corr/rollout_ppl:np.float64(1.2896752119995654) - rollout_corr/rollout_log_ppl:np.float64(0.24973187799332663) - rollout_corr/log_ppl_diff:np.float64(0.0009612479479983449) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024955651024356484) - rollout_corr/log_ppl_diff_max:np.float64(0.008482903242111206) - rollout_corr/log_ppl_diff_min:np.float64(-0.007829368114471436) - rollout_corr/ppl_ratio:np.float64(1.0009662464726716) - rollout_corr/chi2_token:np.float64(0.002142100827768445) - rollout_corr/chi2_seq:np.float64(0.36312326532788575) - actor/pg_loss:np.float64(0.0036202146438881755) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008493167902088317) - actor/ppo_kl:np.float64(1.813888876256442e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(207000.82824485935) - self_distillation/token_reweight_w_std:np.float64(2774328.0958690653) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004315504338592) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07667552542989142) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.5015818998217583) - perf/mfu/actor:np.float64(0.031159245026246143) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.08302688598633) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:1 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.000724691548384726 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.000724691548384726 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:262.7734375 - response_length/max:630.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:262.7734375 - response_length_non_aborted/max:630.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.875 - prompt_length/max:766.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013626180589199066 - timing_s/agent_loop/generate_sequences/min:np.float64(1.074292877689004) - timing_s/agent_loop/generate_sequences/max:np.float64(4.940646631643176) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.7851551758358255) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.940646631643176) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:276 - timing_s/agent_loop/slowest/response_length:630 - timing_s/gen:11.244631016626954 - timing_s/reward:0.05328981205821037 - timing_s/old_log_prob:2.2739136423915625 - timing_s/adv:0.5032551754266024 - timing_s/update_actor:15.453622449189425 - timing_s/step:29.61371649429202 - timing_s/stop_profile:0.00012503564357757568 - timing_per_token_ms/adv:0.0037327377981828066 - timing_per_token_ms/gen:0.16715669714028472 - timing_per_token_ms/update_actor:0.1146224091705317 - perf/total_num_tokens:134822 - perf/time_per_step:29.61371649429202 - perf/throughput:569.0859505340484 (TaskRunner pid=2026458) Training Progress: 79%|███████▉ | 79/100 [55:27<11:02, 31.52s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which of the given options is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". The product contains a structure with a nitrile group (n1) and a substituted benzene ring with a chlorine atom and a trifluoromethyl group (C(F)(F)F). (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A: "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" contains a nitrile group (n1) and a substituted benzene ring with a chlorine atom and a trifluoromethyl group, matching the product structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B: "O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1" contains a chlorine atom but does not have the correct nitrile group and trifluoromethyl group structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C: "O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1" has a different structure with an oxygen atom and a different substitution pattern, not matching the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D: "O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F" contains a different structure with a double bond and a different substitution pattern, not matching the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the correct answer is Option A, as it matches the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70/actor (TaskRunner pid=2026458) step:80 - global_seqlen/min:16122 - global_seqlen/max:23121 - global_seqlen/minmax_diff:6999 - global_seqlen/balanced_min:18851 - global_seqlen/balanced_max:18860 - global_seqlen/mean:18857.375 - actor/entropy:0.22489960491657257 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3807961344718933 - training/rollout_probs_diff_mean:0.005050069652497768 - training/rollout_probs_diff_std:0.013410150073468685 - training/rollout_actor_probs_pearson_corr:0.998170018196106 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.890625 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.890625 - rollout_corr/rollout_is_mean:1.0001280307769775 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.252001963322982e-05 - rollout_corr/rollout_is_max:1.815222144126892 - rollout_corr/rollout_is_min:0.23840588331222534 - rollout_corr/rollout_is_std:0.040402136743068695 - rollout_corr/rollout_is_eff_sample_size:0.9983706838295904 - rollout_corr/rollout_is_seq_mean:1.0001842975616455 - rollout_corr/rollout_is_seq_std:0.002584717469289899 - rollout_corr/rollout_is_seq_max:1.0079708099365234 - rollout_corr/rollout_is_seq_min:0.9935450553894043 - rollout_corr/rollout_is_seq_max_deviation:0.007970809936523438 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.284651028458029) - rollout_corr/training_log_ppl:np.float64(0.2446684459282551) - rollout_corr/kl:np.float64(0.0011069629296169836) - rollout_corr/k3_kl:np.float64(0.0011388705046897485) - rollout_corr/rollout_ppl:np.float64(1.2831353461369872) - rollout_corr/rollout_log_ppl:np.float64(0.24356148090737406) - rollout_corr/log_ppl_diff:np.float64(0.0011069650208810344) - rollout_corr/log_ppl_abs_diff:np.float64(0.002547982890973799) - rollout_corr/log_ppl_diff_max:np.float64(0.011913180351257324) - rollout_corr/log_ppl_diff_min:np.float64(-0.00954301655292511) - rollout_corr/ppl_ratio:np.float64(1.0011127449106425) - rollout_corr/chi2_token:np.float64(0.0022547231055796146) - rollout_corr/chi2_seq:np.float64(0.705732943257317) - actor/pg_loss:np.float64(0.003671735990792513) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012357970999801182) - actor/ppo_kl:np.float64(0.00041334695177397407) - actor/pg_clipfrac_lower:np.float64(2.100134406646248e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.890625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.890625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(147860.0034218824) - self_distillation/token_reweight_w_std:np.float64(2243142.9639211292) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0017643314786255) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08743216533912346) - self_distillation/empty_target_batch:np.float64(0.109375) - actor/grad_norm:np.float64(0.46084368228912354) - perf/mfu/actor:np.float64(0.0348035546181388) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.0935287475586) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6788690476190476) - val-aux/sciknoweval/reward/std@16:np.float64(0.13354484169727368) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7320952380952381) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.10085829139940647) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6251047619047618) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.12484909422849347) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6789857142857142) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.13328046089893666) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7699333333333332) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.06599987955078114) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5759809523809525) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10617677121952601) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6900714285714286) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1031404305918038) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7950666666666667) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.04028583090873896) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5310380952380952) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08297067195315495) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6945809523809524) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07152449490492849) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8101809523809524) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.021522036689052344) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.49653333333333327) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05480896953178682) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6956380952380953) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.04910459662444814) - val-aux/sciknoweval/score/mean@16:np.float64(0.6788690476190476) - val-aux/sciknoweval/score/std@16:np.float64(0.13354484169727368) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7320952380952381) - val-aux/sciknoweval/score/best@2/std:np.float64(0.10085829139940647) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6251047619047618) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.12484909422849347) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6789857142857142) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.13328046089893666) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7699333333333332) - val-aux/sciknoweval/score/best@4/std:np.float64(0.06599987955078114) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5759809523809525) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10617677121952601) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6900714285714286) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1031404305918038) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7950666666666667) - val-aux/sciknoweval/score/best@8/std:np.float64(0.04028583090873896) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5310380952380952) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08297067195315495) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6945809523809524) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07152449490492849) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8101809523809524) - val-aux/sciknoweval/score/best@16/std:np.float64(0.021522036689052344) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.49653333333333327) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05480896953178682) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6956380952380953) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.04910459662444814) - val-core/sciknoweval/acc/mean@16:np.float64(0.6788690476190476) - val-aux/sciknoweval/acc/std@16:np.float64(0.13354484169727368) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7320952380952381) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.10085829139940647) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6251047619047618) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.12484909422849347) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6789857142857142) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.13328046089893666) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7699333333333332) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.06599987955078114) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5759809523809525) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10617677121952601) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6900714285714286) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1031404305918038) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7950666666666667) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.04028583090873896) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5310380952380952) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08297067195315495) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6945809523809524) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07152449490492849) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8101809523809524) - val-core/sciknoweval/acc/best@16/std:np.float64(0.021522036689052344) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.49653333333333327) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05480896953178682) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6956380952380953) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.04910459662444814) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0028665580321103334 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0028665580321103334 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:275.60546875 - response_length/max:780.0 - response_length/min:128.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:275.60546875 - response_length_non_aborted/max:780.0 - response_length_non_aborted/min:128.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.6875 - prompt_length/max:950.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012242421507835388 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5708349235355854) - timing_s/agent_loop/generate_sequences/max:np.float64(5.78690424002707) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9946807451196946) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.78690424002707) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:635 - timing_s/agent_loop/slowest/response_length:780 - timing_s/gen:11.676643127575517 - timing_s/reward:0.060109492391347885 - timing_s/old_log_prob:2.263966953381896 - timing_s/adv:0.5860696248710155 - timing_s/update_actor:15.427983490750194 - timing_s/step:30.10305192694068 - timing_s/testing:71.47364217042923 - timing_s/save_checkpoint:6.949206165969372 - timing_s/stop_profile:0.00013513118028640747 - timing_per_token_ms/adv:0.0038848834002016158 - timing_per_token_ms/gen:0.16549703249345216 - timing_per_token_ms/update_actor:0.10226757098184526 - perf/total_num_tokens:150859 - perf/time_per_step:30.10305192694068 - perf/throughput:626.4273484883312 (TaskRunner pid=2026458) Training Progress: 80%|████████ | 80/100 [57:16<18:12, 54.65s/it] (TaskRunner pid=2026458) step:81 - global_seqlen/min:15749 - global_seqlen/max:23070 - global_seqlen/minmax_diff:7321 - global_seqlen/balanced_min:19345 - global_seqlen/balanced_max:19350 - global_seqlen/mean:19349.125 - actor/entropy:0.24268725514411926 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967529773712158 - training/rollout_probs_diff_mean:0.0049947695806622505 - training/rollout_probs_diff_std:0.013053881004452705 - training/rollout_actor_probs_pearson_corr:0.9983764290809631 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9997670650482178 - rollout_corr/rollout_is_ratio_fraction_high:1.3156336535757873e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.262534614303149e-05 - rollout_corr/rollout_is_max:2.0260822772979736 - rollout_corr/rollout_is_min:0.2609400451183319 - rollout_corr/rollout_is_std:0.03953344747424126 - rollout_corr/rollout_is_eff_sample_size:0.9984388918274393 - rollout_corr/rollout_is_seq_mean:0.999757707118988 - rollout_corr/rollout_is_seq_std:0.002598333638161421 - rollout_corr/rollout_is_seq_max:1.0065761804580688 - rollout_corr/rollout_is_seq_min:0.9917704463005066 - rollout_corr/rollout_is_seq_max_deviation:0.008229553699493408 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2969164904206991) - rollout_corr/training_log_ppl:np.float64(0.2555096454452723) - rollout_corr/kl:np.float64(0.0012044042446035519) - rollout_corr/k3_kl:np.float64(0.001068380127094315) - rollout_corr/rollout_ppl:np.float64(1.295284100342542) - rollout_corr/rollout_log_ppl:np.float64(0.25430524366674945) - rollout_corr/log_ppl_diff:np.float64(0.0012044017785228789) - rollout_corr/log_ppl_abs_diff:np.float64(0.002569538773968816) - rollout_corr/log_ppl_diff_max:np.float64(0.00988227128982544) - rollout_corr/log_ppl_diff_min:np.float64(-0.007966101169586182) - rollout_corr/ppl_ratio:np.float64(1.0012098925653845) - rollout_corr/chi2_token:np.float64(0.0018374514766037464) - rollout_corr/chi2_seq:np.float64(0.6602487270720303) - actor/pg_loss:np.float64(0.0036679430631920695) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007375691257038852) - actor/ppo_kl:np.float64(0.00012186931077984298) - actor/pg_clipfrac_lower:np.float64(8.037551197048742e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(152346.46296780813) - self_distillation/token_reweight_w_std:np.float64(2223816.070475471) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006990339607) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07411914720432833) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.43313899636268616) - perf/mfu/actor:np.float64(0.035338759221170066) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.7124252319336) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0049204700626432896 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0049204700626432896 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:296.91015625 - response_length/max:761.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:296.91015625 - response_length_non_aborted/max:761.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:307.75 - prompt_length/max:735.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.6648626923561096e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7210864126682281) - timing_s/agent_loop/generate_sequences/max:np.float64(5.891790520399809) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.159814590260794) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.891790520399809) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:423 - timing_s/agent_loop/slowest/response_length:761 - timing_s/gen:11.6525589954108 - timing_s/reward:0.05602879822254181 - timing_s/old_log_prob:2.4974898006767035 - timing_s/adv:0.5450900681316853 - timing_s/update_actor:15.660592023283243 - timing_s/step:30.51396298967302 - timing_s/stop_profile:0.00014164485037326813 - timing_per_token_ms/adv:0.0035214129071190897 - timing_per_token_ms/gen:0.15330499013815205 - timing_per_token_ms/update_actor:0.10117119006210386 - perf/total_num_tokens:154793 - perf/time_per_step:30.51396298967302 - perf/throughput:634.1072448225888 (TaskRunner pid=2026458) Training Progress: 81%|████████ | 81/100 [57:46<15:01, 47.43s/it] (TaskRunner pid=2026458) step:82 - global_seqlen/min:15004 - global_seqlen/max:21121 - global_seqlen/minmax_diff:6117 - global_seqlen/balanced_min:17893 - global_seqlen/balanced_max:17896 - global_seqlen/mean:17895.25 - actor/entropy:0.23795437812805176 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31782522797584534 - training/rollout_probs_diff_mean:0.004893430043011904 - training/rollout_probs_diff_std:0.012663507834076881 - training/rollout_actor_probs_pearson_corr:0.9984432458877563 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.000130534172058 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.6648900529835373e-05 - rollout_corr/rollout_is_max:1.7479747533798218 - rollout_corr/rollout_is_min:0.3629961311817169 - rollout_corr/rollout_is_std:0.038970839232206345 - rollout_corr/rollout_is_eff_sample_size:0.998484170923625 - rollout_corr/rollout_is_seq_mean:1.0002964735031128 - rollout_corr/rollout_is_seq_std:0.002696148119866848 - rollout_corr/rollout_is_seq_max:1.0123590230941772 - rollout_corr/rollout_is_seq_min:0.9923802018165588 - rollout_corr/rollout_is_seq_max_deviation:0.012359023094177246 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.296073100529611) - rollout_corr/training_log_ppl:np.float64(0.2552825820748694) - rollout_corr/kl:np.float64(0.0009574908228102696) - rollout_corr/k3_kl:np.float64(0.0011017269548574404) - rollout_corr/rollout_ppl:np.float64(1.2947798878885806) - rollout_corr/rollout_log_ppl:np.float64(0.254325088521) - rollout_corr/log_ppl_diff:np.float64(0.0009574935538694263) - rollout_corr/log_ppl_abs_diff:np.float64(0.002439796517137438) - rollout_corr/log_ppl_diff_max:np.float64(0.014934465289115906) - rollout_corr/log_ppl_diff_min:np.float64(-0.010826826095581055) - rollout_corr/ppl_ratio:np.float64(1.0009630019776523) - rollout_corr/chi2_token:np.float64(0.002452905522659421) - rollout_corr/chi2_seq:np.float64(0.423899469897151) - actor/pg_loss:np.float64(0.0041424884693697095) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013430351091301418) - actor/ppo_kl:np.float64(0.000410171516927349) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_w_mean:np.float64(233069.21662305342) - self_distillation/token_reweight_w_std:np.float64(2612623.9466522) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009098192676902) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0835862404492218) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.45514093339443207) - perf/mfu/actor:np.float64(0.03317730103506178) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.53085327148438) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:1 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003221185877919197 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003221185877919197 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:293.1640625 - response_length/max:790.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:293.1640625 - response_length_non_aborted/max:790.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.0625 - prompt_length/max:408.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010550394654273987 - timing_s/agent_loop/generate_sequences/min:np.float64(1.162438377737999) - timing_s/agent_loop/generate_sequences/max:np.float64(6.011627459898591) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.059660606115358) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.011627459898591) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:298 - timing_s/agent_loop/slowest/response_length:790 - timing_s/gen:12.346360467374325 - timing_s/reward:0.05675779469311237 - timing_s/old_log_prob:2.2316900305449963 - timing_s/adv:0.5058559346944094 - timing_s/update_actor:15.28704277612269 - timing_s/step:30.49560439772904 - timing_s/stop_profile:0.00011858716607093811 - timing_per_token_ms/adv:0.0035334511580895024 - timing_per_token_ms/gen:0.16450846725348867 - timing_per_token_ms/update_actor:0.10678142786579323 - perf/total_num_tokens:143162 - perf/time_per_step:30.49560439772904 - perf/throughput:586.8140787310525 (TaskRunner pid=2026458) Training Progress: 82%|████████▏ | 82/100 [58:17<12:42, 42.36s/it] (TaskRunner pid=2026458) step:83 - global_seqlen/min:15510 - global_seqlen/max:20545 - global_seqlen/minmax_diff:5035 - global_seqlen/balanced_min:18663 - global_seqlen/balanced_max:18665 - global_seqlen/mean:18664.0 - actor/entropy:0.253889262676239 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553972005844116 - training/rollout_probs_diff_mean:0.005415121093392372 - training/rollout_probs_diff_std:0.013402428478002548 - training/rollout_actor_probs_pearson_corr:0.9983406662940979 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6875 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6875 - rollout_corr/rollout_is_mean:1.0002059936523438 - rollout_corr/rollout_is_ratio_fraction_high:2.7475547540234402e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.3737773770117201e-05 - rollout_corr/rollout_is_max:2.8205859661102295 - rollout_corr/rollout_is_min:0.3914983570575714 - rollout_corr/rollout_is_std:0.04187990352511406 - rollout_corr/rollout_is_eff_sample_size:0.9982497385555893 - rollout_corr/rollout_is_seq_mean:1.000227689743042 - rollout_corr/rollout_is_seq_std:0.002809274010360241 - rollout_corr/rollout_is_seq_max:1.009268045425415 - rollout_corr/rollout_is_seq_min:0.9934084415435791 - rollout_corr/rollout_is_seq_max_deviation:0.009268045425415039 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3143565426580608) - rollout_corr/training_log_ppl:np.float64(0.2695897765806876) - rollout_corr/kl:np.float64(0.000838180147326284) - rollout_corr/k3_kl:np.float64(0.001138548173230447) - rollout_corr/rollout_ppl:np.float64(1.313231692649424) - rollout_corr/rollout_log_ppl:np.float64(0.26875159548944794) - rollout_corr/log_ppl_diff:np.float64(0.0008381810912396759) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024882252619136125) - rollout_corr/log_ppl_diff_max:np.float64(0.013383328914642334) - rollout_corr/log_ppl_diff_min:np.float64(-0.00783604383468628) - rollout_corr/ppl_ratio:np.float64(1.0008434464689344) - rollout_corr/chi2_token:np.float64(0.002809858415275812) - rollout_corr/chi2_seq:np.float64(0.9911151258274913) - actor/pg_loss:np.float64(0.004593900288455188) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000947352435105131) - actor/ppo_kl:np.float64(0.00014004049228333315) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(142169.04989654478) - self_distillation/token_reweight_w_std:np.float64(1783576.419247636) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9992821563500911) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07421927264658734) - self_distillation/empty_target_batch:np.float64(0.3125) - actor/grad_norm:np.float64(0.4911562651395798) - perf/mfu/actor:np.float64(0.03439597980816678) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.20166397094727) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:1 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0015317617217078805 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0015317617217078805 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:284.34375 - response_length/max:727.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:284.34375 - response_length_non_aborted/max:727.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.90625 - prompt_length/max:563.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.794160723686218e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2851199880242348) - timing_s/agent_loop/generate_sequences/max:np.float64(5.475394682958722) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0284413718109136) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.475394682958722) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:226 - timing_s/agent_loop/slowest/response_length:727 - timing_s/gen:11.69941053725779 - timing_s/reward:0.055808091536164284 - timing_s/old_log_prob:2.32892463542521 - timing_s/adv:0.5197819676250219 - timing_s/update_actor:15.535809490829706 - timing_s/step:30.227494234219193 - timing_s/stop_profile:0.00012695416808128357 - timing_per_token_ms/adv:0.0034811801303647524 - timing_per_token_ms/gen:0.16072385066020703 - timing_per_token_ms/update_actor:0.10404930274076904 - perf/total_num_tokens:149312 - perf/time_per_step:30.227494234219193 - perf/throughput:617.4511143855027 (TaskRunner pid=2026458) Training Progress: 83%|████████▎ | 83/100 [58:47<10:58, 38.73s/it] (TaskRunner pid=2026458) step:84 - global_seqlen/min:14717 - global_seqlen/max:21368 - global_seqlen/minmax_diff:6651 - global_seqlen/balanced_min:17413 - global_seqlen/balanced_max:17422 - global_seqlen/mean:17419.375 - actor/entropy:0.22988727688789368 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35856932401657104 - training/rollout_probs_diff_mean:0.005215250421315432 - training/rollout_probs_diff_std:0.013718393631279469 - training/rollout_actor_probs_pearson_corr:0.9981431365013123 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.89453125 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.89453125 - rollout_corr/rollout_is_mean:0.9996470212936401 - rollout_corr/rollout_is_ratio_fraction_high:1.3454785403155256e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.072871423792094e-05 - rollout_corr/rollout_is_max:2.1147913932800293 - rollout_corr/rollout_is_min:0.390633225440979 - rollout_corr/rollout_is_std:0.041462238878011703 - rollout_corr/rollout_is_eff_sample_size:0.9982825263238067 - rollout_corr/rollout_is_seq_mean:0.9997439980506897 - rollout_corr/rollout_is_seq_std:0.0025515363086014986 - rollout_corr/rollout_is_seq_max:1.0085729360580444 - rollout_corr/rollout_is_seq_min:0.9917205572128296 - rollout_corr/rollout_is_seq_max_deviation:0.008572936058044434 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2788517312146723) - rollout_corr/training_log_ppl:np.float64(0.24251381697831675) - rollout_corr/kl:np.float64(0.0013615037188472456) - rollout_corr/k3_kl:np.float64(0.0013387326044949077) - rollout_corr/rollout_ppl:np.float64(1.2770492630079389) - rollout_corr/rollout_log_ppl:np.float64(0.2411523106857203) - rollout_corr/log_ppl_diff:np.float64(0.0013615062925964594) - rollout_corr/log_ppl_abs_diff:np.float64(0.002768128179013729) - rollout_corr/log_ppl_diff_max:np.float64(0.011708617210388184) - rollout_corr/log_ppl_diff_min:np.float64(-0.010206863284111023) - rollout_corr/ppl_ratio:np.float64(1.0013677747920156) - rollout_corr/chi2_token:np.float64(0.0024920080322772264) - rollout_corr/chi2_seq:np.float64(0.9803334202151746) - actor/pg_loss:np.float64(0.0052698974031955) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015525902631452482) - actor/ppo_kl:np.float64(0.00019933821041862032) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.89453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.89453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(172811.0304318224) - self_distillation/token_reweight_w_std:np.float64(2611723.788451826) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009630725253373) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09219890262465924) - self_distillation/empty_target_batch:np.float64(0.10546875) - actor/grad_norm:np.float64(0.5317341759800911) - perf/mfu/actor:np.float64(0.03233182941898891) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.27649307250977) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011396203190088272 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011396203190088272 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:290.32421875 - response_length/max:900.0 - response_length/min:117.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:290.32421875 - response_length_non_aborted/max:900.0 - response_length_non_aborted/min:117.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.03125 - prompt_length/max:466.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003882218152284622 - timing_s/agent_loop/generate_sequences/min:np.float64(1.449787463992834) - timing_s/agent_loop/generate_sequences/max:np.float64(6.467833820730448) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1213262150922674) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.467833820730448) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:245 - timing_s/agent_loop/slowest/response_length:900 - timing_s/gen:12.418748388066888 - timing_s/reward:0.06932966038584709 - timing_s/old_log_prob:2.232461268082261 - timing_s/adv:0.5236291699111462 - timing_s/update_actor:15.485476253554225 - timing_s/step:30.817566089332104 - timing_s/stop_profile:0.000115908682346344 - timing_per_token_ms/adv:0.0037575197869552305 - timing_per_token_ms/gen:0.16709159194417458 - timing_per_token_ms/update_actor:0.1111225019091832 - perf/total_num_tokens:139355 - perf/time_per_step:30.817566089332104 - perf/throughput:565.2417504194123 (TaskRunner pid=2026458) Training Progress: 84%|████████▍ | 84/100 [59:18<09:42, 36.38s/it] (TaskRunner pid=2026458) step:85 - global_seqlen/min:16721 - global_seqlen/max:24217 - global_seqlen/minmax_diff:7496 - global_seqlen/balanced_min:19463 - global_seqlen/balanced_max:19465 - global_seqlen/mean:19464.25 - actor/entropy:0.23170782625675201 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5486323833465576 - training/rollout_probs_diff_mean:0.005258036311715841 - training/rollout_probs_diff_std:0.013953270390629768 - training/rollout_actor_probs_pearson_corr:0.998067319393158 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0003063678741455 - rollout_corr/rollout_is_ratio_fraction_high:2.5328639821964316e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.065727964392863e-05 - rollout_corr/rollout_is_max:2.629549741744995 - rollout_corr/rollout_is_min:0.32894784212112427 - rollout_corr/rollout_is_std:0.04162293300032616 - rollout_corr/rollout_is_eff_sample_size:0.9982714780308726 - rollout_corr/rollout_is_seq_mean:1.0004109144210815 - rollout_corr/rollout_is_seq_std:0.002749430714175105 - rollout_corr/rollout_is_seq_max:1.0109564065933228 - rollout_corr/rollout_is_seq_min:0.9916076064109802 - rollout_corr/rollout_is_seq_max_deviation:0.010956406593322754 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2771666692569852) - rollout_corr/training_log_ppl:np.float64(0.24135592477978207) - rollout_corr/kl:np.float64(0.0007578132510701607) - rollout_corr/k3_kl:np.float64(0.0010257670675173358) - rollout_corr/rollout_ppl:np.float64(1.2761707119643688) - rollout_corr/rollout_log_ppl:np.float64(0.24059811038023327) - rollout_corr/log_ppl_diff:np.float64(0.0007578143995488063) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024299477954627946) - rollout_corr/log_ppl_diff_max:np.float64(0.011833712458610535) - rollout_corr/log_ppl_diff_min:np.float64(-0.010344773530960083) - rollout_corr/ppl_ratio:np.float64(1.0007629161700606) - rollout_corr/chi2_token:np.float64(0.0026269492227584124) - rollout_corr/chi2_seq:np.float64(1.3403879939578474) - actor/pg_loss:np.float64(0.002229055855423212) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019035061723116087) - actor/ppo_kl:np.float64(0.00024953764273760726) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(17317.446490210947) - self_distillation/token_reweight_w_std:np.float64(276465.0479241874) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991332795470953) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03143878703122027) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.2347664088010788) - perf/mfu/actor:np.float64(0.03632172645099713) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.27149963378906) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:1 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0004859932523686439 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:0.0004859932523686439 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:308.4453125 - response_length/max:777.0 - response_length/min:123.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:308.4453125 - response_length_non_aborted/max:777.0 - response_length_non_aborted/min:123.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.8125 - prompt_length/max:706.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014561228454113007 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4749425910413265) - timing_s/agent_loop/generate_sequences/max:np.float64(6.026074631139636) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2942245794984046) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.026074631139636) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:235 - timing_s/agent_loop/slowest/response_length:777 - timing_s/gen:12.095296228304505 - timing_s/reward:0.05645551718771458 - timing_s/old_log_prob:2.26230669952929 - timing_s/adv:0.4968310799449682 - timing_s/update_actor:15.393540350720286 - timing_s/step:30.395571315661073 - timing_s/stop_profile:0.00012073665857315063 - timing_per_token_ms/adv:0.0031906641660028527 - timing_per_token_ms/gen:0.15317869644011683 - timing_per_token_ms/update_actor:0.09885777997302932 - perf/total_num_tokens:155714 - perf/time_per_step:30.395571315661073 - perf/throughput:640.3646701640117 (TaskRunner pid=2026458) Training Progress: 85%|████████▌ | 85/100 [59:48<08:38, 34.60s/it] (TaskRunner pid=2026458) step:86 - global_seqlen/min:16265 - global_seqlen/max:20200 - global_seqlen/minmax_diff:3935 - global_seqlen/balanced_min:18647 - global_seqlen/balanced_max:18651 - global_seqlen/mean:18649.75 - actor/entropy:0.23041389882564545 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.898743212223053 - training/rollout_probs_diff_mean:0.005259583238512278 - training/rollout_probs_diff_std:0.014470859430730343 - training/rollout_actor_probs_pearson_corr:0.9979584813117981 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:0.9998511075973511 - rollout_corr/rollout_is_ratio_fraction_high:3.836807809420861e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.394679803634062e-05 - rollout_corr/rollout_is_max:2.1972851753234863 - rollout_corr/rollout_is_min:0.0777815580368042 - rollout_corr/rollout_is_std:0.04242711886763573 - rollout_corr/rollout_is_eff_sample_size:0.9982026989977762 - rollout_corr/rollout_is_seq_mean:0.9998696446418762 - rollout_corr/rollout_is_seq_std:0.0025317175313830376 - rollout_corr/rollout_is_seq_max:1.009598731994629 - rollout_corr/rollout_is_seq_min:0.9924889802932739 - rollout_corr/rollout_is_seq_max_deviation:0.009598731994628906 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2799476818181574) - rollout_corr/training_log_ppl:np.float64(0.24274212954333052) - rollout_corr/kl:np.float64(0.0012324780372026112) - rollout_corr/k3_kl:np.float64(0.0011905511769896293) - rollout_corr/rollout_ppl:np.float64(1.2783302562311292) - rollout_corr/rollout_log_ppl:np.float64(0.24150965179433115) - rollout_corr/log_ppl_diff:np.float64(0.0012324777489993721) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024851207563187927) - rollout_corr/log_ppl_diff_max:np.float64(0.009878866374492645) - rollout_corr/log_ppl_diff_min:np.float64(-0.00684240460395813) - rollout_corr/ppl_ratio:np.float64(1.0012374119833112) - rollout_corr/chi2_token:np.float64(0.0022098186891525984) - rollout_corr/chi2_seq:np.float64(0.6068784659728408) - actor/pg_loss:np.float64(0.0033142786705866456) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009858607454589219) - actor/ppo_kl:np.float64(0.0001475137922213321) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(165894.66037644562) - self_distillation/token_reweight_w_std:np.float64(2346342.531521928) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011869312729686) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07381428830558434) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.42134569585323334) - perf/mfu/actor:np.float64(0.0348608857845462) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.1626739501953) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:1 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0060413735918700695 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0060413735918700695 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:305.4296875 - response_length/max:825.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:305.4296875 - response_length_non_aborted/max:825.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.375 - prompt_length/max:503.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016151554882526398 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6213552672415972) - timing_s/agent_loop/generate_sequences/max:np.float64(6.171470927074552) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.300390396914736) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.171470927074552) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:825 - timing_s/gen:12.024075420573354 - timing_s/reward:0.05586143955588341 - timing_s/old_log_prob:2.3416189160197973 - timing_s/adv:0.5560324247926474 - timing_s/update_actor:15.314254153519869 - timing_s/step:30.382415380328894 - timing_s/stop_profile:0.0001287907361984253 - timing_per_token_ms/adv:0.0037268088365302976 - timing_per_token_ms/gen:0.15378022024009916 - timing_per_token_ms/update_actor:0.10264383003471808 - perf/total_num_tokens:149198 - perf/time_per_step:30.382415380328894 - perf/throughput:613.8336852597567 (TaskRunner pid=2026458) Training Progress: 86%|████████▌ | 86/100 [1:00:19<07:46, 33.35s/it] (TaskRunner pid=2026458) (TaskRunner pid=2026458) step:87 - global_seqlen/min:16648 - global_seqlen/max:20097 - global_seqlen/minmax_diff:3449 - global_seqlen/balanced_min:18341 - global_seqlen/balanced_max:18344 - global_seqlen/mean:18342.375 - actor/entropy:0.24005606770515442 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44461295008659363 - training/rollout_probs_diff_mean:0.005454737227410078 - training/rollout_probs_diff_std:0.014133700169622898 - training/rollout_actor_probs_pearson_corr:0.998093843460083 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6796875 - self_distillation/correct_sample_fraction:0.53125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6796875 - rollout_corr/rollout_is_mean:0.9998968243598938 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.625958146993071e-05 - rollout_corr/rollout_is_max:1.91322922706604 - rollout_corr/rollout_is_min:0.3312951922416687 - rollout_corr/rollout_is_std:0.0426471121609211 - rollout_corr/rollout_is_eff_sample_size:0.998184050703313 - rollout_corr/rollout_is_seq_mean:0.999923050403595 - rollout_corr/rollout_is_seq_std:0.0029033836908638477 - rollout_corr/rollout_is_seq_max:1.0088640451431274 - rollout_corr/rollout_is_seq_min:0.990547776222229 - rollout_corr/rollout_is_seq_max_deviation:0.009452223777770996 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2908267816528678) - rollout_corr/training_log_ppl:np.float64(0.25219142163405195) - rollout_corr/kl:np.float64(0.0013515868227500505) - rollout_corr/k3_kl:np.float64(0.001383622248795291) - rollout_corr/rollout_ppl:np.float64(1.2890665112063289) - rollout_corr/rollout_log_ppl:np.float64(0.25083983279182576) - rollout_corr/log_ppl_diff:np.float64(0.0013515888422261924) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028635623457375914) - rollout_corr/log_ppl_diff_max:np.float64(0.012585937976837158) - rollout_corr/log_ppl_diff_min:np.float64(-0.009711101651191711) - rollout_corr/ppl_ratio:np.float64(1.0013586606364697) - rollout_corr/chi2_token:np.float64(0.0029732428956776857) - rollout_corr/chi2_seq:np.float64(0.6213931296952069) - actor/pg_loss:np.float64(0.005519538884982467) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011970178129558917) - actor/ppo_kl:np.float64(0.00032003364054844496) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53125) - self_distillation/token_reweight_w_mean:np.float64(104019.92419685167) - self_distillation/token_reweight_w_std:np.float64(1491533.3107083726) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000025857705623) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07910635319421999) - self_distillation/empty_target_batch:np.float64(0.3203125) - actor/grad_norm:np.float64(0.44405730813741684) - perf/mfu/actor:np.float64(0.03427082030528879) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.2020378112793) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:1 - critic/score/mean:0.53125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009922783821821213 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009922783821821213 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:277.73046875 - response_length/max:642.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:277.73046875 - response_length_non_aborted/max:642.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.46875 - prompt_length/max:845.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011511892080307007 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3540627993643284) - timing_s/agent_loop/generate_sequences/max:np.float64(5.077815180644393) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9762204922371893) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.077815180644393) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:172 - timing_s/agent_loop/slowest/response_length:642 - timing_s/gen:11.154959298670292 - timing_s/reward:0.057516856119036674 - timing_s/old_log_prob:2.265875956043601 - timing_s/adv:0.5384714491665363 - timing_s/update_actor:15.248491061851382 - timing_s/step:29.354633087292314 - timing_s/stop_profile:0.0001263357698917389 - timing_per_token_ms/adv:0.003669586470989555 - timing_per_token_ms/gen:0.15689333603384426 - timing_per_token_ms/update_actor:0.10391573516141846 - perf/total_num_tokens:146739 - perf/time_per_step:29.354633087292314 - perf/throughput:624.8545142926844 (TaskRunner pid=2026458) Training Progress: 87%|████████▋ | 87/100 [1:00:48<06:58, 32.18s/it] (TaskRunner pid=2026458) step:88 - global_seqlen/min:14981 - global_seqlen/max:20248 - global_seqlen/minmax_diff:5267 - global_seqlen/balanced_min:17171 - global_seqlen/balanced_max:17176 - global_seqlen/mean:17174.125 - actor/entropy:0.21586236357688904 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5279986262321472 - training/rollout_probs_diff_mean:0.005149980541318655 - training/rollout_probs_diff_std:0.014039847999811172 - training/rollout_actor_probs_pearson_corr:0.9979734420776367 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.57421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:0.9998298287391663 - rollout_corr/rollout_is_ratio_fraction_high:2.9164297302486375e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.4582148651243187e-05 - rollout_corr/rollout_is_max:2.4087512493133545 - rollout_corr/rollout_is_min:0.3157358169555664 - rollout_corr/rollout_is_std:0.04135211929678917 - rollout_corr/rollout_is_eff_sample_size:0.998292149229813 - rollout_corr/rollout_is_seq_mean:0.9997440576553345 - rollout_corr/rollout_is_seq_std:0.0026299559976905584 - rollout_corr/rollout_is_seq_max:1.0093685388565063 - rollout_corr/rollout_is_seq_min:0.9928037524223328 - rollout_corr/rollout_is_seq_max_deviation:0.009368538856506348 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.263124991208315) - rollout_corr/training_log_ppl:np.float64(0.23035154843819328) - rollout_corr/kl:np.float64(0.0012507079140435806) - rollout_corr/k3_kl:np.float64(0.0012822421521718752) - rollout_corr/rollout_ppl:np.float64(1.2614940297789872) - rollout_corr/rollout_log_ppl:np.float64(0.22910083959868643) - rollout_corr/log_ppl_diff:np.float64(0.0012507088395068422) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027966901106992736) - rollout_corr/log_ppl_diff_max:np.float64(0.014727145433425903) - rollout_corr/log_ppl_diff_min:np.float64(-0.014272093772888184) - rollout_corr/ppl_ratio:np.float64(1.0012579679023474) - rollout_corr/chi2_token:np.float64(0.002700346987694502) - rollout_corr/chi2_seq:np.float64(0.8957935450598598) - actor/pg_loss:np.float64(0.0045979529386386275) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008685116490596556) - actor/ppo_kl:np.float64(0.00010619230280539682) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.57421875) - self_distillation/token_reweight_w_mean:np.float64(126031.5825594815) - self_distillation/token_reweight_w_std:np.float64(1914441.9060592402) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010992304887623) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07210871722782031) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.4531179219484329) - perf/mfu/actor:np.float64(0.032191482757026275) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.15955352783203) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:1 - critic/score/mean:0.57421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.57421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0014600376598536968 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0014600376598536968 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:267.87890625 - response_length/max:807.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:267.87890625 - response_length_non_aborted/max:807.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.8125 - prompt_length/max:482.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012998655438423157 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3904380090534687) - timing_s/agent_loop/generate_sequences/max:np.float64(5.8301611468195915) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8218457526090788) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.8301611468195915) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:236 - timing_s/agent_loop/slowest/response_length:807 - timing_s/gen:11.686277892440557 - timing_s/reward:0.054150139912962914 - timing_s/old_log_prob:2.385304968804121 - timing_s/adv:0.4924223218113184 - timing_s/update_actor:15.133386766538024 - timing_s/step:29.843838276341558 - timing_s/stop_profile:5.164928734302521e-05 - timing_per_token_ms/adv:0.003584042286079483 - timing_per_token_ms/gen:0.170411040034422 - timing_per_token_ms/update_actor:0.11014670883187662 - perf/total_num_tokens:137393 - perf/time_per_step:29.843838276341558 - perf/throughput:575.466360626094 (TaskRunner pid=2026458) Training Progress: 88%|████████▊ | 88/100 [1:01:18<06:17, 31.49s/it] (TaskRunner pid=2026458) step:89 - global_seqlen/min:16152 - global_seqlen/max:22412 - global_seqlen/minmax_diff:6260 - global_seqlen/balanced_min:18887 - global_seqlen/balanced_max:18895 - global_seqlen/mean:18893.375 - actor/entropy:0.22386007010936737 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2151239514350891 - training/rollout_probs_diff_mean:0.005173899233341217 - training/rollout_probs_diff_std:0.013607512228190899 - training/rollout_actor_probs_pearson_corr:0.9981110692024231 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.0000429153442383 - rollout_corr/rollout_is_ratio_fraction_high:2.6122277631657198e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.83668365329504e-05 - rollout_corr/rollout_is_max:2.51365327835083 - rollout_corr/rollout_is_min:0.1054598018527031 - rollout_corr/rollout_is_std:0.04179442301392555 - rollout_corr/rollout_is_eff_sample_size:0.9982565097660324 - rollout_corr/rollout_is_seq_mean:1.0000097751617432 - rollout_corr/rollout_is_seq_std:0.0027215098962187767 - rollout_corr/rollout_is_seq_max:1.0083972215652466 - rollout_corr/rollout_is_seq_min:0.9915516376495361 - rollout_corr/rollout_is_seq_max_deviation:0.008448362350463867 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2823212435469031) - rollout_corr/training_log_ppl:np.float64(0.24520724279864226) - rollout_corr/kl:np.float64(0.0008697826999508607) - rollout_corr/k3_kl:np.float64(0.0010809666277964425) - rollout_corr/rollout_ppl:np.float64(1.2811899655498564) - rollout_corr/rollout_log_ppl:np.float64(0.24433745855640154) - rollout_corr/log_ppl_diff:np.float64(0.000869784242240712) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022608327853959054) - rollout_corr/log_ppl_diff_max:np.float64(0.010239839553833008) - rollout_corr/log_ppl_diff_min:np.float64(-0.0071149468421936035) - rollout_corr/ppl_ratio:np.float64(1.0008741298224777) - rollout_corr/chi2_token:np.float64(0.0025997732300311327) - rollout_corr/chi2_seq:np.float64(0.5740627928171307) - actor/pg_loss:np.float64(0.0036603367188945413) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008643267137813382) - actor/ppo_kl:np.float64(-9.36468737013918e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_w_mean:np.float64(95742.81214469112) - self_distillation/token_reweight_w_std:np.float64(1397763.4353546314) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014505761209875) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07612263548071496) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.43359871953725815) - perf/mfu/actor:np.float64(0.035258064824575705) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.1541633605957) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:1 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00761464424431324 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00761464424431324 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:299.07421875 - response_length/max:1025.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:299.07421875 - response_length_non_aborted/max:1025.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.34375 - prompt_length/max:720.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.2774325013160706e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3900669310241938) - timing_s/agent_loop/generate_sequences/max:np.float64(7.259778779000044) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.168158708969713) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.259778779000044) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:1025 - timing_s/gen:12.976460268720984 - timing_s/reward:0.055463384836912155 - timing_s/old_log_prob:2.4733975417912006 - timing_s/adv:0.5145350992679596 - timing_s/update_actor:15.292766243219376 - timing_s/step:31.39907766878605 - timing_s/stop_profile:4.6625733375549316e-05 - timing_per_token_ms/adv:0.003404203188074918 - timing_per_token_ms/gen:0.16948735379649418 - timing_per_token_ms/update_actor:0.10117809975202535 - perf/total_num_tokens:151147 - perf/time_per_step:31.39907766878605 - perf/throughput:601.7175153772744 (TaskRunner pid=2026458) Training Progress: 89%|████████▉ | 89/100 [1:01:50<05:46, 31.47s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which of the given options is the correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". The product contains a structure with a chlorine atom, a trifluoromethyl group (C(F)(F)F), and a specific aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A contains a structure with a chlorine atom, a trifluoromethyl group, and the correct aromatic ring structure. The formula is "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", which matches the required components of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B contains a chlorine atom but does not have the correct trifluoromethyl group or the correct aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C contains a trifluoromethyl group but lacks the correct chlorine atom and the correct aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D contains a trifluoromethyl group but lacks the correct chlorine atom and the correct aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the correct answer is Option A, as it contains all the necessary components of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80/actor (TaskRunner pid=2026458) step:90 - global_seqlen/min:14894 - global_seqlen/max:19933 - global_seqlen/minmax_diff:5039 - global_seqlen/balanced_min:18191 - global_seqlen/balanced_max:18194 - global_seqlen/mean:18192.75 - actor/entropy:0.21710193157196045 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3145221471786499 - training/rollout_probs_diff_mean:0.004972291644662619 - training/rollout_probs_diff_std:0.013361528515815735 - training/rollout_actor_probs_pearson_corr:0.9982044100761414 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.53515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:1.0000026226043701 - rollout_corr/rollout_is_ratio_fraction_high:2.6879552024183795e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.0319329855265096e-05 - rollout_corr/rollout_is_max:3.3088538646698 - rollout_corr/rollout_is_min:0.46167826652526855 - rollout_corr/rollout_is_std:0.041136063635349274 - rollout_corr/rollout_is_eff_sample_size:0.9983105639498434 - rollout_corr/rollout_is_seq_mean:1.0000332593917847 - rollout_corr/rollout_is_seq_std:0.0027625819202512503 - rollout_corr/rollout_is_seq_max:1.0086408853530884 - rollout_corr/rollout_is_seq_min:0.9904849529266357 - rollout_corr/rollout_is_seq_max_deviation:0.009515047073364258 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2710709455423057) - rollout_corr/training_log_ppl:np.float64(0.23564479569904506) - rollout_corr/kl:np.float64(0.0014168473381213076) - rollout_corr/k3_kl:np.float64(0.0014046167735841664) - rollout_corr/rollout_ppl:np.float64(1.2692075530067086) - rollout_corr/rollout_log_ppl:np.float64(0.2342279483564198) - rollout_corr/log_ppl_diff:np.float64(0.0014168473426252604) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028690413455478847) - rollout_corr/log_ppl_diff_max:np.float64(0.03278002142906189) - rollout_corr/log_ppl_diff_min:np.float64(-0.008600443601608276) - rollout_corr/ppl_ratio:np.float64(1.0014270655810833) - rollout_corr/chi2_token:np.float64(0.002751560416072607) - rollout_corr/chi2_seq:np.float64(0.7767476551234722) - actor/pg_loss:np.float64(0.0019601467065513134) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009453206475882325) - actor/ppo_kl:np.float64(0.0005355499455532708) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53515625) - self_distillation/token_reweight_w_mean:np.float64(99385.55734024942) - self_distillation/token_reweight_w_std:np.float64(1449312.0253919773) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012841576244682) - self_distillation/token_reweight_w_clip_frac:np.float64(0.045599023724207655) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.30510950461030006) - perf/mfu/actor:np.float64(0.03390671346645702) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.11214065551758) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6821428571428572) - val-aux/sciknoweval/reward/std@16:np.float64(0.1287410217939347) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7336095238095238) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.0988808144488825) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6304571428571429) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.11689339649607558) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6818380952380951) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.127482055226586) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.771042857142857) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.06672262251716281) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5835476190476191) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.09690360977896326) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6921333333333333) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.10110972005167294) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7954380952380954) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.04301822225190089) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5451904761904762) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07359385049232874) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6976380952380952) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07254164422675807) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8122666666666666) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.027241398706241732) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.5151047619047618) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05053109084853488) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7015285714285714) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.05338104590757411) - val-aux/sciknoweval/score/mean@16:np.float64(0.6821428571428572) - val-aux/sciknoweval/score/std@16:np.float64(0.1287410217939347) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7336095238095238) - val-aux/sciknoweval/score/best@2/std:np.float64(0.0988808144488825) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6304571428571429) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.11689339649607558) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6818380952380951) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.127482055226586) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.771042857142857) - val-aux/sciknoweval/score/best@4/std:np.float64(0.06672262251716281) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5835476190476191) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.09690360977896326) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6921333333333333) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.10110972005167294) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7954380952380954) - val-aux/sciknoweval/score/best@8/std:np.float64(0.04301822225190089) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5451904761904762) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07359385049232874) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6976380952380952) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07254164422675807) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8122666666666666) - val-aux/sciknoweval/score/best@16/std:np.float64(0.027241398706241732) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.5151047619047618) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05053109084853488) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7015285714285714) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.05338104590757411) - val-core/sciknoweval/acc/mean@16:np.float64(0.6821428571428572) - val-aux/sciknoweval/acc/std@16:np.float64(0.1287410217939347) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7336095238095238) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.0988808144488825) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6304571428571429) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.11689339649607558) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6818380952380951) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.127482055226586) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.771042857142857) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.06672262251716281) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5835476190476191) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.09690360977896326) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6921333333333333) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.10110972005167294) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7954380952380954) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.04301822225190089) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5451904761904762) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07359385049232874) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6976380952380952) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07254164422675807) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8122666666666666) - val-core/sciknoweval/acc/best@16/std:np.float64(0.027241398706241732) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.5151047619047618) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05053109084853488) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7015285714285714) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.05338104590757411) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.999702380952381) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0011526736149426837) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999714285714286) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0003677473252630062) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9995190476190476) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.001434898686445427) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9997809523809524) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0009975480142912052) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9990666666666667) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0018903262505010432) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9999142857142858) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0006331005658902953) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9981523809523809) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0023204503433425093) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9999714285714286) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0003677473252630062) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9970095238095239) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0023016113846578) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:1 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002840832807123661 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002840832807123661 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:290.6484375 - response_length/max:721.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:290.6484375 - response_length_non_aborted/max:721.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.875 - prompt_length/max:621.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.825903594493866e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7425885982811451) - timing_s/agent_loop/generate_sequences/max:np.float64(5.7153176963329315) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0369765825307695) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.7153176963329315) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:721 - timing_s/gen:11.199985072016716 - timing_s/reward:0.05536814592778683 - timing_s/old_log_prob:2.218005755916238 - timing_s/adv:0.4808800220489502 - timing_s/update_actor:15.297142412513494 - timing_s/step:29.337590511888266 - timing_s/testing:111.78063779324293 - timing_s/save_checkpoint:6.6935704834759235 - timing_s/stop_profile:0.00011844001710414886 - timing_per_token_ms/adv:0.003304063583357039 - timing_per_token_ms/gen:0.15052529462700207 - timing_per_token_ms/update_actor:0.10510465990925984 - perf/total_num_tokens:145542 - perf/time_per_step:29.337590511888266 - perf/throughput:620.1173880529786 (TaskRunner pid=2026458) Training Progress: 90%|█████████ | 90/100 [1:04:17<11:03, 66.39s/it] (TaskRunner pid=2026458) step:91 - global_seqlen/min:15865 - global_seqlen/max:25244 - global_seqlen/minmax_diff:9379 - global_seqlen/balanced_min:18811 - global_seqlen/balanced_max:18818 - global_seqlen/mean:18815.125 - actor/entropy:0.2314702272415161 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21055352687835693 - training/rollout_probs_diff_mean:0.005051769316196442 - training/rollout_probs_diff_std:0.013311025686562061 - training/rollout_actor_probs_pearson_corr:0.9982390999794006 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.60546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9999205470085144 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.47056513116695e-05 - rollout_corr/rollout_is_max:1.7587093114852905 - rollout_corr/rollout_is_min:0.23516227304935455 - rollout_corr/rollout_is_std:0.039755210280418396 - rollout_corr/rollout_is_eff_sample_size:0.9984218983709217 - rollout_corr/rollout_is_seq_mean:0.9999246001243591 - rollout_corr/rollout_is_seq_std:0.002499330323189497 - rollout_corr/rollout_is_seq_max:1.0098474025726318 - rollout_corr/rollout_is_seq_min:0.9917200207710266 - rollout_corr/rollout_is_seq_max_deviation:0.009847402572631836 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2789378436282277) - rollout_corr/training_log_ppl:np.float64(0.24201013351557776) - rollout_corr/kl:np.float64(0.0009981181076650358) - rollout_corr/k3_kl:np.float64(0.0010161043154539584) - rollout_corr/rollout_ppl:np.float64(1.2776579768396914) - rollout_corr/rollout_log_ppl:np.float64(0.2410120142158121) - rollout_corr/log_ppl_diff:np.float64(0.0009981192997656763) - rollout_corr/log_ppl_abs_diff:np.float64(0.002134583191946149) - rollout_corr/log_ppl_diff_max:np.float64(0.00839272141456604) - rollout_corr/log_ppl_diff_min:np.float64(-0.010266929864883423) - rollout_corr/ppl_ratio:np.float64(1.0010020262561738) - rollout_corr/chi2_token:np.float64(0.0021203176584094763) - rollout_corr/chi2_seq:np.float64(0.651163290720433) - actor/pg_loss:np.float64(0.002993874717503786) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000568103197565506) - actor/ppo_kl:np.float64(6.333214638765128e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.60546875) - self_distillation/token_reweight_w_mean:np.float64(82354.48971123807) - self_distillation/token_reweight_w_std:np.float64(1252927.9291606762) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009026953484863) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06239277429995127) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.33179403841495514) - perf/mfu/actor:np.float64(0.034982824171721626) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.52531814575195) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:1 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002352050505578518 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002352050505578518 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:301.84765625 - response_length/max:674.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:301.84765625 - response_length_non_aborted/max:674.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.125 - prompt_length/max:1009.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.271526217460632e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2008809298276901) - timing_s/agent_loop/generate_sequences/max:np.float64(5.367214092984796) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2526293304326828) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.367214092984796) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:208 - timing_s/agent_loop/slowest/response_length:645 - timing_s/gen:11.187385501340032 - timing_s/reward:0.05164775624871254 - timing_s/old_log_prob:2.4035099502652884 - timing_s/adv:0.5490099098533392 - timing_s/update_actor:15.261925529688597 - timing_s/step:29.56569225527346 - timing_s/stop_profile:0.00012163445353507996 - timing_per_token_ms/adv:0.0036473974385855742 - timing_per_token_ms/gen:0.14477741903821556 - timing_per_token_ms/update_actor:0.10139399505509927 - perf/total_num_tokens:150521 - perf/time_per_step:29.56569225527346 - perf/throughput:636.3837124985313 (TaskRunner pid=2026458) Training Progress: 91%|█████████ | 91/100 [1:04:47<08:18, 55.36s/it] (TaskRunner pid=2026458) step:92 - global_seqlen/min:16281 - global_seqlen/max:21475 - global_seqlen/minmax_diff:5194 - global_seqlen/balanced_min:18650 - global_seqlen/balanced_max:18651 - global_seqlen/mean:18650.625 - actor/entropy:0.22619383037090302 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967529773712158 - training/rollout_probs_diff_mean:0.005189774092286825 - training/rollout_probs_diff_std:0.013963738456368446 - training/rollout_actor_probs_pearson_corr:0.9980257153511047 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:0.9999138116836548 - rollout_corr/rollout_is_ratio_fraction_high:1.3226638657215517e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011903974518645555 - rollout_corr/rollout_is_max:2.0864930152893066 - rollout_corr/rollout_is_min:0.0921701192855835 - rollout_corr/rollout_is_std:0.04230894520878792 - rollout_corr/rollout_is_eff_sample_size:0.9982129142824776 - rollout_corr/rollout_is_seq_mean:0.9999446272850037 - rollout_corr/rollout_is_seq_std:0.0027264179661870003 - rollout_corr/rollout_is_seq_max:1.0081113576889038 - rollout_corr/rollout_is_seq_min:0.9917219281196594 - rollout_corr/rollout_is_seq_max_deviation:0.008278071880340576 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2724861456081271) - rollout_corr/training_log_ppl:np.float64(0.2372018521418795) - rollout_corr/kl:np.float64(0.0009739822945320498) - rollout_corr/k3_kl:np.float64(0.0010123744499423992) - rollout_corr/rollout_ppl:np.float64(1.2711997576989233) - rollout_corr/rollout_log_ppl:np.float64(0.23622786854684819) - rollout_corr/log_ppl_diff:np.float64(0.0009739835950313136) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024230604904005304) - rollout_corr/log_ppl_diff_max:np.float64(0.012229472398757935) - rollout_corr/log_ppl_diff_min:np.float64(-0.010651081800460815) - rollout_corr/ppl_ratio:np.float64(1.0009791306219995) - rollout_corr/chi2_token:np.float64(0.0021006297320127487) - rollout_corr/chi2_seq:np.float64(0.7930810896214098) - actor/pg_loss:np.float64(0.002591841504909098) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004880208070972003) - actor/ppo_kl:np.float64(-2.9975729340492308e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_w_mean:np.float64(72610.33176031034) - self_distillation/token_reweight_w_std:np.float64(1092734.639293266) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988934537395835) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06326231267303228) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.3366621397435665) - perf/mfu/actor:np.float64(0.03433711138530011) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.60525131225586) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:1 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006365319713950157 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006365319713950157 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:295.33203125 - response_length/max:674.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:295.33203125 - response_length_non_aborted/max:674.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:287.5 - prompt_length/max:436.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.356330752372742e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4242578689008951) - timing_s/agent_loop/generate_sequences/max:np.float64(5.455314517021179) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2662257757328916) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.455314517021179) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:193 - timing_s/agent_loop/slowest/response_length:674 - timing_s/gen:11.507964111864567 - timing_s/reward:0.056888729333877563 - timing_s/old_log_prob:2.3034613393247128 - timing_s/adv:0.4925750717520714 - timing_s/update_actor:15.52449726127088 - timing_s/step:29.975734889507294 - timing_s/stop_profile:0.000132659450173378 - timing_per_token_ms/adv:0.0033013308652663875 - timing_per_token_ms/gen:0.15221168060134338 - timing_per_token_ms/update_actor:0.10404810335626072 - perf/total_num_tokens:149205 - perf/time_per_step:29.975734889507294 - perf/throughput:622.1907509106129 (TaskRunner pid=2026458) Training Progress: 92%|█████████▏| 92/100 [1:05:17<06:22, 47.76s/it] (TaskRunner pid=2026458) step:93 - global_seqlen/min:15058 - global_seqlen/max:22608 - global_seqlen/minmax_diff:7550 - global_seqlen/balanced_min:18025 - global_seqlen/balanced_max:18028 - global_seqlen/mean:18026.625 - actor/entropy:0.20888566970825195 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24053943157196045 - training/rollout_probs_diff_mean:0.004918403923511505 - training/rollout_probs_diff_std:0.013119952753186226 - training/rollout_actor_probs_pearson_corr:0.9981909394264221 - self_distillation/success_group_fraction:1.0 - self_distillation/success_sample_fraction:0.98828125 - self_distillation/correct_sample_fraction:0.87890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.98828125 - rollout_corr/rollout_is_mean:0.9998151659965515 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.263365528662689e-05 - rollout_corr/rollout_is_max:1.6370371580123901 - rollout_corr/rollout_is_min:0.2820625901222229 - rollout_corr/rollout_is_std:0.03976345434784889 - rollout_corr/rollout_is_eff_sample_size:0.9984207100389592 - rollout_corr/rollout_is_seq_mean:0.9998314380645752 - rollout_corr/rollout_is_seq_std:0.0023552256170660257 - rollout_corr/rollout_is_seq_max:1.0079485177993774 - rollout_corr/rollout_is_seq_min:0.9893273115158081 - rollout_corr/rollout_is_seq_max_deviation:0.010672688484191895 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2420122399926186) - rollout_corr/training_log_ppl:np.float64(0.2130839267920237) - rollout_corr/kl:np.float64(0.0009293865148691793) - rollout_corr/k3_kl:np.float64(0.000889873810649533) - rollout_corr/rollout_ppl:np.float64(1.2408293401822448) - rollout_corr/rollout_log_ppl:np.float64(0.2121545401896583) - rollout_corr/log_ppl_diff:np.float64(0.0009293866023654118) - rollout_corr/log_ppl_abs_diff:np.float64(0.00208455762185622) - rollout_corr/log_ppl_diff_max:np.float64(0.008339852094650269) - rollout_corr/log_ppl_diff_min:np.float64(-0.005493566393852234) - rollout_corr/ppl_ratio:np.float64(1.0009328972082585) - rollout_corr/chi2_token:np.float64(0.00168294133618474) - rollout_corr/chi2_seq:np.float64(0.3678914944175631) - actor/pg_loss:np.float64(0.00139771425165236) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002747084317888948) - actor/ppo_kl:np.float64(-7.94472919167788e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.98828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.98828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.87890625) - self_distillation/token_reweight_w_mean:np.float64(72035.54537807149) - self_distillation/token_reweight_w_std:np.float64(1075218.813784783) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001510763540864) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03214944276260212) - self_distillation/empty_target_batch:np.float64(0.01171875) - actor/grad_norm:np.float64(0.2579457201063633) - perf/mfu/actor:np.float64(0.03364477703539987) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.39699172973633) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:1 - critic/score/mean:0.87890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.87890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036366567946970463 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:0.0036366567946970463 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:296.86328125 - response_length/max:640.0 - response_length/min:140.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:296.86328125 - response_length_non_aborted/max:640.0 - response_length_non_aborted/min:140.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:709.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017446838319301605 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6789815910160542) - timing_s/agent_loop/generate_sequences/max:np.float64(5.267441829666495) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.250441880845756) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.267441829666495) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:308 - timing_s/agent_loop/slowest/response_length:640 - timing_s/gen:11.287975236773491 - timing_s/reward:0.0554285254329443 - timing_s/old_log_prob:2.2384933289140463 - timing_s/adv:0.5122805014252663 - timing_s/update_actor:15.286979779601097 - timing_s/step:29.47672649472952 - timing_s/stop_profile:0.00014898553490638733 - timing_per_token_ms/adv:0.003552249113639313 - timing_per_token_ms/gen:0.14853185305700872 - timing_per_token_ms/update_actor:0.106002786015138 - perf/total_num_tokens:144213 - perf/time_per_step:29.47672649472952 - perf/throughput:611.5545090538865 (TaskRunner pid=2026458) Training Progress: 93%|█████████▎| 93/100 [1:05:47<04:56, 42.29s/it] (TaskRunner pid=2026458) step:94 - global_seqlen/min:15786 - global_seqlen/max:20010 - global_seqlen/minmax_diff:4224 - global_seqlen/balanced_min:17735 - global_seqlen/balanced_max:17737 - global_seqlen/mean:17736.0 - actor/entropy:0.22735926508903503 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311311483383179 - training/rollout_probs_diff_mean:0.005040409974753857 - training/rollout_probs_diff_std:0.013486634008586407 - training/rollout_actor_probs_pearson_corr:0.9981728792190552 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.73046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:0.9998102784156799 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0001036699177348055 - rollout_corr/rollout_is_max:1.7523070573806763 - rollout_corr/rollout_is_min:0.05793754756450653 - rollout_corr/rollout_is_std:0.03992203250527382 - rollout_corr/rollout_is_eff_sample_size:0.9984081138940653 - rollout_corr/rollout_is_seq_mean:0.9999222159385681 - rollout_corr/rollout_is_seq_std:0.002507504541426897 - rollout_corr/rollout_is_seq_max:1.0082566738128662 - rollout_corr/rollout_is_seq_min:0.9931005835533142 - rollout_corr/rollout_is_seq_max_deviation:0.008256673812866211 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2725239442661405) - rollout_corr/training_log_ppl:np.float64(0.23770636528206524) - rollout_corr/kl:np.float64(0.001132996877289294) - rollout_corr/k3_kl:np.float64(0.0012387144401486694) - rollout_corr/rollout_ppl:np.float64(1.2710481900721788) - rollout_corr/rollout_log_ppl:np.float64(0.23657336641917937) - rollout_corr/log_ppl_diff:np.float64(0.00113299886288587) - rollout_corr/log_ppl_abs_diff:np.float64(0.002540186673286371) - rollout_corr/log_ppl_diff_max:np.float64(0.02026122808456421) - rollout_corr/log_ppl_diff_min:np.float64(-0.015430659055709839) - rollout_corr/ppl_ratio:np.float64(1.0011393770109862) - rollout_corr/chi2_token:np.float64(0.002516603097319603) - rollout_corr/chi2_seq:np.float64(0.9786048375535756) - actor/pg_loss:np.float64(0.002404728322289884) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001062956328951259) - actor/ppo_kl:np.float64(0.0001939034154609942) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_w_mean:np.float64(96479.46470297547) - self_distillation/token_reweight_w_std:np.float64(1265708.629157153) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9990413470659405) - self_distillation/token_reweight_w_clip_frac:np.float64(0.054301650350680575) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.35001390427351) - perf/mfu/actor:np.float64(0.032954674991088666) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.4532012939453) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:1 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00508630508556962 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00508630508556962 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:301.4375 - response_length/max:631.0 - response_length/min:135.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:301.4375 - response_length_non_aborted/max:631.0 - response_length_non_aborted/min:135.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:252.8125 - prompt_length/max:384.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017067231237888336 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4371750131249428) - timing_s/agent_loop/generate_sequences/max:np.float64(5.477192688733339) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2628613221095293) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.477192688733339) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:221 - timing_s/agent_loop/slowest/response_length:631 - timing_s/gen:11.104732798412442 - timing_s/reward:0.05309521034359932 - timing_s/old_log_prob:2.3046134896576405 - timing_s/adv:0.4992757458239794 - timing_s/update_actor:15.339771771803498 - timing_s/step:29.391625313088298 - timing_s/stop_profile:0.000131336972117424 - timing_per_token_ms/adv:0.0035188017719890293 - timing_per_token_ms/gen:0.14390333815068995 - timing_per_token_ms/update_actor:0.10811183307822718 - perf/total_num_tokens:141888 - perf/time_per_step:29.391625313088298 - perf/throughput:603.4371971971906 (TaskRunner pid=2026458) Training Progress: 94%|█████████▍| 94/100 [1:06:16<03:50, 38.44s/it] (TaskRunner pid=2026458) step:95 - global_seqlen/min:14931 - global_seqlen/max:21336 - global_seqlen/minmax_diff:6405 - global_seqlen/balanced_min:17803 - global_seqlen/balanced_max:17812 - global_seqlen/mean:17810.75 - actor/entropy:0.22719933092594147 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3546398878097534 - training/rollout_probs_diff_mean:0.004889350384473801 - training/rollout_probs_diff_std:0.012968857772648335 - training/rollout_actor_probs_pearson_corr:0.9983217716217041 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.6640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:0.9999182224273682 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.3313806448422838e-05 - rollout_corr/rollout_is_max:1.646693468093872 - rollout_corr/rollout_is_min:0.012675890699028969 - rollout_corr/rollout_is_std:0.03914937749505043 - rollout_corr/rollout_is_eff_sample_size:0.9984694339690914 - rollout_corr/rollout_is_seq_mean:0.9998860955238342 - rollout_corr/rollout_is_seq_std:0.002519140485674143 - rollout_corr/rollout_is_seq_max:1.0078048706054688 - rollout_corr/rollout_is_seq_min:0.993129551410675 - rollout_corr/rollout_is_seq_max_deviation:0.00780487060546875 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.269720992539078) - rollout_corr/training_log_ppl:np.float64(0.23537427955307066) - rollout_corr/kl:np.float64(0.000860856562256096) - rollout_corr/k3_kl:np.float64(0.0010757347620256041) - rollout_corr/rollout_ppl:np.float64(1.2686140895821154) - rollout_corr/rollout_log_ppl:np.float64(0.2345134206698276) - rollout_corr/log_ppl_diff:np.float64(0.0008608588832430542) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024572820402681828) - rollout_corr/log_ppl_diff_max:np.float64(0.022655338048934937) - rollout_corr/log_ppl_diff_min:np.float64(-0.008849263191223145) - rollout_corr/ppl_ratio:np.float64(1.0008665048517287) - rollout_corr/chi2_token:np.float64(0.002582071116194129) - rollout_corr/chi2_seq:np.float64(1.1929384064860642) - actor/pg_loss:np.float64(0.0030269279377534986) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001186676310680923) - actor/ppo_kl:np.float64(-0.00011882417348374474) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6640625) - self_distillation/token_reweight_w_mean:np.float64(142327.77909193072) - self_distillation/token_reweight_w_std:np.float64(1811469.8086368854) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001867031678557) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07137267180951312) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.4086311012506485) - perf/mfu/actor:np.float64(0.03299815181672398) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.32865142822266) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007821861072443426 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0007821861072443426 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:293.3984375 - response_length/max:742.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:293.3984375 - response_length_non_aborted/max:742.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.1875 - prompt_length/max:646.0 - prompt_length/min:159.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011094845831394196 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2119843941181898) - timing_s/agent_loop/generate_sequences/max:np.float64(5.851273892447352) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2298050526515) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.851273892447352) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:411 - timing_s/agent_loop/slowest/response_length:742 - timing_s/gen:11.692811168730259 - timing_s/reward:0.051967186853289604 - timing_s/old_log_prob:2.230885935947299 - timing_s/adv:0.5060996375977993 - timing_s/update_actor:15.389210676774383 - timing_s/step:29.97092897631228 - timing_s/stop_profile:0.0001416075974702835 - timing_per_token_ms/adv:0.003551925365283602 - timing_per_token_ms/gen:0.15567582437398828 - timing_per_token_ms/update_actor:0.10800507191425392 - perf/total_num_tokens:142486 - perf/time_per_step:29.97092897631228 - perf/throughput:594.2675321834984 (TaskRunner pid=2026458) Training Progress: 95%|█████████▌| 95/100 [1:06:46<02:59, 35.91s/it] (TaskRunner pid=2026458) step:96 - global_seqlen/min:15504 - global_seqlen/max:21947 - global_seqlen/minmax_diff:6443 - global_seqlen/balanced_min:17847 - global_seqlen/balanced_max:17850 - global_seqlen/mean:17848.25 - actor/entropy:0.24302901327610016 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7704145312309265 - training/rollout_probs_diff_mean:0.0050596678629517555 - training/rollout_probs_diff_std:0.013549817726016045 - training/rollout_actor_probs_pearson_corr:0.9982749223709106 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.6015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:1.0000941753387451 - rollout_corr/rollout_is_ratio_fraction_high:4.025656744488515e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.709428271278739e-05 - rollout_corr/rollout_is_max:2.5671136379241943 - rollout_corr/rollout_is_min:0.13419120013713837 - rollout_corr/rollout_is_std:0.039693690836429596 - rollout_corr/rollout_is_eff_sample_size:0.9984272458997584 - rollout_corr/rollout_is_seq_mean:1.000090479850769 - rollout_corr/rollout_is_seq_std:0.0026221859734505415 - rollout_corr/rollout_is_seq_max:1.009491205215454 - rollout_corr/rollout_is_seq_min:0.9908090233802795 - rollout_corr/rollout_is_seq_max_deviation:0.009491205215454102 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2904391190968454) - rollout_corr/training_log_ppl:np.float64(0.2496467249293346) - rollout_corr/kl:np.float64(0.0008864650635374005) - rollout_corr/k3_kl:np.float64(0.0009545528546368587) - rollout_corr/rollout_ppl:np.float64(1.2892436645925045) - rollout_corr/rollout_log_ppl:np.float64(0.24876026091806125) - rollout_corr/log_ppl_diff:np.float64(0.0008864640112733468) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021889270428800955) - rollout_corr/log_ppl_diff_max:np.float64(0.013020426034927368) - rollout_corr/log_ppl_diff_min:np.float64(-0.006367728114128113) - rollout_corr/ppl_ratio:np.float64(1.0008906812872738) - rollout_corr/chi2_token:np.float64(0.0020458281505852938) - rollout_corr/chi2_seq:np.float64(0.6757269599474967) - actor/pg_loss:np.float64(0.0028436718275770545) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006703252311126562) - actor/ppo_kl:np.float64(0.0001225946314420412) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6015625) - self_distillation/token_reweight_w_mean:np.float64(73408.10212582699) - self_distillation/token_reweight_w_std:np.float64(1221313.2646529041) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004472548607737) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07176223069836851) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.3494473174214363) - perf/mfu/actor:np.float64(0.03336222886368918) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.36438751220703) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:1 - critic/score/mean:0.6015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0025512599386274815 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0025512599386274815 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:291.1015625 - response_length/max:636.0 - response_length/min:116.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:291.1015625 - response_length_non_aborted/max:636.0 - response_length_non_aborted/min:116.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.65625 - prompt_length/max:456.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012776069343090057 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3372939508408308) - timing_s/agent_loop/generate_sequences/max:np.float64(5.064935522153974) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.084805842998321) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.064935522153974) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:338 - timing_s/agent_loop/slowest/response_length:572 - timing_s/gen:11.018641920760274 - timing_s/reward:0.05196058563888073 - timing_s/old_log_prob:2.2254901994019747 - timing_s/adv:0.4944445975124836 - timing_s/update_actor:15.122869728133082 - timing_s/step:29.00846175290644 - timing_s/stop_profile:2.9573217034339905e-05 - timing_per_token_ms/adv:0.0034628366752516605 - timing_per_token_ms/gen:0.14785757119723403 - timing_per_token_ms/update_actor:0.10591283268761001 - perf/total_num_tokens:142786 - perf/time_per_step:29.00846175290644 - perf/throughput:615.2773681014552 (TaskRunner pid=2026458) Training Progress: 96%|█████████▌| 96/100 [1:07:15<02:15, 33.85s/it] (TaskRunner pid=2026458) step:97 - global_seqlen/min:16071 - global_seqlen/max:23461 - global_seqlen/minmax_diff:7390 - global_seqlen/balanced_min:18811 - global_seqlen/balanced_max:18815 - global_seqlen/mean:18813.875 - actor/entropy:0.2445484697818756 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4984980821609497 - training/rollout_probs_diff_mean:0.005207015201449394 - training/rollout_probs_diff_std:0.013489038683474064 - training/rollout_actor_probs_pearson_corr:0.9982696175575256 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.6484375 - self_distillation/correct_sample_fraction:0.52734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6484375 - rollout_corr/rollout_is_mean:0.9999282360076904 - rollout_corr/rollout_is_ratio_fraction_high:1.330795930698514e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.65397965349257e-05 - rollout_corr/rollout_is_max:3.3646888732910156 - rollout_corr/rollout_is_min:0.3885670304298401 - rollout_corr/rollout_is_std:0.04048614203929901 - rollout_corr/rollout_is_eff_sample_size:0.9983633169728505 - rollout_corr/rollout_is_seq_mean:0.9999846816062927 - rollout_corr/rollout_is_seq_std:0.002680700272321701 - rollout_corr/rollout_is_seq_max:1.0078043937683105 - rollout_corr/rollout_is_seq_min:0.9888970255851746 - rollout_corr/rollout_is_seq_max_deviation:0.01110297441482544 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3025756301358342) - rollout_corr/training_log_ppl:np.float64(0.2600360629003262) - rollout_corr/kl:np.float64(0.001345034299074399) - rollout_corr/k3_kl:np.float64(0.0010471967440253138) - rollout_corr/rollout_ppl:np.float64(1.3007347490638494) - rollout_corr/rollout_log_ppl:np.float64(0.2586910269019427) - rollout_corr/log_ppl_diff:np.float64(0.001345035998383537) - rollout_corr/log_ppl_abs_diff:np.float64(0.002600655221613124) - rollout_corr/log_ppl_diff_max:np.float64(0.013873368501663208) - rollout_corr/log_ppl_diff_min:np.float64(-0.008637338876724243) - rollout_corr/ppl_ratio:np.float64(1.0013508247211576) - rollout_corr/chi2_token:np.float64(0.001424239482730627) - rollout_corr/chi2_seq:np.float64(0.4958528857678175) - actor/pg_loss:np.float64(0.0023965955479070544) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000528412345374818) - actor/ppo_kl:np.float64(0.00046769405014401855) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.52734375) - self_distillation/token_reweight_w_mean:np.float64(76633.93915633648) - self_distillation/token_reweight_w_std:np.float64(1211350.6270655608) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995253349188715) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0652933964447584) - self_distillation/empty_target_batch:np.float64(0.3515625) - actor/grad_norm:np.float64(0.31651656329631805) - perf/mfu/actor:np.float64(0.03500956607334557) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.4477882385254) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:1 - critic/score/mean:0.52734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.52734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004978840239346027 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004978840239346027 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:293.52734375 - response_length/max:730.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:293.52734375 - response_length_non_aborted/max:730.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:294.40625 - prompt_length/max:900.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.985447645187378e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1220364272594452) - timing_s/agent_loop/generate_sequences/max:np.float64(5.692030295729637) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1152001094887964) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.692030295729637) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:730 - timing_s/gen:11.270145887508988 - timing_s/reward:0.05703514441847801 - timing_s/old_log_prob:2.3115832544863224 - timing_s/adv:0.5203931517899036 - timing_s/update_actor:15.463589219376445 - timing_s/step:29.711128624156117 - timing_s/stop_profile:0.00012691877782344818 - timing_per_token_ms/adv:0.0034575090976068437 - timing_per_token_ms/gen:0.14998264492379848 - timing_per_token_ms/update_actor:0.10274059184628662 - perf/total_num_tokens:150511 - perf/time_per_step:29.711128624156117 - perf/throughput:633.2265340032793 (TaskRunner pid=2026458) Training Progress: 97%|█████████▋| 97/100 [1:07:45<01:37, 32.62s/it] (TaskRunner pid=2026458) step:98 - global_seqlen/min:14853 - global_seqlen/max:23186 - global_seqlen/minmax_diff:8333 - global_seqlen/balanced_min:19016 - global_seqlen/balanced_max:19021 - global_seqlen/mean:19019.875 - actor/entropy:0.23993536829948425 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2247922420501709 - training/rollout_probs_diff_mean:0.005032053682953119 - training/rollout_probs_diff_std:0.013124685734510422 - training/rollout_actor_probs_pearson_corr:0.99836266040802 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.58203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:0.9997885227203369 - rollout_corr/rollout_is_ratio_fraction_high:2.6424617317388766e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.9636925066588446e-05 - rollout_corr/rollout_is_max:2.206674814224243 - rollout_corr/rollout_is_min:0.19827817380428314 - rollout_corr/rollout_is_std:0.04013793542981148 - rollout_corr/rollout_is_eff_sample_size:0.9983908838332385 - rollout_corr/rollout_is_seq_mean:0.9997617602348328 - rollout_corr/rollout_is_seq_std:0.0025118952617049217 - rollout_corr/rollout_is_seq_max:1.0087740421295166 - rollout_corr/rollout_is_seq_min:0.9924179315567017 - rollout_corr/rollout_is_seq_max_deviation:0.008774042129516602 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2941957069560885) - rollout_corr/training_log_ppl:np.float64(0.254570292920107) - rollout_corr/kl:np.float64(0.0012180306670934726) - rollout_corr/k3_kl:np.float64(0.000970762781207668) - rollout_corr/rollout_ppl:np.float64(1.2925921510905027) - rollout_corr/rollout_log_ppl:np.float64(0.25335226202150807) - rollout_corr/log_ppl_diff:np.float64(0.001218030898598954) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023932511976454407) - rollout_corr/log_ppl_diff_max:np.float64(0.008434802293777466) - rollout_corr/log_ppl_diff_min:np.float64(-0.005523428320884705) - rollout_corr/ppl_ratio:np.float64(1.0012223366647959) - rollout_corr/chi2_token:np.float64(0.0014376682229340076) - rollout_corr/chi2_seq:np.float64(0.6267413701862097) - actor/pg_loss:np.float64(0.003052193787880242) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006503687072836328) - actor/ppo_kl:np.float64(0.000135291372203028) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58203125) - self_distillation/token_reweight_w_mean:np.float64(107416.14729232923) - self_distillation/token_reweight_w_std:np.float64(1626133.0662139303) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999873915920034) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0583975906483829) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.36374419182538986) - perf/mfu/actor:np.float64(0.03551806757350935) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.40508651733398) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:1 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001578870927914977 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001578870927914977 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:295.65234375 - response_length/max:718.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:295.65234375 - response_length_non_aborted/max:718.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:631.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011442229151725769 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4412503708153963) - timing_s/agent_loop/generate_sequences/max:np.float64(5.591642700135708) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1361403591217822) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.591642700135708) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:211 - timing_s/agent_loop/slowest/response_length:718 - timing_s/gen:11.411643352359533 - timing_s/reward:0.05464274063706398 - timing_s/old_log_prob:2.2537942565977573 - timing_s/adv:0.5091272983700037 - timing_s/update_actor:15.217187775298953 - timing_s/step:29.54008462652564 - timing_s/stop_profile:0.00011652149260044098 - timing_per_token_ms/adv:0.003346021585118223 - timing_per_token_ms/gen:0.1507741534525022 - timing_per_token_ms/update_actor:0.1000084633528017 - perf/total_num_tokens:152159 - perf/time_per_step:29.54008462652564 - perf/throughput:643.8666388559031 (TaskRunner pid=2026458) Training Progress: 98%|█████████▊| 98/100 [1:08:14<01:03, 31.71s/it] (TaskRunner pid=2026458) step:99 - global_seqlen/min:16270 - global_seqlen/max:23777 - global_seqlen/minmax_diff:7507 - global_seqlen/balanced_min:18894 - global_seqlen/balanced_max:18898 - global_seqlen/mean:18896.125 - actor/entropy:0.23954525589942932 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5315115451812744 - training/rollout_probs_diff_mean:0.005148096010088921 - training/rollout_probs_diff_std:0.013385861180722713 - training/rollout_actor_probs_pearson_corr:0.9983036518096924 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.00002920627594 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.40447464218596e-05 - rollout_corr/rollout_is_max:1.9394363164901733 - rollout_corr/rollout_is_min:0.2951362729072571 - rollout_corr/rollout_is_std:0.04090065881609917 - rollout_corr/rollout_is_eff_sample_size:0.9983299298399158 - rollout_corr/rollout_is_seq_mean:1.0000901222229004 - rollout_corr/rollout_is_seq_std:0.0026464853435754776 - rollout_corr/rollout_is_seq_max:1.00861656665802 - rollout_corr/rollout_is_seq_min:0.9930309057235718 - rollout_corr/rollout_is_seq_max_deviation:0.00861656665802002 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2940249065868556) - rollout_corr/training_log_ppl:np.float64(0.25375668989727274) - rollout_corr/kl:np.float64(0.0008266884788987738) - rollout_corr/k3_kl:np.float64(0.0009614558624662095) - rollout_corr/rollout_ppl:np.float64(1.2929239594377577) - rollout_corr/rollout_log_ppl:np.float64(0.2529299992165761) - rollout_corr/log_ppl_diff:np.float64(0.000826690680696629) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025123875675490126) - rollout_corr/log_ppl_diff_max:np.float64(0.009278476238250732) - rollout_corr/log_ppl_diff_min:np.float64(-0.007872521877288818) - rollout_corr/ppl_ratio:np.float64(1.000831762328744) - rollout_corr/chi2_token:np.float64(0.002216932363808155) - rollout_corr/chi2_seq:np.float64(0.6580137938726693) - actor/pg_loss:np.float64(0.004063432686962187) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007904909580247477) - actor/ppo_kl:np.float64(4.468652047506794e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(131557.28193879593) - self_distillation/token_reweight_w_std:np.float64(2127779.07550747) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999837173614651) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07958601297286805) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.4794868528842926) - perf/mfu/actor:np.float64(0.035181320287522105) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.34246444702148) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003239069599658251 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003239069599658251 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:278.12890625 - response_length/max:681.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:278.12890625 - response_length_non_aborted/max:681.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.375 - prompt_length/max:836.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017566047608852386 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2292504087090492) - timing_s/agent_loop/generate_sequences/max:np.float64(5.23788345977664) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0256088213573094) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.23788345977664) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:203 - timing_s/agent_loop/slowest/response_length:681 - timing_s/gen:11.295774277299643 - timing_s/reward:0.05612022429704666 - timing_s/old_log_prob:2.242926351726055 - timing_s/adv:0.5197359751909971 - timing_s/update_actor:15.403783183544874 - timing_s/step:29.607021318748593 - timing_s/stop_profile:0.00011566095054149628 - timing_per_token_ms/adv:0.0034381121472722392 - timing_per_token_ms/gen:0.1586462869524254 - timing_per_token_ms/update_actor:0.101897764644503 - perf/total_num_tokens:151169 - perf/time_per_step:29.607021318748593 - perf/throughput:638.2312086232755 (TaskRunner pid=2026458) Training Progress: 99%|█████████▉| 99/100 [1:08:44<00:31, 31.09s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which of the given options is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". The product contains a structure with a chlorine atom, a trifluoromethyl group (C(F)(F)F), and a specific aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A: "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" contains the correct structure with a chlorine atom, a trifluoromethyl group, and the necessary aromatic ring, matching the product structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B: "O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1" contains a chlorine atom but lacks the correct trifluoromethyl group and the correct aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C: "O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1" contains a trifluoromethyl group and a chlorine atom but has an incorrect aromatic ring structure and additional functional groups not present in the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D: "O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F" contains a trifluoromethyl group but lacks the correct chlorine atom and the correct aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the only option that matches the product structure is Option A. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90/actor (TaskRunner pid=2026458) step:100 - global_seqlen/min:17070 - global_seqlen/max:22010 - global_seqlen/minmax_diff:4940 - global_seqlen/balanced_min:19621 - global_seqlen/balanced_max:19622 - global_seqlen/mean:19621.75 - actor/entropy:0.21838146448135376 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30163663625717163 - training/rollout_probs_diff_mean:0.004920980893075466 - training/rollout_probs_diff_std:0.013346283696591854 - training/rollout_actor_probs_pearson_corr:0.9981935024261475 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71875 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71875 - rollout_corr/rollout_is_mean:1.0000070333480835 - rollout_corr/rollout_is_ratio_fraction_high:1.2321644135226961e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.16082179476507e-05 - rollout_corr/rollout_is_max:2.1037662029266357 - rollout_corr/rollout_is_min:0.39213550090789795 - rollout_corr/rollout_is_std:0.04028838127851486 - rollout_corr/rollout_is_eff_sample_size:0.9983795954951445 - rollout_corr/rollout_is_seq_mean:1.0000463724136353 - rollout_corr/rollout_is_seq_std:0.002407139865681529 - rollout_corr/rollout_is_seq_max:1.0090001821517944 - rollout_corr/rollout_is_seq_min:0.9934689998626709 - rollout_corr/rollout_is_seq_max_deviation:0.009000182151794434 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2685748748481274) - rollout_corr/training_log_ppl:np.float64(0.23310650100756902) - rollout_corr/kl:np.float64(0.0009287315268036167) - rollout_corr/k3_kl:np.float64(0.001084987000297133) - rollout_corr/rollout_ppl:np.float64(1.2673596912063658) - rollout_corr/rollout_log_ppl:np.float64(0.2321777678298531) - rollout_corr/log_ppl_diff:np.float64(0.0009287331777159125) - rollout_corr/log_ppl_abs_diff:np.float64(0.002280793822137639) - rollout_corr/log_ppl_diff_max:np.float64(0.015468984842300415) - rollout_corr/log_ppl_diff_min:np.float64(-0.010868668556213379) - rollout_corr/ppl_ratio:np.float64(1.0009334608912468) - rollout_corr/chi2_token:np.float64(0.002497109118849039) - rollout_corr/chi2_seq:np.float64(1.0585334873758256) - actor/pg_loss:np.float64(0.0023272527614608407) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004233177678543143) - actor/ppo_kl:np.float64(0.00010989303795483352) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(73094.00544022233) - self_distillation/token_reweight_w_std:np.float64(1188161.9467263469) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000056762015447) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03758277394808829) - self_distillation/empty_target_batch:np.float64(0.28125) - actor/grad_norm:np.float64(0.25844827108085155) - perf/mfu/actor:np.float64(0.03632957727570209) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.29161834716797) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6895833333333333) - val-aux/sciknoweval/reward/std@16:np.float64(0.10973085235155812) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.733895238095238) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.08320072408312382) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6464904761904762) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.10098054289944847) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6907285714285715) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.10956288078402857) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7644238095238095) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.05746985351073585) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.6048666666666667) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.08537783027231041) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6991761904761905) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.08403267534041585) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7858285714285714) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.038120394142420314) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5691952380952381) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.06319778055664713) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7047571428571429) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.05779624954713381) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8013714285714286) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.02391792727193744) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.5435285714285715) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03927314998986781) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7074904761904762) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.03834834240388913) - val-aux/sciknoweval/score/mean@16:np.float64(0.6895833333333333) - val-aux/sciknoweval/score/std@16:np.float64(0.10973085235155812) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.733895238095238) - val-aux/sciknoweval/score/best@2/std:np.float64(0.08320072408312382) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6464904761904762) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.10098054289944847) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6907285714285715) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.10956288078402857) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7644238095238095) - val-aux/sciknoweval/score/best@4/std:np.float64(0.05746985351073585) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.6048666666666667) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.08537783027231041) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6991761904761905) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.08403267534041585) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7858285714285714) - val-aux/sciknoweval/score/best@8/std:np.float64(0.038120394142420314) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5691952380952381) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.06319778055664713) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7047571428571429) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.05779624954713381) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8013714285714286) - val-aux/sciknoweval/score/best@16/std:np.float64(0.02391792727193744) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.5435285714285715) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03927314998986781) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7074904761904762) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.03834834240388913) - val-core/sciknoweval/acc/mean@16:np.float64(0.6895833333333333) - val-aux/sciknoweval/acc/std@16:np.float64(0.10973085235155812) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.733895238095238) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.08320072408312382) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6464904761904762) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.10098054289944847) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6907285714285715) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.10956288078402857) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7644238095238095) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.05746985351073585) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.6048666666666667) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.08537783027231041) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6991761904761905) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.08403267534041585) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7858285714285714) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.038120394142420314) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5691952380952381) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.06319778055664713) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7047571428571429) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.05779624954713381) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8013714285714286) - val-core/sciknoweval/acc/best@16/std:np.float64(0.02391792727193744) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.5435285714285715) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03927314998986781) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7074904761904762) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.03834834240388913) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.999702380952381) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0011526736149426837) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999904761904761) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0002127457895589398) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9994476190476191) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0015248805759714617) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9997) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0011569664768572288) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9988761904761905) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0020220104047945714) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9999476190476191) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0004966783087710074) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9980238095238095) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.002346295386522406) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9967095238095238) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0022003864842959084) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0011474530911073089 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0011474530911073089 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:317.0234375 - response_length/max:616.0 - response_length/min:122.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:317.0234375 - response_length_non_aborted/max:616.0 - response_length_non_aborted/min:122.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.15625 - prompt_length/max:625.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018987245857715607 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5517499037086964) - timing_s/agent_loop/generate_sequences/max:np.float64(5.5460103284567595) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.556732240387646) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.5460103284567595) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:602 - timing_s/gen:11.436559945344925 - timing_s/reward:0.05615238472819328 - timing_s/old_log_prob:2.225490480661392 - timing_s/adv:0.4910304043442011 - timing_s/update_actor:15.488128297030926 - timing_s/step:29.756166722625494 - timing_s/testing:108.75369282439351 - timing_s/save_checkpoint:7.179236024618149 - timing_s/stop_profile:0.00012516416609287262 - timing_per_token_ms/adv:0.003128100222611395 - timing_per_token_ms/gen:0.14091722252082264 - timing_per_token_ms/update_actor:0.0986668384384097 - perf/total_num_tokens:156974 - perf/time_per_step:29.756166722625494 - perf/throughput:659.4179345379304 (TaskRunner pid=2026458) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2026458) "np.float64(0.6895833333333333), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2026458) "np.float64(0.10973085235155812), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2026458) "np.float64(0.733895238095238), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2026458) "np.float64(0.08320072408312382), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6464904761904762), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2026458) "np.float64(0.10098054289944847), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6907285714285715), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2026458) "np.float64(0.10956288078402857), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2026458) "np.float64(0.7644238095238095), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2026458) "np.float64(0.05746985351073585), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6048666666666667), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2026458) "np.float64(0.08537783027231041), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6991761904761905), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2026458) "np.float64(0.08403267534041585), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7858285714285714), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2026458) "np.float64(0.038120394142420314), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2026458) "np.float64(0.5691952380952381), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2026458) "np.float64(0.06319778055664713), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7047571428571429), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2026458) "np.float64(0.05779624954713381), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2026458) "np.float64(0.8013714285714286), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2026458) "np.float64(0.02391792727193744), 'val-aux/sciknoweval/reward/worst@16/mean': " (TaskRunner pid=2026458) "np.float64(0.5435285714285715), 'val-aux/sciknoweval/reward/worst@16/std': " (TaskRunner pid=2026458) "np.float64(0.03927314998986781), 'val-aux/sciknoweval/reward/maj@16/mean': " (TaskRunner pid=2026458) "np.float64(0.7074904761904762), 'val-aux/sciknoweval/reward/maj@16/std': " (TaskRunner pid=2026458) "np.float64(0.03834834240388913), 'val-aux/sciknoweval/score/mean@16': " (TaskRunner pid=2026458) "np.float64(0.6895833333333333), 'val-aux/sciknoweval/score/std@16': " (TaskRunner pid=2026458) "np.float64(0.10973085235155812), 'val-aux/sciknoweval/score/best@2/mean': " (TaskRunner pid=2026458) "np.float64(0.733895238095238), 'val-aux/sciknoweval/score/best@2/std': " (TaskRunner pid=2026458) "np.float64(0.08320072408312382), 'val-aux/sciknoweval/score/worst@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6464904761904762), 'val-aux/sciknoweval/score/worst@2/std': " (TaskRunner pid=2026458) "np.float64(0.10098054289944847), 'val-aux/sciknoweval/score/maj@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6907285714285715), 'val-aux/sciknoweval/score/maj@2/std': " (TaskRunner pid=2026458) "np.float64(0.10956288078402857), 'val-aux/sciknoweval/score/best@4/mean': " (TaskRunner pid=2026458) "np.float64(0.7644238095238095), 'val-aux/sciknoweval/score/best@4/std': " (TaskRunner pid=2026458) "np.float64(0.05746985351073585), 'val-aux/sciknoweval/score/worst@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6048666666666667), 'val-aux/sciknoweval/score/worst@4/std': " (TaskRunner pid=2026458) "np.float64(0.08537783027231041), 'val-aux/sciknoweval/score/maj@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6991761904761905), 'val-aux/sciknoweval/score/maj@4/std': " (TaskRunner pid=2026458) "np.float64(0.08403267534041585), 'val-aux/sciknoweval/score/best@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7858285714285714), 'val-aux/sciknoweval/score/best@8/std': " (TaskRunner pid=2026458) "np.float64(0.038120394142420314), 'val-aux/sciknoweval/score/worst@8/mean': " (TaskRunner pid=2026458) "np.float64(0.5691952380952381), 'val-aux/sciknoweval/score/worst@8/std': " (TaskRunner pid=2026458) "np.float64(0.06319778055664713), 'val-aux/sciknoweval/score/maj@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7047571428571429), 'val-aux/sciknoweval/score/maj@8/std': " (TaskRunner pid=2026458) "np.float64(0.05779624954713381), 'val-aux/sciknoweval/score/best@16/mean': " (TaskRunner pid=2026458) "np.float64(0.8013714285714286), 'val-aux/sciknoweval/score/best@16/std': " (TaskRunner pid=2026458) "np.float64(0.02391792727193744), 'val-aux/sciknoweval/score/worst@16/mean': " (TaskRunner pid=2026458) "np.float64(0.5435285714285715), 'val-aux/sciknoweval/score/worst@16/std': " (TaskRunner pid=2026458) "np.float64(0.03927314998986781), 'val-aux/sciknoweval/score/maj@16/mean': " (TaskRunner pid=2026458) "np.float64(0.7074904761904762), 'val-aux/sciknoweval/score/maj@16/std': " (TaskRunner pid=2026458) "np.float64(0.03834834240388913), 'val-core/sciknoweval/acc/mean@16': " (TaskRunner pid=2026458) "np.float64(0.6895833333333333), 'val-aux/sciknoweval/acc/std@16': " (TaskRunner pid=2026458) "np.float64(0.10973085235155812), 'val-aux/sciknoweval/acc/best@2/mean': " (TaskRunner pid=2026458) "np.float64(0.733895238095238), 'val-aux/sciknoweval/acc/best@2/std': " (TaskRunner pid=2026458) "np.float64(0.08320072408312382), 'val-aux/sciknoweval/acc/worst@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6464904761904762), 'val-aux/sciknoweval/acc/worst@2/std': " (TaskRunner pid=2026458) "np.float64(0.10098054289944847), 'val-aux/sciknoweval/acc/maj@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6907285714285715), 'val-aux/sciknoweval/acc/maj@2/std': " (TaskRunner pid=2026458) "np.float64(0.10956288078402857), 'val-aux/sciknoweval/acc/best@4/mean': " (TaskRunner pid=2026458) "np.float64(0.7644238095238095), 'val-aux/sciknoweval/acc/best@4/std': " (TaskRunner pid=2026458) "np.float64(0.05746985351073585), 'val-aux/sciknoweval/acc/worst@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6048666666666667), 'val-aux/sciknoweval/acc/worst@4/std': " (TaskRunner pid=2026458) "np.float64(0.08537783027231041), 'val-aux/sciknoweval/acc/maj@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6991761904761905), 'val-aux/sciknoweval/acc/maj@4/std': " (TaskRunner pid=2026458) "np.float64(0.08403267534041585), 'val-aux/sciknoweval/acc/best@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7858285714285714), 'val-aux/sciknoweval/acc/best@8/std': " (TaskRunner pid=2026458) "np.float64(0.038120394142420314), 'val-aux/sciknoweval/acc/worst@8/mean': " (TaskRunner pid=2026458) "np.float64(0.5691952380952381), 'val-aux/sciknoweval/acc/worst@8/std': " (TaskRunner pid=2026458) "np.float64(0.06319778055664713), 'val-aux/sciknoweval/acc/maj@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7047571428571429), 'val-aux/sciknoweval/acc/maj@8/std': " (TaskRunner pid=2026458) "np.float64(0.05779624954713381), 'val-core/sciknoweval/acc/best@16/mean': " (TaskRunner pid=2026458) "np.float64(0.8013714285714286), 'val-core/sciknoweval/acc/best@16/std': " (TaskRunner pid=2026458) "np.float64(0.02391792727193744), 'val-aux/sciknoweval/acc/worst@16/mean': " (TaskRunner pid=2026458) "np.float64(0.5435285714285715), 'val-aux/sciknoweval/acc/worst@16/std': " (TaskRunner pid=2026458) "np.float64(0.03927314998986781), 'val-core/sciknoweval/acc/maj@16/mean': " (TaskRunner pid=2026458) "np.float64(0.7074904761904762), 'val-core/sciknoweval/acc/maj@16/std': " (TaskRunner pid=2026458) 'np.float64(0.03834834240388913), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/mean@16': " (TaskRunner pid=2026458) 'np.float64(0.999702380952381), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2026458) 'np.float64(0.0011526736149426837), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@2/mean': " (TaskRunner pid=2026458) 'np.float64(0.9999904761904761), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2026458) 'np.float64(0.0002127457895589398), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': " (TaskRunner pid=2026458) 'np.float64(0.9994476190476191), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2026458) 'np.float64(0.0015248805759714617), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(0.9997), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2026458) 'np.float64(0.0011569664768572288), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@4/mean': np.float64(1.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@4/std': np.float64(0.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': " (TaskRunner pid=2026458) 'np.float64(0.9988761904761905), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2026458) 'np.float64(0.0020220104047945714), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': " (TaskRunner pid=2026458) 'np.float64(0.9999476190476191), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2026458) 'np.float64(0.0004966783087710074), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': " (TaskRunner pid=2026458) 'np.float64(0.9980238095238095), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2026458) 'np.float64(0.002346295386522406), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': np.float64(1.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@8/std': np.float64(0.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': " (TaskRunner pid=2026458) 'np.float64(0.9967095238095238), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2026458) 'np.float64(0.0022003864842959084), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': np.float64(1.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@16/std': np.float64(0.0), " (TaskRunner pid=2026458) "'val-aux/num_turns/min': np.int32(2), 'val-aux/num_turns/max': np.int32(2), " (TaskRunner pid=2026458) "'val-aux/num_turns/mean': np.float64(2.0)}") (TaskRunner pid=2026458) Training Progress: 100%|██████████| 100/100 [1:11:10<00:00, 65.49s/it] Training Progress: 100%|██████████| 100/100 [1:11:10<00:00, 42.70s/it] (TaskRunner pid=2026458) wandb: updating run metadata (TaskRunner pid=2026458) wandb: uploading output.log; uploading wandb-summary.json (TaskRunner pid=2026458) wandb: uploading output.log (TaskRunner pid=2026458) wandb: uploading history steps 98-99, summary, console lines 651-774 (TaskRunner pid=2026458) wandb: (TaskRunner pid=2026458) wandb: Run history: (TaskRunner pid=2026458) wandb: actor/entropy ▂▁▂▅▄█▅▅▃▆▆▆▇█▆▅▆▆▄▆▃▂▄▅▄▃▃▄▃▄▃▃▄▂▂▂▂▂▃▂ (TaskRunner pid=2026458) wandb: actor/grad_norm ▆▄▆▆▆▄▆▅▄▅▆▇▅▄▄▄▃▄▃▃▄▃█▃▄▄▃▁▂▃▃▂▂▃▃▁▃▃▂▂ (TaskRunner pid=2026458) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2026458) wandb: actor/lr ▁▃▄▅▅▇██████████████████████████████████ (TaskRunner pid=2026458) wandb: actor/pg_clipfrac ▁▅▆▆▅▅▄▅▃█▃▆▄▄▆▃▃▆▃▃▃▅▃▇▆▄▃▄▄▄▅▃▃▄▅▂▄▂▅▂ (TaskRunner pid=2026458) wandb: actor/pg_clipfrac_lower ▁▁▁▁▁▅▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▃▁▁▁▁▁▁▁ (TaskRunner pid=2026458) wandb: actor/pg_loss █▅▅▆▆▄▇▅▄▄▃▃▃▄▁▂▃▂▄▃▃▃▃▂▃▂▃▂▂▂▃▃▃▄▄▁▂▂▂▂ (TaskRunner pid=2026458) wandb: actor/ppo_kl ▃▄▁▃▆▂▅▃▄▂▁▃▃█▆▆▄▄▅▅▃▅▆▂▆▆▆▂█▄▃▄▆▄▅▆█▄▇▄ (TaskRunner pid=2026458) wandb: critic/advantages/max █████████▃██▆████████████████▆██▆▆▆▁████ (TaskRunner pid=2026458) wandb: critic/advantages/mean ▃█▅▄▃▂▃▄▁▂▃▄▄▁▄▃▄▂▃▅▄▄▃▄▃▂▄▃▃▄▄▂▄▁▄▃▃▄▂▃ (TaskRunner pid=2026458) wandb: +220 ... (TaskRunner pid=2026458) wandb: (TaskRunner pid=2026458) wandb: Run summary: (TaskRunner pid=2026458) wandb: actor/entropy 0.21838 (TaskRunner pid=2026458) wandb: actor/grad_norm 0.25845 (TaskRunner pid=2026458) wandb: actor/kl_loss 0 (TaskRunner pid=2026458) wandb: actor/lr 0.0 (TaskRunner pid=2026458) wandb: actor/pg_clipfrac 0.00042 (TaskRunner pid=2026458) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2026458) wandb: actor/pg_loss 0.00233 (TaskRunner pid=2026458) wandb: actor/ppo_kl 0.00011 (TaskRunner pid=2026458) wandb: critic/advantages/max 0.75 (TaskRunner pid=2026458) wandb: critic/advantages/mean -0.00115 (TaskRunner pid=2026458) wandb: +220 ... (TaskRunner pid=2026458) wandb: (TaskRunner pid=2026458) wandb: 🚀 View run qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/bz6p2yxy (TaskRunner pid=2026458) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2026458) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2026458) wandb: Find logs at: ./wandb/run-20260702_062054-bz6p2yxy/logs (TaskRunner pid=2026458) Exception ignored in atexit callback: .teardown_atexit at 0x7a92e933fb00> (TaskRunner pid=2026458) Traceback (most recent call last): (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 65, in teardown_atexit (TaskRunner pid=2026458) conn.teardown(hooks.exit_code) (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 299, in teardown (TaskRunner pid=2026458) self._asyncer.run(publish_teardown_and_close) (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run (TaskRunner pid=2026458) return future.result() (TaskRunner pid=2026458) ^^^^^^^^^^^^^^^ (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result (TaskRunner pid=2026458) return self.__get_result() (TaskRunner pid=2026458) ^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (TaskRunner pid=2026458) raise self._exception (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap (TaskRunner pid=2026458) return await fn() (TaskRunner pid=2026458) ^^^^^^^^^^ (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 297, in publish_teardown_and_close (TaskRunner pid=2026458) await self._client.close() (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 69, in close (TaskRunner pid=2026458) await self._writer.wait_closed() (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 364, in wait_closed (TaskRunner pid=2026458) await self._protocol._get_close_waiter(self) (TaskRunner pid=2026458) BrokenPipeError: [Errno 32] Broken pipe main_ppo exit code: 0 [2026-07-02 07:32:12] Finished chemistry rlsd_tr [2026-07-02 07:32:12] Starting physics grpo Experiment: qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Dataset: physics Method: grpo Config: baseline_grpo Model: Qwen/Qwen3-4B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet Ray tmp: /tmp/ray_q3g_physics_grpo_Qwen3_4B 2026-07-02 07:32:14,409 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Config: baseline_grpo Task: datasets/sciknoweval/physics Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-GRPO-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_physics_grpo_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/physics +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.actor.policy_loss.loss_mode=vanilla actor_rollout_ref.actor.kl_loss_coef=0.0 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_physics_grpo_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/physics', 'EXPERIMENT': 'qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 07:32:24,172 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2049544) TaskRunner hostname: mole-workspace-rw, PID: 2049544 (TaskRunner pid=2049544) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2049544) 'calculate_entropy': False, (TaskRunner pid=2049544) 'calculate_sum_pi_squared': False, (TaskRunner pid=2049544) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2049544) 'async_save': False, (TaskRunner pid=2049544) 'load_contents': ['model', (TaskRunner pid=2049544) 'optimizer', (TaskRunner pid=2049544) 'extra'], (TaskRunner pid=2049544) 'save_contents': ['model', (TaskRunner pid=2049544) 'optimizer', (TaskRunner pid=2049544) 'extra']}, (TaskRunner pid=2049544) 'clip_ratio': 0.2, (TaskRunner pid=2049544) 'clip_ratio_c': 3.0, (TaskRunner pid=2049544) 'clip_ratio_high': 0.28, (TaskRunner pid=2049544) 'clip_ratio_low': 0.2, (TaskRunner pid=2049544) 'data_loader_seed': 42, (TaskRunner pid=2049544) 'entropy_checkpointing': False, (TaskRunner pid=2049544) 'entropy_coeff': 0, (TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2049544) 'freeze_vision_tower': False, (TaskRunner pid=2049544) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2049544) 'dtype': 'bfloat16', (TaskRunner pid=2049544) 'entropy_checkpointing': False, (TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2049544) 'forward_only': False, (TaskRunner pid=2049544) 'forward_prefetch': False, (TaskRunner pid=2049544) 'fsdp_size': -1, (TaskRunner pid=2049544) 'full_determinism': False, (TaskRunner pid=2049544) 'model_dtype': 'fp32', (TaskRunner pid=2049544) 'offload_policy': False, (TaskRunner pid=2049544) 'optimizer_offload': False, (TaskRunner pid=2049544) 'param_offload': False, (TaskRunner pid=2049544) 'reshard_after_forward': True, (TaskRunner pid=2049544) 'seed': 42, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_orig_params': False, (TaskRunner pid=2049544) 'use_torch_compile': True, (TaskRunner pid=2049544) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2049544) 'grad_clip': 1.0, (TaskRunner pid=2049544) 'kl_loss_coef': 0.0, (TaskRunner pid=2049544) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2049544) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2049544) 'loss_scale_factor': None, (TaskRunner pid=2049544) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2049544) 'betas': [0.9, 0.999], (TaskRunner pid=2049544) 'clip_grad': 1.0, (TaskRunner pid=2049544) 'lr': 1e-06, (TaskRunner pid=2049544) 'lr_scheduler_type': 'constant', (TaskRunner pid=2049544) 'lr_warmup_steps': 10, (TaskRunner pid=2049544) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2049544) 'min_lr_ratio': 0.0, (TaskRunner pid=2049544) 'num_cycles': 0.5, (TaskRunner pid=2049544) 'optimizer': 'AdamW', (TaskRunner pid=2049544) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2049544) 'override_optimizer_config': None, (TaskRunner pid=2049544) 'total_training_steps': -1, (TaskRunner pid=2049544) 'warmup_style': None, (TaskRunner pid=2049544) 'weight_decay': 0.01}, (TaskRunner pid=2049544) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2049544) 'clip_cov_lb': 1.0, (TaskRunner pid=2049544) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2049544) 'clip_cov_ub': 5.0, (TaskRunner pid=2049544) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2049544) 'loss_mode': 'vanilla', (TaskRunner pid=2049544) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2049544) 'ppo_epochs': 1, (TaskRunner pid=2049544) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2049544) 'ppo_micro_batch_size': None, (TaskRunner pid=2049544) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2049544) 'ppo_mini_batch_size': 8, (TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'all_ranks': False, (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'ranks': [], (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'tool': None, (TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2049544) 'analysis': True, (TaskRunner pid=2049544) 'contents': [], (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'level': 'level0'}, (TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'discrete': False}, (TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2049544) 'step_end': None, (TaskRunner pid=2049544) 'step_start': 0}, (TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2049544) 'rollout_n': 8, (TaskRunner pid=2049544) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2049544) 'mode': 'disabled', (TaskRunner pid=2049544) 'record_file': None, (TaskRunner pid=2049544) 'replay_file': None}, (TaskRunner pid=2049544) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2049544) 'alpha': 0.5, (TaskRunner pid=2049544) 'distillation_add_tail': True, (TaskRunner pid=2049544) 'distillation_topk': 100, (TaskRunner pid=2049544) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2049544) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2049544) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'loss_weight': 0.0, (TaskRunner pid=2049544) 'mask': 'all'}, (TaskRunner pid=2049544) 'feedback_template': '\n' (TaskRunner pid=2049544) 'The ' (TaskRunner pid=2049544) 'following ' (TaskRunner pid=2049544) 'is ' (TaskRunner pid=2049544) 'feedback ' (TaskRunner pid=2049544) 'from ' (TaskRunner pid=2049544) 'your ' (TaskRunner pid=2049544) 'unsuccessful ' (TaskRunner pid=2049544) 'earlier ' (TaskRunner pid=2049544) 'attempt:\n' (TaskRunner pid=2049544) '\n' (TaskRunner pid=2049544) '{feedback_raw}', (TaskRunner pid=2049544) 'full_logit_distillation': True, (TaskRunner pid=2049544) 'include_environment_feedback': True, (TaskRunner pid=2049544) 'is_clip': 2, (TaskRunner pid=2049544) 'max_reprompt_len': 22528, (TaskRunner pid=2049544) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2049544) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2049544) '\n' (TaskRunner pid=2049544) 'Correctly ' (TaskRunner pid=2049544) 'solve ' (TaskRunner pid=2049544) 'the ' (TaskRunner pid=2049544) 'original ' (TaskRunner pid=2049544) 'question.', (TaskRunner pid=2049544) 'reprompt_truncation': 'right', (TaskRunner pid=2049544) 'solution_template': '\n' (TaskRunner pid=2049544) 'Correct ' (TaskRunner pid=2049544) 'solution:\n' (TaskRunner pid=2049544) '\n' (TaskRunner pid=2049544) '{successful_previous_attempt}', (TaskRunner pid=2049544) 'srpo_dynamic_weighting': False, (TaskRunner pid=2049544) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2049544) 'success_reward_threshold': 0.5, (TaskRunner pid=2049544) 'teacher_regularization': 'ema', (TaskRunner pid=2049544) 'teacher_update_rate': 0.0, (TaskRunner pid=2049544) 'token_reweight_decay_steps': None, (TaskRunner pid=2049544) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2049544) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2049544) 'shuffle': False, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2049544) 'tau_neg': 1.05, (TaskRunner pid=2049544) 'tau_pos': 1.0, (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_dynamic_bsz': False, (TaskRunner pid=2049544) 'use_fused_kernels': False, (TaskRunner pid=2049544) 'use_kl_loss': False, (TaskRunner pid=2049544) 'use_prefix_grouper': False, (TaskRunner pid=2049544) 'use_remove_padding': True, (TaskRunner pid=2049544) 'use_torch_compile': True}, (TaskRunner pid=2049544) 'hybrid_engine': True, (TaskRunner pid=2049544) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2049544) 'custom_chat_template': None, (TaskRunner pid=2049544) 'enable_activation_offload': False, (TaskRunner pid=2049544) 'enable_gradient_checkpointing': True, (TaskRunner pid=2049544) 'exclude_modules': None, (TaskRunner pid=2049544) 'external_lib': None, (TaskRunner pid=2049544) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2049544) 'hf_config_path': None, (TaskRunner pid=2049544) 'lora_adapter_path': None, (TaskRunner pid=2049544) 'lora_alpha': 16, (TaskRunner pid=2049544) 'lora_rank': 0, (TaskRunner pid=2049544) 'override_config': {}, (TaskRunner pid=2049544) 'path': 'Qwen/Qwen3-4B', (TaskRunner pid=2049544) 'target_modules': 'all-linear', (TaskRunner pid=2049544) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2049544) 'num_shards': 4}, (TaskRunner pid=2049544) 'tokenizer_path': None, (TaskRunner pid=2049544) 'trust_remote_code': True, (TaskRunner pid=2049544) 'use_fused_kernels': False, (TaskRunner pid=2049544) 'use_liger': False, (TaskRunner pid=2049544) 'use_remove_padding': True, (TaskRunner pid=2049544) 'use_shm': False}, (TaskRunner pid=2049544) 'nccl_timeout': 600, (TaskRunner pid=2049544) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2049544) 'entropy_checkpointing': False, (TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2049544) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2049544) 'dtype': 'bfloat16', (TaskRunner pid=2049544) 'entropy_checkpointing': False, (TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2049544) 'forward_only': True, (TaskRunner pid=2049544) 'forward_prefetch': False, (TaskRunner pid=2049544) 'fsdp_size': -1, (TaskRunner pid=2049544) 'full_determinism': False, (TaskRunner pid=2049544) 'model_dtype': 'fp32', (TaskRunner pid=2049544) 'offload_policy': False, (TaskRunner pid=2049544) 'optimizer_offload': False, (TaskRunner pid=2049544) 'param_offload': False, (TaskRunner pid=2049544) 'reshard_after_forward': True, (TaskRunner pid=2049544) 'seed': 42, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_orig_params': False, (TaskRunner pid=2049544) 'use_torch_compile': True, (TaskRunner pid=2049544) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2049544) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2049544) 'log_prob_micro_batch_size': None, (TaskRunner pid=2049544) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2049544) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'all_ranks': False, (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'ranks': [], (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'tool': None, (TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2049544) 'analysis': True, (TaskRunner pid=2049544) 'contents': [], (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'level': 'level0'}, (TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'discrete': False}, (TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2049544) 'step_end': None, (TaskRunner pid=2049544) 'step_start': 0}, (TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2049544) 'rollout_n': 8, (TaskRunner pid=2049544) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2049544) 'mode': 'disabled', (TaskRunner pid=2049544) 'record_file': None, (TaskRunner pid=2049544) 'replay_file': None}, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_torch_compile': True}, (TaskRunner pid=2049544) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2049544) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2049544) 'agent_loop_config_path': None, (TaskRunner pid=2049544) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2049544) 'name': None, (TaskRunner pid=2049544) 'path': None}, (TaskRunner pid=2049544) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2049544) 'num_workers': 8}, (TaskRunner pid=2049544) 'calculate_log_probs': True, (TaskRunner pid=2049544) 'cudagraph_capture_sizes': None, (TaskRunner pid=2049544) 'data_parallel_size': 1, (TaskRunner pid=2049544) 'disable_log_stats': True, (TaskRunner pid=2049544) 'do_sample': True, (TaskRunner pid=2049544) 'dtype': 'bfloat16', (TaskRunner pid=2049544) 'enable_chunked_prefill': True, (TaskRunner pid=2049544) 'enable_prefix_caching': True, (TaskRunner pid=2049544) 'enable_rollout_routing_replay': False, (TaskRunner pid=2049544) 'enforce_eager': False, (TaskRunner pid=2049544) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2049544) 'expert_parallel_size': 1, (TaskRunner pid=2049544) 'free_cache_engine': True, (TaskRunner pid=2049544) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2049544) 'ignore_eos': False, (TaskRunner pid=2049544) 'layered_summon': False, (TaskRunner pid=2049544) 'load_format': 'dummy', (TaskRunner pid=2049544) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2049544) 'log_prob_micro_batch_size': None, (TaskRunner pid=2049544) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2049544) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2049544) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2049544) 'max_model_len': 10240, (TaskRunner pid=2049544) 'max_num_batched_tokens': 10240, (TaskRunner pid=2049544) 'max_num_seqs': 1024, (TaskRunner pid=2049544) 'mode': 'async', (TaskRunner pid=2049544) 'multi_stage_wake_up': False, (TaskRunner pid=2049544) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'format': 'hermes', (TaskRunner pid=2049544) 'interaction_config_path': None, (TaskRunner pid=2049544) 'max_assistant_turns': None, (TaskRunner pid=2049544) 'max_parallel_calls': 1, (TaskRunner pid=2049544) 'max_tool_response_length': 256, (TaskRunner pid=2049544) 'max_user_turns': None, (TaskRunner pid=2049544) 'num_repeat_rollouts': None, (TaskRunner pid=2049544) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2049544) 'tool_config_path': None, (TaskRunner pid=2049544) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2049544) 'use_inference_chat_template': False}, (TaskRunner pid=2049544) 'n': 8, (TaskRunner pid=2049544) 'name': 'vllm', (TaskRunner pid=2049544) 'over_sample_rate': 0, (TaskRunner pid=2049544) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'all_ranks': False, (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'ranks': [], (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'tool': None, (TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2049544) 'analysis': True, (TaskRunner pid=2049544) 'contents': [], (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'level': 'level0'}, (TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'discrete': False}, (TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2049544) 'step_end': None, (TaskRunner pid=2049544) 'step_start': 0}, (TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2049544) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2049544) 'port': 9090, (TaskRunner pid=2049544) 'served_model_name': 'Qwen/Qwen3-4B'}, (TaskRunner pid=2049544) 'prompt_length': 2048, (TaskRunner pid=2049544) 'quantization': None, (TaskRunner pid=2049544) 'quantization_config_file': None, (TaskRunner pid=2049544) 'response_length': 8192, (TaskRunner pid=2049544) 'scheduling_policy': 'fcfs', (TaskRunner pid=2049544) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2049544) 'skip_rollout': False, (TaskRunner pid=2049544) 'skip_tokenizer_init': True, (TaskRunner pid=2049544) 'temperature': 1.0, (TaskRunner pid=2049544) 'tensor_model_parallel_size': 2, (TaskRunner pid=2049544) 'top_k': -1, (TaskRunner pid=2049544) 'top_p': 1.0, (TaskRunner pid=2049544) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2049544) 'backend': None, (TaskRunner pid=2049544) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2049544) 'token2text': False}, (TaskRunner pid=2049544) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2049544) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2049544) 'do_sample': True, (TaskRunner pid=2049544) 'n': 16, (TaskRunner pid=2049544) 'temperature': 0.6, (TaskRunner pid=2049544) 'top_k': -1, (TaskRunner pid=2049544) 'top_p': 0.95}}}, (TaskRunner pid=2049544) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2049544) 'adv_estimator': 'grpo', (TaskRunner pid=2049544) 'gamma': 1.0, (TaskRunner pid=2049544) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2049544) 'horizon': 10000, (TaskRunner pid=2049544) 'kl_coef': 0.001, (TaskRunner pid=2049544) 'target_kl': 0.1, (TaskRunner pid=2049544) 'type': 'fixed'}, (TaskRunner pid=2049544) 'kl_penalty': 'kl', (TaskRunner pid=2049544) 'lam': 1.0, (TaskRunner pid=2049544) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2049544) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2049544) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2049544) 'loss_type': 'ppo_clip', (TaskRunner pid=2049544) 'rollout_is': 'token', (TaskRunner pid=2049544) 'rollout_is_batch_normalize': False, (TaskRunner pid=2049544) 'rollout_is_threshold': 2.0, (TaskRunner pid=2049544) 'rollout_rs': None, (TaskRunner pid=2049544) 'rollout_rs_threshold': None}, (TaskRunner pid=2049544) 'use_kl_in_reward': False, (TaskRunner pid=2049544) 'use_pf_ppo': False}, (TaskRunner pid=2049544) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2049544) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2049544) 'async_save': False, (TaskRunner pid=2049544) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2049544) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2049544) 'cliprange_value': 0.5, (TaskRunner pid=2049544) 'data_loader_seed': 42, (TaskRunner pid=2049544) 'enable': None, (TaskRunner pid=2049544) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2049544) 'forward_micro_batch_size': None, (TaskRunner pid=2049544) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2049544) 'grad_clip': 1.0, (TaskRunner pid=2049544) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2049544) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2049544) 'enable_activation_offload': False, (TaskRunner pid=2049544) 'enable_gradient_checkpointing': True, (TaskRunner pid=2049544) 'external_lib': None, (TaskRunner pid=2049544) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2049544) 'dtype': 'bfloat16', (TaskRunner pid=2049544) 'entropy_checkpointing': False, (TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2049544) 'forward_only': False, (TaskRunner pid=2049544) 'forward_prefetch': False, (TaskRunner pid=2049544) 'fsdp_size': -1, (TaskRunner pid=2049544) 'full_determinism': False, (TaskRunner pid=2049544) 'model_dtype': 'fp32', (TaskRunner pid=2049544) 'offload_policy': False, (TaskRunner pid=2049544) 'optimizer_offload': False, (TaskRunner pid=2049544) 'param_offload': False, (TaskRunner pid=2049544) 'reshard_after_forward': True, (TaskRunner pid=2049544) 'seed': 42, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_orig_params': False, (TaskRunner pid=2049544) 'use_torch_compile': True, (TaskRunner pid=2049544) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2049544) 'lora_alpha': 16, (TaskRunner pid=2049544) 'lora_rank': 0, (TaskRunner pid=2049544) 'override_config': {}, (TaskRunner pid=2049544) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2049544) 'target_modules': 'all-linear', (TaskRunner pid=2049544) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2049544) 'tokenizer_path': 'Qwen/Qwen3-4B', (TaskRunner pid=2049544) 'trust_remote_code': True, (TaskRunner pid=2049544) 'use_remove_padding': False, (TaskRunner pid=2049544) 'use_shm': False}, (TaskRunner pid=2049544) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2049544) 'betas': [0.9, 0.999], (TaskRunner pid=2049544) 'clip_grad': 1.0, (TaskRunner pid=2049544) 'lr': 1e-05, (TaskRunner pid=2049544) 'lr_scheduler_type': 'constant', (TaskRunner pid=2049544) 'lr_warmup_steps': -1, (TaskRunner pid=2049544) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2049544) 'min_lr_ratio': 0.0, (TaskRunner pid=2049544) 'num_cycles': 0.5, (TaskRunner pid=2049544) 'optimizer': 'AdamW', (TaskRunner pid=2049544) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2049544) 'override_optimizer_config': None, (TaskRunner pid=2049544) 'total_training_steps': -1, (TaskRunner pid=2049544) 'warmup_style': None, (TaskRunner pid=2049544) 'weight_decay': 0.01}, (TaskRunner pid=2049544) 'ppo_epochs': 1, (TaskRunner pid=2049544) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2049544) 'ppo_micro_batch_size': None, (TaskRunner pid=2049544) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2049544) 'ppo_mini_batch_size': 8, (TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'all_ranks': False, (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'ranks': [], (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'tool': None, (TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2049544) 'analysis': True, (TaskRunner pid=2049544) 'contents': [], (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'level': 'level0'}, (TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'discrete': False}, (TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2049544) 'step_end': None, (TaskRunner pid=2049544) 'step_start': 0}, (TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2049544) 'rollout_n': 8, (TaskRunner pid=2049544) 'shuffle': False, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_dynamic_bsz': False}, (TaskRunner pid=2049544) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2049544) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2049544) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2049544) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2049544) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2049544) 'dataloader_num_workers': 8, (TaskRunner pid=2049544) 'filter_overlong_prompts': True, (TaskRunner pid=2049544) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2049544) 'image_key': 'images', (TaskRunner pid=2049544) 'image_patch_size': 14, (TaskRunner pid=2049544) 'max_prompt_length': 2048, (TaskRunner pid=2049544) 'max_response_length': 8192, (TaskRunner pid=2049544) 'prompt_key': 'prompt', (TaskRunner pid=2049544) 'return_full_prompt': False, (TaskRunner pid=2049544) 'return_multi_modal_inputs': True, (TaskRunner pid=2049544) 'return_raw_chat': True, (TaskRunner pid=2049544) 'return_raw_input_ids': False, (TaskRunner pid=2049544) 'reward_fn_key': 'data_source', (TaskRunner pid=2049544) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2049544) 'seed': None, (TaskRunner pid=2049544) 'shuffle': True, (TaskRunner pid=2049544) 'tokenizer': None, (TaskRunner pid=2049544) 'tool_config_path': None, (TaskRunner pid=2049544) 'train_batch_size': 32, (TaskRunner pid=2049544) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet'], (TaskRunner pid=2049544) 'train_max_samples': 3200, (TaskRunner pid=2049544) 'truncation': 'error', (TaskRunner pid=2049544) 'trust_remote_code': True, (TaskRunner pid=2049544) 'use_shm': False, (TaskRunner pid=2049544) 'val_batch_size': None, (TaskRunner pid=2049544) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet'], (TaskRunner pid=2049544) 'val_max_samples': -1, (TaskRunner pid=2049544) 'validation_shuffle': False, (TaskRunner pid=2049544) 'video_key': 'videos'}, (TaskRunner pid=2049544) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2049544) 'cuda-memory-usage': 'true', (TaskRunner pid=2049544) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2049544) 'capture-range-end': None, (TaskRunner pid=2049544) 'cuda-graph-trace': 'graph', (TaskRunner pid=2049544) 'cuda-memory-usage': 'true', (TaskRunner pid=2049544) 'kill': 'none', (TaskRunner pid=2049544) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2049544) 'torch_memory': {'context': 'all', (TaskRunner pid=2049544) 'kw_args': {}, (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'stacks': 'all', (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2049544) 'profile_continuous_steps': False, (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'steps': None, (TaskRunner pid=2049544) 'tool': None}, (TaskRunner pid=2049544) 'max_model_len': 10240, (TaskRunner pid=2049544) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_physics_grpo_Qwen3_4B', (TaskRunner pid=2049544) 'include_dashboard': False, (TaskRunner pid=2049544) 'num_cpus': None, (TaskRunner pid=2049544) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2049544) 'TASK': 'datasets/sciknoweval/physics', (TaskRunner pid=2049544) 'USER': 'root'}}}, (TaskRunner pid=2049544) 'timeline_json_file': None}, (TaskRunner pid=2049544) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2049544) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2049544) 'name': 'custom_reward_manager', (TaskRunner pid=2049544) 'path': None}, (TaskRunner pid=2049544) 'name': 'naive', (TaskRunner pid=2049544) 'source': 'register'}, (TaskRunner pid=2049544) 'reward_model': {'enable': False, (TaskRunner pid=2049544) 'enable_resource_pool': False, (TaskRunner pid=2049544) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2049544) 'launch_reward_fn_async': False, (TaskRunner pid=2049544) 'max_length': None, (TaskRunner pid=2049544) 'micro_batch_size': None, (TaskRunner pid=2049544) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2049544) 'model': {'external_lib': None, (TaskRunner pid=2049544) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2049544) 'forward_prefetch': False, (TaskRunner pid=2049544) 'fsdp_size': -1, (TaskRunner pid=2049544) 'param_offload': False, (TaskRunner pid=2049544) 'reshard_after_forward': True, (TaskRunner pid=2049544) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2049544) 'input_tokenizer': 'Qwen/Qwen3-4B', (TaskRunner pid=2049544) 'override_config': {}, (TaskRunner pid=2049544) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2049544) 'trust_remote_code': False, (TaskRunner pid=2049544) 'use_fused_kernels': False, (TaskRunner pid=2049544) 'use_remove_padding': False, (TaskRunner pid=2049544) 'use_shm': False}, (TaskRunner pid=2049544) 'n_gpus_per_node': 8, (TaskRunner pid=2049544) 'nnodes': 0, (TaskRunner pid=2049544) 'num_workers': 1, (TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'all_ranks': False, (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'ranks': [], (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'tool': None, (TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2049544) 'analysis': True, (TaskRunner pid=2049544) 'contents': [], (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'level': 'level0'}, (TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'discrete': False}, (TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2049544) 'step_end': None, (TaskRunner pid=2049544) 'step_start': 0}, (TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2049544) 'reward_loop_class_name': None, (TaskRunner pid=2049544) 'reward_loop_module_path': None, (TaskRunner pid=2049544) 'reward_loop_source': 'register', (TaskRunner pid=2049544) 'reward_manager': 'naive', (TaskRunner pid=2049544) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2049544) 'cudagraph_capture_sizes': None, (TaskRunner pid=2049544) 'data_parallel_size': 1, (TaskRunner pid=2049544) 'disable_log_stats': True, (TaskRunner pid=2049544) 'dtype': 'bfloat16', (TaskRunner pid=2049544) 'enable_chunked_prefill': True, (TaskRunner pid=2049544) 'enable_prefix_caching': True, (TaskRunner pid=2049544) 'enforce_eager': True, (TaskRunner pid=2049544) 'engine_kwargs': {}, (TaskRunner pid=2049544) 'expert_parallel_size': 1, (TaskRunner pid=2049544) 'free_cache_engine': True, (TaskRunner pid=2049544) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2049544) 'limit_images': None, (TaskRunner pid=2049544) 'load_format': 'auto', (TaskRunner pid=2049544) 'max_model_len': None, (TaskRunner pid=2049544) 'max_num_batched_tokens': 8192, (TaskRunner pid=2049544) 'max_num_seqs': 1024, (TaskRunner pid=2049544) 'name': '???', (TaskRunner pid=2049544) 'prompt_length': 2048, (TaskRunner pid=2049544) 'response_length': 2048, (TaskRunner pid=2049544) 'skip_tokenizer_init': False, (TaskRunner pid=2049544) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2049544) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2049544) 'memory_limit_mb': 1024, (TaskRunner pid=2049544) 'url': None}, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_dynamic_bsz': False, (TaskRunner pid=2049544) 'use_reward_loop': False}, (TaskRunner pid=2049544) 'trainer': {'balance_batch': True, (TaskRunner pid=2049544) 'critic_warmup': 0, (TaskRunner pid=2049544) 'default_hdfs_dir': None, (TaskRunner pid=2049544) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2049544) 'del_local_ckpt_after_load': False, (TaskRunner pid=2049544) 'device': 'cuda', (TaskRunner pid=2049544) 'esi_redundant_time': 0, (TaskRunner pid=2049544) 'experiment_name': 'qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2049544) 'group_name': 'QWEN3-GRPO-generalization', (TaskRunner pid=2049544) 'log_val_generations': 0, (TaskRunner pid=2049544) 'logger': ['console', 'wandb'], (TaskRunner pid=2049544) 'max_actor_ckpt_to_keep': 1, (TaskRunner pid=2049544) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2049544) 'n_gpus_per_node': 8, (TaskRunner pid=2049544) 'nnodes': 1, (TaskRunner pid=2049544) 'project_name': 'SDPO-root', (TaskRunner pid=2049544) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2049544) 'resume_from_path': None, (TaskRunner pid=2049544) 'resume_mode': 'auto', (TaskRunner pid=2049544) 'rollout_data_dir': None, (TaskRunner pid=2049544) 'save_freq': 10, (TaskRunner pid=2049544) 'test_freq': 10, (TaskRunner pid=2049544) 'total_epochs': 30, (TaskRunner pid=2049544) 'total_training_steps': 100, (TaskRunner pid=2049544) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2049544) 'val_before_train': False, (TaskRunner pid=2049544) 'val_only': False, (TaskRunner pid=2049544) 'validation_data_dir': None}, (TaskRunner pid=2049544) 'transfer_queue': {'enable': False}, (TaskRunner pid=2049544) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/physics', (TaskRunner pid=2049544) 'dir': '/users/root/SDPO', (TaskRunner pid=2049544) 'log_dir': '/users/root/output', (TaskRunner pid=2049544) 'task': 'datasets/sciknoweval/physics'}} (TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2049544) use_critic=need_critic(config), (TaskRunner pid=2049544) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2049544) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2049544) Using dataset class: RLHFDataset (TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2049544) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2049544) dataset len: 720 (TaskRunner pid=2049544) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2049544) WARNING:2026-07-02 07:32:33,214:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2049544) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/720 [00:00 (TaskRunner pid=2049544) bind role actor_rollout method chat_completion to class .WorkerDict'> (TaskRunner pid=2049544) bind role actor_rollout method generate to class .WorkerDict'> (TaskRunner pid=2049544) bind role actor_rollout method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2049544) bind role actor_rollout method sleep to class .WorkerDict'> (TaskRunner pid=2049544) bind role actor_rollout method wake_up to class .WorkerDict'> (TaskRunner pid=2049544) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 80/80 [00:00<00:00, 121.99 examples/s] (TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2049544) self.use_critic = need_critic(self.config) (WorkerDict pid=2049933) [W702 07:32:42.949021168 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:59177 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2049932) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2049932) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2049932) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2049934) [W702 07:32:42.120202714 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:59177 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2049932) Model config after override: Qwen3Config { (WorkerDict pid=2049932) "architectures": [ (WorkerDict pid=2049932) "Qwen3ForCausalLM" (WorkerDict pid=2049932) ], (WorkerDict pid=2049932) "attention_bias": false, (WorkerDict pid=2049932) "attention_dropout": 0.0, (WorkerDict pid=2049932) "dtype": "bfloat16", (WorkerDict pid=2049932) "eos_token_id": 151645, (WorkerDict pid=2049932) "head_dim": 128, (WorkerDict pid=2049932) "hidden_act": "silu", (WorkerDict pid=2049932) "hidden_size": 2560, (WorkerDict pid=2049932) "initializer_range": 0.02, (WorkerDict pid=2049932) "intermediate_size": 9728, (WorkerDict pid=2049932) "layer_types": [ (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention" (WorkerDict pid=2049932) ], (WorkerDict pid=2049932) "max_position_embeddings": 40960, (WorkerDict pid=2049932) "max_window_layers": 36, (WorkerDict pid=2049932) "model_type": "qwen3", (WorkerDict pid=2049932) "num_attention_heads": 32, (WorkerDict pid=2049932) "num_hidden_layers": 36, (WorkerDict pid=2049932) "num_key_value_heads": 8, (WorkerDict pid=2049932) "pad_token_id": 151643, (WorkerDict pid=2049932) "rms_norm_eps": 1e-06, (WorkerDict pid=2049932) "rope_scaling": null, (WorkerDict pid=2049932) "rope_theta": 1000000, (WorkerDict pid=2049932) "sliding_window": null, (WorkerDict pid=2049932) "tie_word_embeddings": true, (WorkerDict pid=2049932) "transformers_version": "4.57.1", (WorkerDict pid=2049932) "use_cache": true, (WorkerDict pid=2049932) "use_sliding_window": false, (WorkerDict pid=2049932) "vocab_size": 151936 (WorkerDict pid=2049932) } (WorkerDict pid=2049932) (WorkerDict pid=2049932) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2049932) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2049932) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2049932) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2049932) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2049932) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2049932) Actor use_remove_padding=True (WorkerDict pid=2049932) Actor use_fused_kernels=False (WorkerDict pid=2049932) Actor use_prefix_grouper=False (WorkerDict pid=2049939) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2049939) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2049932) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2049932) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2049939) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.92s/it] [repeated 12x across cluster] (WorkerDict pid=2049939) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.97s/it] [repeated 4x across cluster] (WorkerDict pid=2049938) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.88s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.94s/it] [repeated 2x across cluster] (WorkerDict pid=2049932) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2049932) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2049939) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2049939) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2049932) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2049933) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2049933) warnings.warn( (WorkerDict pid=2049939) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster] (WorkerDict pid=2049939) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster] (TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2049544) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2049932) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster] (WorkerDict pid=2049932) warnings.warn( [repeated 7x across cluster] (vLLMHttpServer pid=2050805) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2050805) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2050806) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2050806) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2050805) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2050805) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2050803) ['serve', (vLLMHttpServer pid=2050803) 'Qwen/Qwen3-4B', (vLLMHttpServer pid=2050803) '--dtype', (vLLMHttpServer pid=2050803) 'bfloat16', (vLLMHttpServer pid=2050803) '--load_format', (vLLMHttpServer pid=2050803) 'dummy', (vLLMHttpServer pid=2050803) '--trust_remote_code', (vLLMHttpServer pid=2050803) '--max_model_len', (vLLMHttpServer pid=2050803) '40960', (vLLMHttpServer pid=2050803) '--max_num_seqs', (vLLMHttpServer pid=2050803) '1024', (vLLMHttpServer pid=2050803) '--enable_chunked_prefill', (vLLMHttpServer pid=2050803) '--max_num_batched_tokens', (vLLMHttpServer pid=2050803) '10240', (vLLMHttpServer pid=2050803) '--enable_prefix_caching', (vLLMHttpServer pid=2050803) '--enable_sleep_mode', (vLLMHttpServer pid=2050803) '--logprobs_mode', (vLLMHttpServer pid=2050803) 'processed_logprobs', (vLLMHttpServer pid=2050803) '--disable_custom_all_reduce', (vLLMHttpServer pid=2050803) '--gpu_memory_utilization', (vLLMHttpServer pid=2050803) '0.8', (vLLMHttpServer pid=2050803) '--disable_log_stats', (vLLMHttpServer pid=2050803) '--tensor_parallel_size', (vLLMHttpServer pid=2050803) '2', (vLLMHttpServer pid=2050803) '--seed', (vLLMHttpServer pid=2050803) '0', (vLLMHttpServer pid=2050803) '--override_generation_config', (vLLMHttpServer pid=2050803) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2050803) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2050803) '--hf_overrides', (vLLMHttpServer pid=2050803) '{}', (vLLMHttpServer pid=2050803) '--scheduling_policy', (vLLMHttpServer pid=2050803) 'fcfs'] (WorkerDict pid=2049938) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (vLLMHttpServer pid=2050805) WARNING 07-02 07:33:26 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2049936) WARNING 07-02 07:33:34 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2050803) WARNING 07-02 07:33:27 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2049934) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2049934) 2026-07-02 07:33:49,587 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2049934) 2026-07-02 07:33:49,606 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (vLLMHttpServer pid=2050803) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 2x across cluster] (vLLMHttpServer pid=2050803) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 2x across cluster] (vLLMHttpServer pid=2050803) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2050803) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. We need to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substituting the given values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest option is B: 56 pC. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2049544) step:10 - global_seqlen/min:15442 - global_seqlen/max:25018 - global_seqlen/minmax_diff:9576 - global_seqlen/balanced_min:21213 - global_seqlen/balanced_max:21224 - global_seqlen/mean:21220.5 - actor/entropy:0.12735222280025482 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40465062856674194 - training/rollout_probs_diff_mean:0.0038589579053223133 - training/rollout_probs_diff_std:0.013698565773665905 - training/rollout_actor_probs_pearson_corr:0.9970982670783997 - rollout_corr/rollout_is_mean:1.0000947713851929 - rollout_corr/rollout_is_ratio_fraction_high:3.965264113503508e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00020817638142034411 - rollout_corr/rollout_is_max:3.167215347290039 - rollout_corr/rollout_is_min:0.19278468191623688 - rollout_corr/rollout_is_std:0.03876073658466339 - rollout_corr/rollout_is_eff_sample_size:0.9985000968313013 - rollout_corr/rollout_is_seq_mean:1.000048041343689 - rollout_corr/rollout_is_seq_std:0.002764123724773526 - rollout_corr/rollout_is_seq_max:1.0100802183151245 - rollout_corr/rollout_is_seq_min:0.9903356432914734 - rollout_corr/rollout_is_seq_max_deviation:0.010080218315124512 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1353368978016078) - rollout_corr/training_log_ppl:np.float64(0.12389787597203394) - rollout_corr/kl:np.float64(0.000877203190441378) - rollout_corr/k3_kl:np.float64(0.0008888360377170557) - rollout_corr/rollout_ppl:np.float64(1.134331472683698) - rollout_corr/rollout_log_ppl:np.float64(0.12302067201380851) - rollout_corr/log_ppl_diff:np.float64(0.0008772039582254365) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020878618815913796) - rollout_corr/log_ppl_diff_max:np.float64(0.012568935751914978) - rollout_corr/log_ppl_diff_min:np.float64(-0.007140792906284332) - rollout_corr/ppl_ratio:np.float64(1.000881330575794) - rollout_corr/chi2_token:np.float64(0.0018501763697713614) - rollout_corr/chi2_seq:np.float64(0.666111497906968) - actor/pg_loss:np.float64(9.953684639185667e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005019576683480409) - actor/ppo_kl:np.float64(2.3842763525294686e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3116539753973484) - perf/mfu/actor:np.float64(0.0619170865443955) - perf/max_memory_allocated_gb:np.float64(116.3979811668396) - perf/max_memory_reserved_gb:np.float64(120.6328125) - perf/cpu_memory_used_gb:np.float64(98.92141723632812) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.59453125) - val-aux/sciknoweval/reward/std@16:np.float64(0.1688854331093934) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6633625000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13606702193420137) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5265375000000001) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14690490565059572) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.595925) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.16812839582698166) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7161875) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09695252008325414) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.46636249999999996) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1154193735626829) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6132000000000001) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13566324564246196) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.752675) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06479543702598764) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.41965) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08082900337088106) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.622625) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1013687608239918) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.777675) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03976601626053729) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.38728750000000006) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.047588411442781434) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6271749999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07429798598036827) - val-aux/sciknoweval/score/mean@16:np.float64(0.59453125) - val-aux/sciknoweval/score/std@16:np.float64(0.1688854331093934) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6633625000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.13606702193420137) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5265375000000001) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.14690490565059572) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.595925) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.16812839582698166) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7161875) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09695252008325414) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.46636249999999996) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1154193735626829) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6132000000000001) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13566324564246196) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.752675) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06479543702598764) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.41965) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08082900337088106) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.622625) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.1013687608239918) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.777675) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03976601626053729) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.38728750000000006) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.047588411442781434) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6271749999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07429798598036827) - val-core/sciknoweval/acc/mean@16:np.float64(0.59453125) - val-aux/sciknoweval/acc/std@16:np.float64(0.1688854331093934) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6633625000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.13606702193420137) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5265375000000001) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.14690490565059572) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.595925) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.16812839582698166) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7161875) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09695252008325414) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.46636249999999996) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1154193735626829) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6132000000000001) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13566324564246196) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.752675) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06479543702598764) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.41965) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08082900337088106) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.622625) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.1013687608239918) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.777675) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03976601626053729) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.38728750000000006) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.047588411442781434) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6271749999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07429798598036827) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99609375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.015128841196122723) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9997875) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0035548642615965665) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.992525) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0202257687927191) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.996175) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.014896175746267976) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.986025) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.026012467615322592) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.998875) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.008222303730994859) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9748249999999998) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.030646040504587458) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9996625) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0039815903242650785) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9594374999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.02978871055390577) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999500000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0010787124614257437) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008162248879671097 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008162248879671097 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:394.046875 - response_length/max:1270.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:394.046875 - response_length_non_aborted/max:1270.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.09375 - prompt_length/max:382.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013142451643943787 - timing_s/agent_loop/generate_sequences/min:np.float64(0.941465875133872) - timing_s/agent_loop/generate_sequences/max:np.float64(8.937132587656379) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.961100311316841) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.937132587656379) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1270 - timing_s/gen:14.806582853198051 - timing_s/reward:0.059918224811553955 - timing_s/old_log_prob:2.401917176321149 - timing_s/adv:0.15370049141347408 - timing_s/update_actor:9.923850152641535 - timing_s/step:27.43245600350201 - timing_s/testing:88.77899980731308 - timing_s/save_checkpoint:6.549710331484675 - timing_s/stop_profile:0.00011613219976425171 - timing_per_token_ms/update_actor:0.058456740844004235 - timing_per_token_ms/gen:0.14678003542168655 - timing_per_token_ms/adv:0.0009053774146077736 - perf/total_num_tokens:169764 - perf/time_per_step:27.43245600350201 - perf/throughput:773.5545077440753 (TaskRunner pid=2049544) Training Progress: 10%|█ | 10/100 [06:47<1:27:44, 58.49s/it] (TaskRunner pid=2049544) step:11 - global_seqlen/min:16821 - global_seqlen/max:31114 - global_seqlen/minmax_diff:14293 - global_seqlen/balanced_min:23187 - global_seqlen/balanced_max:29684 - global_seqlen/mean:24008.625 - actor/entropy:0.11019539088010788 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5948736071586609 - training/rollout_probs_diff_mean:0.003139072796329856 - training/rollout_probs_diff_std:0.012617148458957672 - training/rollout_actor_probs_pearson_corr:0.997302770614624 - rollout_corr/rollout_is_mean:1.0000996589660645 - rollout_corr/rollout_is_ratio_fraction_high:7.389344682451338e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.852459334069863e-05 - rollout_corr/rollout_is_max:4.887231826782227 - rollout_corr/rollout_is_min:0.26104938983917236 - rollout_corr/rollout_is_std:0.03467484563589096 - rollout_corr/rollout_is_eff_sample_size:0.9987993368491189 - rollout_corr/rollout_is_seq_mean:1.0002052783966064 - rollout_corr/rollout_is_seq_std:0.0023445014376193285 - rollout_corr/rollout_is_seq_max:1.0106254816055298 - rollout_corr/rollout_is_seq_min:0.99307781457901 - rollout_corr/rollout_is_seq_max_deviation:0.010625481605529785 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1195416245609522) - rollout_corr/training_log_ppl:np.float64(0.1096741780929733) - rollout_corr/kl:np.float64(0.0007273018066071835) - rollout_corr/k3_kl:np.float64(0.0009105393462505162) - rollout_corr/rollout_ppl:np.float64(1.1187340491451323) - rollout_corr/rollout_log_ppl:np.float64(0.10894687551262905) - rollout_corr/log_ppl_diff:np.float64(0.0007273025803442579) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019026359186682384) - rollout_corr/log_ppl_diff_max:np.float64(0.022861547768115997) - rollout_corr/log_ppl_diff_min:np.float64(-0.007030755281448364) - rollout_corr/ppl_ratio:np.float64(1.000731429317966) - rollout_corr/chi2_token:np.float64(0.002295879879966378) - rollout_corr/chi2_seq:np.float64(2.1700741790700704) - actor/pg_loss:np.float64(8.732720743864775e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011307528586712579) - actor/ppo_kl:np.float64(0.00020700877354862257) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.46738822013139725) - perf/mfu/actor:np.float64(0.06503294604815439) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.65916442871094) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01939600333571434 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01939600333571434 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:475.76953125 - response_length/max:8192.0 - response_length/min:69.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:475.76953125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:274.5 - prompt_length/max:437.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001276358962059021 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8453756030648947) - timing_s/agent_loop/generate_sequences/max:np.float64(51.0958620775491) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.681928445308586) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(51.0958620775491) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.71189962141216 - timing_s/reward:0.06205694004893303 - timing_s/old_log_prob:2.802142111584544 - timing_s/adv:0.1481334324926138 - timing_s/update_actor:10.832650827243924 - timing_s/step:70.63796724379063 - timing_s/stop_profile:0.00011838413774967194 - timing_per_token_ms/update_actor:0.05639978771818421 - timing_per_token_ms/gen:0.46562640805120126 - timing_per_token_ms/adv:0.0007712511258590079 - perf/total_num_tokens:192069 - perf/time_per_step:70.63796724379063 - perf/throughput:339.8827278981539 (TaskRunner pid=2049544) Training Progress: 11%|█ | 11/100 [07:58<1:32:18, 62.23s/it] (TaskRunner pid=2049544) step:12 - global_seqlen/min:18163 - global_seqlen/max:27095 - global_seqlen/minmax_diff:8932 - global_seqlen/balanced_min:23218 - global_seqlen/balanced_max:23238 - global_seqlen/mean:23234.125 - actor/entropy:0.11550334095954895 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4828122854232788 - training/rollout_probs_diff_mean:0.0033159777522087097 - training/rollout_probs_diff_std:0.012482362799346447 - training/rollout_actor_probs_pearson_corr:0.9973734021186829 - rollout_corr/rollout_is_mean:0.9999693036079407 - rollout_corr/rollout_is_ratio_fraction_high:5.898561721551232e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010111819574376568 - rollout_corr/rollout_is_max:2.5878732204437256 - rollout_corr/rollout_is_min:0.125954732298851 - rollout_corr/rollout_is_std:0.03412209078669548 - rollout_corr/rollout_is_eff_sample_size:0.9988369179946218 - rollout_corr/rollout_is_seq_mean:1.0000054836273193 - rollout_corr/rollout_is_seq_std:0.0017772279679775238 - rollout_corr/rollout_is_seq_max:1.008151888847351 - rollout_corr/rollout_is_seq_min:0.9952659010887146 - rollout_corr/rollout_is_seq_max_deviation:0.008151888847351074 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1121750511229038) - rollout_corr/training_log_ppl:np.float64(0.10409450641964213) - rollout_corr/kl:np.float64(0.0006273885898906428) - rollout_corr/k3_kl:np.float64(0.000663624501786586) - rollout_corr/rollout_ppl:np.float64(1.1114575085230172) - rollout_corr/rollout_log_ppl:np.float64(0.10346711749298265) - rollout_corr/log_ppl_diff:np.float64(0.0006273889266594779) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014779563243791927) - rollout_corr/log_ppl_diff_max:np.float64(0.006399750709533691) - rollout_corr/log_ppl_diff_min:np.float64(-0.006532169878482819) - rollout_corr/ppl_ratio:np.float64(1.000629240879789) - rollout_corr/chi2_token:np.float64(0.0014743746723979712) - rollout_corr/chi2_seq:np.float64(1.0935420689638704) - actor/pg_loss:np.float64(0.00015071697998791933) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004757110639275197) - actor/ppo_kl:np.float64(2.2654555305479107e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2736458256840706) - perf/mfu/actor:np.float64(0.0671891188671343) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.26230239868164) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0065000043250620365 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0065000043250620365 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:463.56640625 - response_length/max:1365.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:463.56640625 - response_length_non_aborted/max:1365.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.5 - prompt_length/max:364.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.028241038322449e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0592438988387585) - timing_s/agent_loop/generate_sequences/max:np.float64(10.160819062963128) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.8243368406328955) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.160819062963128) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1365 - timing_s/gen:15.931704020127654 - timing_s/reward:0.06278514675796032 - timing_s/old_log_prob:2.4699926115572453 - timing_s/adv:0.16309965774416924 - timing_s/update_actor:10.090087844058871 - timing_s/step:28.80461708456278 - timing_s/stop_profile:0.0001237243413925171 - timing_per_token_ms/update_actor:0.054284849569646326 - timing_per_token_ms/gen:0.13424876779155878 - timing_per_token_ms/adv:0.0008774790192452332 - perf/total_num_tokens:185873 - perf/time_per_step:28.80461708456278 - perf/throughput:806.6111391722626 (TaskRunner pid=2049544) Training Progress: 12%|█▏ | 12/100 [08:26<1:16:22, 52.08s/it] (TaskRunner pid=2049544) step:13 - global_seqlen/min:20734 - global_seqlen/max:28493 - global_seqlen/minmax_diff:7759 - global_seqlen/balanced_min:23965 - global_seqlen/balanced_max:23977 - global_seqlen/mean:23970.125 - actor/entropy:0.1322651505470276 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5841533541679382 - training/rollout_probs_diff_mean:0.003578777192160487 - training/rollout_probs_diff_std:0.01281787734478712 - training/rollout_actor_probs_pearson_corr:0.997634768486023 - rollout_corr/rollout_is_mean:0.9998666048049927 - rollout_corr/rollout_is_ratio_fraction_high:8.307234566018451e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.00010799404844874516 - rollout_corr/rollout_is_max:2.1130197048187256 - rollout_corr/rollout_is_min:0.289154976606369 - rollout_corr/rollout_is_std:0.03564285859465599 - rollout_corr/rollout_is_eff_sample_size:0.9987307228760026 - rollout_corr/rollout_is_seq_mean:0.9998607635498047 - rollout_corr/rollout_is_seq_std:0.0017514645587652922 - rollout_corr/rollout_is_seq_max:1.0048136711120605 - rollout_corr/rollout_is_seq_min:0.9948503971099854 - rollout_corr/rollout_is_seq_max_deviation:0.0051496028900146484 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1441420195624232) - rollout_corr/training_log_ppl:np.float64(0.12973640582640655) - rollout_corr/kl:np.float64(0.0009200233825930049) - rollout_corr/k3_kl:np.float64(0.000771395374357553) - rollout_corr/rollout_ppl:np.float64(1.1430599559098482) - rollout_corr/rollout_log_ppl:np.float64(0.12881638239923632) - rollout_corr/log_ppl_diff:np.float64(0.0009200234271702357) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016153893011505716) - rollout_corr/log_ppl_diff_max:np.float64(0.010122627019882202) - rollout_corr/log_ppl_diff_min:np.float64(-0.007734514772891998) - rollout_corr/ppl_ratio:np.float64(1.0009225921239704) - rollout_corr/chi2_token:np.float64(0.0012212155852466822) - rollout_corr/chi2_seq:np.float64(0.878853014903143) - actor/pg_loss:np.float64(-4.01295255869627e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007891653215210681) - actor/ppo_kl:np.float64(0.00011588802720652325) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.40201497077941895) - perf/mfu/actor:np.float64(0.0691041251464859) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.31333541870117) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005807795561850071 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005807795561850071 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:470.22265625 - response_length/max:1333.0 - response_length/min:126.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:470.22265625 - response_length_non_aborted/max:1333.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.84375 - prompt_length/max:363.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014058127999305725 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4221638720482588) - timing_s/agent_loop/generate_sequences/max:np.float64(9.880332147702575) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.85110625057132) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.880332147702575) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:301 - timing_s/agent_loop/slowest/response_length:1333 - timing_s/gen:15.408700393512845 - timing_s/reward:0.062333619222044945 - timing_s/old_log_prob:2.5665055103600025 - timing_s/adv:0.1314475554972887 - timing_s/update_actor:10.066378388553858 - timing_s/step:28.321861196309328 - timing_s/stop_profile:0.00011406466364860535 - timing_per_token_ms/update_actor:0.05249439869709616 - timing_per_token_ms/gen:0.12800369168124182 - timing_per_token_ms/adv:0.0006854759596439772 - perf/total_num_tokens:191761 - perf/time_per_step:28.321861196309328 - perf/throughput:846.3470968187496 (TaskRunner pid=2049544) Training Progress: 13%|█▎ | 13/100 [08:55<1:05:06, 44.90s/it] (TaskRunner pid=2049544) step:14 - global_seqlen/min:17304 - global_seqlen/max:31074 - global_seqlen/minmax_diff:13770 - global_seqlen/balanced_min:23194 - global_seqlen/balanced_max:23602 - global_seqlen/mean:23258.875 - actor/entropy:0.11846989393234253 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45436424016952515 - training/rollout_probs_diff_mean:0.003607378341257572 - training/rollout_probs_diff_std:0.013258584775030613 - training/rollout_actor_probs_pearson_corr:0.9971495866775513 - rollout_corr/rollout_is_mean:0.9998986721038818 - rollout_corr/rollout_is_ratio_fraction_high:5.1646224164869636e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014633095997851342 - rollout_corr/rollout_is_max:2.4299769401550293 - rollout_corr/rollout_is_min:0.09816896170377731 - rollout_corr/rollout_is_std:0.037202417850494385 - rollout_corr/rollout_is_eff_sample_size:0.9986175363719597 - rollout_corr/rollout_is_seq_mean:0.9998643398284912 - rollout_corr/rollout_is_seq_std:0.002263482892885804 - rollout_corr/rollout_is_seq_max:1.00752854347229 - rollout_corr/rollout_is_seq_min:0.9923214316368103 - rollout_corr/rollout_is_seq_max_deviation:0.007678568363189697 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.13216158375144) - rollout_corr/training_log_ppl:np.float64(0.12105548306863056) - rollout_corr/kl:np.float64(0.0011141863628356674) - rollout_corr/k3_kl:np.float64(0.0007862591612095571) - rollout_corr/rollout_ppl:np.float64(1.1308507332578301) - rollout_corr/rollout_log_ppl:np.float64(0.1199412965834199) - rollout_corr/log_ppl_diff:np.float64(0.0011141864852106664) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018289975014340598) - rollout_corr/log_ppl_diff_max:np.float64(0.01189817488193512) - rollout_corr/log_ppl_diff_min:np.float64(-0.006560362875461578) - rollout_corr/ppl_ratio:np.float64(1.0011174657847732) - rollout_corr/chi2_token:np.float64(0.0008673113770782948) - rollout_corr/chi2_seq:np.float64(0.27274479530751705) - actor/pg_loss:np.float64(8.668552618473768e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010782241606648313) - actor/ppo_kl:np.float64(0.00016768844109904535) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.42187249660491943) - perf/mfu/actor:np.float64(0.06630021670401551) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.41584777832031) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009940821677446365 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009940821677446365 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:453.80859375 - response_length/max:1963.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:453.80859375 - response_length_non_aborted/max:1963.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.03125 - prompt_length/max:375.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013650394976139069 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0465375203639269) - timing_s/agent_loop/generate_sequences/max:np.float64(12.73308464512229) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.650266676486353) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.73308464512229) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:301 - timing_s/agent_loop/slowest/response_length:1963 - timing_s/gen:18.954351030290127 - timing_s/reward:0.06252953223884106 - timing_s/old_log_prob:2.473697155714035 - timing_s/adv:0.14756855927407742 - timing_s/update_actor:9.925982998684049 - timing_s/step:31.649879788979888 - timing_s/stop_profile:0.00011733733117580414 - timing_per_token_ms/update_actor:0.05334513706426068 - timing_per_token_ms/gen:0.16315344119036046 - timing_per_token_ms/adv:0.0007930766173884023 - perf/total_num_tokens:186071 - perf/time_per_step:31.649879788979888 - perf/throughput:734.8803583165097 (TaskRunner pid=2049544) Training Progress: 14%|█▍ | 14/100 [09:26<58:38, 40.91s/it] (TaskRunner pid=2049544) step:15 - global_seqlen/min:15342 - global_seqlen/max:29838 - global_seqlen/minmax_diff:14496 - global_seqlen/balanced_min:23634 - global_seqlen/balanced_max:23672 - global_seqlen/mean:23657.75 - actor/entropy:0.1110260859131813 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9913476705551147 - training/rollout_probs_diff_mean:0.0030704238452017307 - training/rollout_probs_diff_std:0.01288246363401413 - training/rollout_actor_probs_pearson_corr:0.9973083138465881 - rollout_corr/rollout_is_mean:0.9999945163726807 - rollout_corr/rollout_is_ratio_fraction_high:3.303655466879718e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.085052442969754e-05 - rollout_corr/rollout_is_max:3.397390604019165 - rollout_corr/rollout_is_min:0.008578135631978512 - rollout_corr/rollout_is_std:0.03370378911495209 - rollout_corr/rollout_is_eff_sample_size:0.9988653435875123 - rollout_corr/rollout_is_seq_mean:0.9999395608901978 - rollout_corr/rollout_is_seq_std:0.0021285165566951036 - rollout_corr/rollout_is_seq_max:1.016195297241211 - rollout_corr/rollout_is_seq_min:0.9913737177848816 - rollout_corr/rollout_is_seq_max_deviation:0.016195297241210938 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.116935707628727) - rollout_corr/training_log_ppl:np.float64(0.10718949149304535) - rollout_corr/kl:np.float64(0.0008361445555777891) - rollout_corr/k3_kl:np.float64(0.0006998892279455049) - rollout_corr/rollout_ppl:np.float64(1.1159698073752224) - rollout_corr/rollout_log_ppl:np.float64(0.10635334601101931) - rollout_corr/log_ppl_diff:np.float64(0.0008361454820260406) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016929644480114803) - rollout_corr/log_ppl_diff_max:np.float64(0.01026906818151474) - rollout_corr/log_ppl_diff_min:np.float64(-0.012994319200515747) - rollout_corr/ppl_ratio:np.float64(1.000839062500745) - rollout_corr/chi2_token:np.float64(0.0011396266054362059) - rollout_corr/chi2_seq:np.float64(1.5898604474496096) - actor/pg_loss:np.float64(-2.7647125534713268e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006954610362299718) - actor/ppo_kl:np.float64(0.00011087026399003719) - actor/pg_clipfrac_lower:np.float64(1.0444518920849077e-05) - actor/grad_norm:np.float64(0.29682732000947) - perf/mfu/actor:np.float64(0.06723820699139785) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.3052020072937) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0049513536505401134 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0049513536505401134 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:472.9609375 - response_length/max:1790.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:472.9609375 - response_length_non_aborted/max:1790.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:460.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010203756392002106 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7883450742810965) - timing_s/agent_loop/generate_sequences/max:np.float64(12.01114228926599) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.683357301946671) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.01114228926599) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:1790 - timing_s/gen:17.793763728812337 - timing_s/reward:0.06301617436110973 - timing_s/old_log_prob:2.4648690931499004 - timing_s/adv:0.15438295900821686 - timing_s/update_actor:10.23603860847652 - timing_s/step:30.79816811159253 - timing_s/stop_profile:0.00011844001710414886 - timing_per_token_ms/update_actor:0.05408396090327969 - timing_per_token_ms/gen:0.14696116328988204 - timing_per_token_ms/adv:0.0008157102799728253 - perf/total_num_tokens:189262 - perf/time_per_step:30.79816811159253 - perf/throughput:768.1544536765856 (TaskRunner pid=2049544) Training Progress: 15%|█▌ | 15/100 [09:57<53:39, 37.88s/it] (TaskRunner pid=2049544) step:16 - global_seqlen/min:18400 - global_seqlen/max:27667 - global_seqlen/minmax_diff:9267 - global_seqlen/balanced_min:22148 - global_seqlen/balanced_max:22154 - global_seqlen/mean:22151.125 - actor/entropy:0.12429671734571457 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.739967942237854 - training/rollout_probs_diff_mean:0.0033937261905521154 - training/rollout_probs_diff_std:0.012556870467960835 - training/rollout_actor_probs_pearson_corr:0.9975789189338684 - rollout_corr/rollout_is_mean:1.000154972076416 - rollout_corr/rollout_is_ratio_fraction_high:2.7087302441941574e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.126190368784592e-05 - rollout_corr/rollout_is_max:4.322726726531982 - rollout_corr/rollout_is_min:0.12595464289188385 - rollout_corr/rollout_is_std:0.034952208399772644 - rollout_corr/rollout_is_eff_sample_size:0.9987801905807853 - rollout_corr/rollout_is_seq_mean:1.0001484155654907 - rollout_corr/rollout_is_seq_std:0.0019322986481711268 - rollout_corr/rollout_is_seq_max:1.007652997970581 - rollout_corr/rollout_is_seq_min:0.992692768573761 - rollout_corr/rollout_is_seq_max_deviation:0.007652997970581055 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.128569976426661) - rollout_corr/training_log_ppl:np.float64(0.11689590534660965) - rollout_corr/kl:np.float64(0.0006918034323035727) - rollout_corr/k3_kl:np.float64(0.0007205118381676812) - rollout_corr/rollout_ppl:np.float64(1.1277636121958494) - rollout_corr/rollout_log_ppl:np.float64(0.11620410031173378) - rollout_corr/log_ppl_diff:np.float64(0.0006918050348758698) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016668791504343972) - rollout_corr/log_ppl_diff_max:np.float64(0.011440090835094452) - rollout_corr/log_ppl_diff_min:np.float64(-0.005701005458831787) - rollout_corr/ppl_ratio:np.float64(1.000694399466738) - rollout_corr/chi2_token:np.float64(0.0014990693889558315) - rollout_corr/chi2_seq:np.float64(0.6086118693929166) - actor/pg_loss:np.float64(3.552588168531656e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008225217134167906) - actor/ppo_kl:np.float64(0.0001884127358140475) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3684859052300453) - perf/mfu/actor:np.float64(0.05525549139638902) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(118.55257987976074) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014105712994933128 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014105712994933128 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:432.62890625 - response_length/max:1203.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:432.62890625 - response_length_non_aborted/max:1203.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.59375 - prompt_length/max:344.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.934604167938232e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0808530896902084) - timing_s/agent_loop/generate_sequences/max:np.float64(8.854041801765561) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.365803626016714) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.854041801765561) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:334 - timing_s/agent_loop/slowest/response_length:1203 - timing_s/gen:14.700905803591013 - timing_s/reward:0.06069855950772762 - timing_s/old_log_prob:2.4508451614528894 - timing_s/adv:0.14457714557647705 - timing_s/update_actor:11.594925744459033 - timing_s/step:29.03880488872528 - timing_s/stop_profile:0.00011275149881839752 - timing_per_token_ms/update_actor:0.0654307949622143 - timing_per_token_ms/gen:0.13273596023214732 - timing_per_token_ms/adv:0.0008158566753182798 - perf/total_num_tokens:177209 - perf/time_per_step:29.03880488872528 - perf/throughput:762.8111792093924 (TaskRunner pid=2049544) Training Progress: 16%|█▌ | 16/100 [10:26<49:20, 35.25s/it] (TaskRunner pid=2049544) step:17 - global_seqlen/min:17199 - global_seqlen/max:28844 - global_seqlen/minmax_diff:11645 - global_seqlen/balanced_min:22404 - global_seqlen/balanced_max:28744 - global_seqlen/mean:23204.125 - actor/entropy:0.12878382205963135 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7316876649856567 - training/rollout_probs_diff_mean:0.003660086076706648 - training/rollout_probs_diff_std:0.013964627869427204 - training/rollout_actor_probs_pearson_corr:0.9971086978912354 - rollout_corr/rollout_is_mean:0.9999353289604187 - rollout_corr/rollout_is_ratio_fraction_high:5.108600453240797e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.000221372683881782 - rollout_corr/rollout_is_max:3.039975881576538 - rollout_corr/rollout_is_min:0.002080337144434452 - rollout_corr/rollout_is_std:0.037413306534290314 - rollout_corr/rollout_is_eff_sample_size:0.9986019633462979 - rollout_corr/rollout_is_seq_mean:0.9999227523803711 - rollout_corr/rollout_is_seq_std:0.0023962773848325014 - rollout_corr/rollout_is_seq_max:1.0085341930389404 - rollout_corr/rollout_is_seq_min:0.9915060997009277 - rollout_corr/rollout_is_seq_max_deviation:0.00853419303894043 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1416860399767756) - rollout_corr/training_log_ppl:np.float64(0.1297323260405392) - rollout_corr/kl:np.float64(0.0012029753426432421) - rollout_corr/k3_kl:np.float64(0.0010000968625991646) - rollout_corr/rollout_ppl:np.float64(1.1402671793475747) - rollout_corr/rollout_log_ppl:np.float64(0.12852934912734781) - rollout_corr/log_ppl_diff:np.float64(0.0012029769131913781) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022540156060131267) - rollout_corr/log_ppl_diff_max:np.float64(0.022689037024974823) - rollout_corr/log_ppl_diff_min:np.float64(-0.0066199153661727905) - rollout_corr/ppl_ratio:np.float64(1.0012089894153178) - rollout_corr/chi2_token:np.float64(0.0014632882084697485) - rollout_corr/chi2_seq:np.float64(0.5507866519037634) - actor/pg_loss:np.float64(0.0002590372459962964) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013125582909196964) - actor/ppo_kl:np.float64(0.00025115833264877097) - actor/pg_clipfrac_lower:np.float64(7.70463520893827e-06) - actor/grad_norm:np.float64(0.3952488452196121) - perf/mfu/actor:np.float64(0.06094650606696392) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(110.15214920043945) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.022923780605196953 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.022923780605196953 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:458.78515625 - response_length/max:8192.0 - response_length/min:68.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:458.78515625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:68.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017055682837963104 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8617520779371262) - timing_s/agent_loop/generate_sequences/max:np.float64(50.32327074185014) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.232911160150252) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.32327074185014) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:236 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:57.11839736439288 - timing_s/reward:0.06247155740857124 - timing_s/old_log_prob:2.831888886168599 - timing_s/adv:0.1662355735898018 - timing_s/update_actor:10.9862146563828 - timing_s/step:71.25081997551024 - timing_s/stop_profile:0.00011354684829711914 - timing_per_token_ms/update_actor:0.05918244415800423 - timing_per_token_ms/gen:0.4863251059131442 - timing_per_token_ms/adv:0.0008955065833650363 - perf/total_num_tokens:185633 - perf/time_per_step:71.25081997551024 - perf/throughput:325.6681818956685 (TaskRunner pid=2049544) Training Progress: 17%|█▋ | 17/100 [11:38<1:03:45, 46.09s/it] (TaskRunner pid=2049544) step:18 - global_seqlen/min:17313 - global_seqlen/max:30331 - global_seqlen/minmax_diff:13018 - global_seqlen/balanced_min:25566 - global_seqlen/balanced_max:25574 - global_seqlen/mean:25570.75 - actor/entropy:0.1224943995475769 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5235333442687988 - training/rollout_probs_diff_mean:0.0032062120735645294 - training/rollout_probs_diff_std:0.012459194287657738 - training/rollout_actor_probs_pearson_corr:0.9975793361663818 - rollout_corr/rollout_is_mean:0.9999059438705444 - rollout_corr/rollout_is_ratio_fraction_high:7.298523996723816e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012407491158228368 - rollout_corr/rollout_is_max:2.7950527667999268 - rollout_corr/rollout_is_min:0.06372947990894318 - rollout_corr/rollout_is_std:0.03364891931414604 - rollout_corr/rollout_is_eff_sample_size:0.9988689117678543 - rollout_corr/rollout_is_seq_mean:0.9998814463615417 - rollout_corr/rollout_is_seq_std:0.0018376336665824056 - rollout_corr/rollout_is_seq_max:1.007022738456726 - rollout_corr/rollout_is_seq_min:0.9937419891357422 - rollout_corr/rollout_is_seq_max_deviation:0.007022738456726074 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1279504764825106) - rollout_corr/training_log_ppl:np.float64(0.11757075608329615) - rollout_corr/kl:np.float64(0.0008173328948331005) - rollout_corr/k3_kl:np.float64(0.0006817538003787149) - rollout_corr/rollout_ppl:np.float64(1.1270086439326406) - rollout_corr/rollout_log_ppl:np.float64(0.1167534222076938) - rollout_corr/log_ppl_diff:np.float64(0.000817333875602344) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016419640196545515) - rollout_corr/log_ppl_diff_max:np.float64(0.008472427725791931) - rollout_corr/log_ppl_diff_min:np.float64(-0.006251677870750427) - rollout_corr/ppl_ratio:np.float64(1.0008197950664908) - rollout_corr/chi2_token:np.float64(0.0010528604034334421) - rollout_corr/chi2_seq:np.float64(0.8530804882757366) - actor/pg_loss:np.float64(0.0001363652991130948) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009771198135695158) - actor/ppo_kl:np.float64(8.36708865672442e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4684727042913437) - perf/mfu/actor:np.float64(0.0710728954199909) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.35284042358398) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.40234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004924679175019264 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004924679175019264 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:535.2109375 - response_length/max:1581.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:535.2109375 - response_length_non_aborted/max:1581.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.875 - prompt_length/max:349.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012665800750255585 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8409140948206186) - timing_s/agent_loop/generate_sequences/max:np.float64(11.426093036308885) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.7903019126315485) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.426093036308885) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1581 - timing_s/gen:17.257411571219563 - timing_s/reward:0.06468412093818188 - timing_s/old_log_prob:2.512243326753378 - timing_s/adv:0.16402030177414417 - timing_s/update_actor:10.535967061296105 - timing_s/step:30.621809538453817 - timing_s/stop_profile:0.00020102225244045258 - timing_per_token_ms/update_actor:0.05150399900910271 - timing_per_token_ms/gen:0.1259536366445733 - timing_per_token_ms/adv:0.000801796494892329 - perf/total_num_tokens:204566 - perf/time_per_step:30.621809538453817 - perf/throughput:835.0502594527971 (TaskRunner pid=2049544) Training Progress: 18%|█▊ | 18/100 [12:08<56:39, 41.46s/it] (TaskRunner pid=2049544) step:19 - global_seqlen/min:17713 - global_seqlen/max:31095 - global_seqlen/minmax_diff:13382 - global_seqlen/balanced_min:23780 - global_seqlen/balanced_max:23986 - global_seqlen/mean:23818.0 - actor/entropy:0.10652168840169907 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43985795974731445 - training/rollout_probs_diff_mean:0.003031388856470585 - training/rollout_probs_diff_std:0.012130273506045341 - training/rollout_actor_probs_pearson_corr:0.9975038766860962 - rollout_corr/rollout_is_mean:1.000024437904358 - rollout_corr/rollout_is_ratio_fraction_high:5.6943903473438695e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010575295891612768 - rollout_corr/rollout_is_max:3.544985771179199 - rollout_corr/rollout_is_min:0.2985078692436218 - rollout_corr/rollout_is_std:0.03386257588863373 - rollout_corr/rollout_is_eff_sample_size:0.9988546391994405 - rollout_corr/rollout_is_seq_mean:1.0000839233398438 - rollout_corr/rollout_is_seq_std:0.002387973479926586 - rollout_corr/rollout_is_seq_max:1.0142525434494019 - rollout_corr/rollout_is_seq_min:0.9915294051170349 - rollout_corr/rollout_is_seq_max_deviation:0.014252543449401855 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1093668043613434) - rollout_corr/training_log_ppl:np.float64(0.10132705738578807) - rollout_corr/kl:np.float64(0.0006060766052620181) - rollout_corr/k3_kl:np.float64(0.0007317086579945453) - rollout_corr/rollout_ppl:np.float64(1.1086922818794847) - rollout_corr/rollout_log_ppl:np.float64(0.10072098037198884) - rollout_corr/log_ppl_diff:np.float64(0.0006060770137992222) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016289143422909547) - rollout_corr/log_ppl_diff_max:np.float64(0.014358222484588623) - rollout_corr/log_ppl_diff_min:np.float64(-0.009524352848529816) - rollout_corr/ppl_ratio:np.float64(1.0006090987008065) - rollout_corr/chi2_token:np.float64(0.001838616793975234) - rollout_corr/chi2_seq:np.float64(0.987432076362893) - actor/pg_loss:np.float64(0.0002443152479827404) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007472785102891066) - actor/ppo_kl:np.float64(-8.193314219084868e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4798855558037758) - perf/mfu/actor:np.float64(0.0686439488762442) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(105.47739028930664) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009253384196199477 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009253384196199477 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:480.1875 - response_length/max:1871.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:480.1875 - response_length_non_aborted/max:1871.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.125 - prompt_length/max:339.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001460295170545578 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0999032072722912) - timing_s/agent_loop/generate_sequences/max:np.float64(12.776574049144983) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.886515708894876) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.776574049144983) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:309 - timing_s/agent_loop/slowest/response_length:1871 - timing_s/gen:18.46205241046846 - timing_s/reward:0.06260044127702713 - timing_s/old_log_prob:2.540384005755186 - timing_s/adv:0.14328701607882977 - timing_s/update_actor:10.04507009498775 - timing_s/step:31.340227657929063 - timing_s/stop_profile:0.00011619366705417633 - timing_per_token_ms/update_actor:0.05271785044392765 - timing_per_token_ms/gen:0.15018590077499397 - timing_per_token_ms/adv:0.0007519891262848989 - perf/total_num_tokens:190544 - perf/time_per_step:31.340227657929063 - perf/throughput:759.9817161498524 (TaskRunner pid=2049544) Training Progress: 19%|█▉ | 19/100 [12:40<51:52, 38.43s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. We are asked to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the known values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0) \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest option is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10/actor (TaskRunner pid=2049544) step:20 - global_seqlen/min:17554 - global_seqlen/max:28214 - global_seqlen/minmax_diff:10660 - global_seqlen/balanced_min:22414 - global_seqlen/balanced_max:22431 - global_seqlen/mean:22418.0 - actor/entropy:0.11887378990650177 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7652756571769714 - training/rollout_probs_diff_mean:0.003253082511946559 - training/rollout_probs_diff_std:0.01256453339010477 - training/rollout_actor_probs_pearson_corr:0.9974572062492371 - rollout_corr/rollout_is_mean:0.9998491406440735 - rollout_corr/rollout_is_ratio_fraction_high:2.658820267242845e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014180374273564667 - rollout_corr/rollout_is_max:2.1670210361480713 - rollout_corr/rollout_is_min:0.20142976939678192 - rollout_corr/rollout_is_std:0.03435710072517395 - rollout_corr/rollout_is_eff_sample_size:0.9988207434840839 - rollout_corr/rollout_is_seq_mean:0.9999091625213623 - rollout_corr/rollout_is_seq_std:0.001986477058380842 - rollout_corr/rollout_is_seq_max:1.0120420455932617 - rollout_corr/rollout_is_seq_min:0.9940481185913086 - rollout_corr/rollout_is_seq_max_deviation:0.012042045593261719 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1164336544461548) - rollout_corr/training_log_ppl:np.float64(0.10743120299957809) - rollout_corr/kl:np.float64(0.0007887022689292422) - rollout_corr/k3_kl:np.float64(0.0007427555091403804) - rollout_corr/rollout_ppl:np.float64(1.1155504160560668) - rollout_corr/rollout_log_ppl:np.float64(0.1066425001481548) - rollout_corr/log_ppl_diff:np.float64(0.0007887028514232952) - rollout_corr/log_ppl_abs_diff:np.float64(0.001653945950238267) - rollout_corr/log_ppl_diff_max:np.float64(0.012162841856479645) - rollout_corr/log_ppl_diff_min:np.float64(-0.010080903768539429) - rollout_corr/ppl_ratio:np.float64(1.0007913769222796) - rollout_corr/chi2_token:np.float64(0.0014412584714591503) - rollout_corr/chi2_seq:np.float64(0.749183313222602) - actor/pg_loss:np.float64(0.00014212285168468952) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000699050789080502) - actor/ppo_kl:np.float64(5.158489911716302e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.37614476680755615) - perf/mfu/actor:np.float64(0.06503999971026932) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(105.41239166259766) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.61328125) - val-aux/sciknoweval/reward/std@16:np.float64(0.18125889304909915) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6852) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15465207513909812) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5414125) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14903691503963334) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6131875000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.18035006026777042) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7420375) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.12007545620567092) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.482225) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1146577346150142) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6277625) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.14023131776921524) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.78975) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09577876488826616) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.43423750000000005) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07714076320360426) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6336375000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09981662846229708) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8320624999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.07157349089044285) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4047375) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04145535225486681) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6367624999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.0671080518080989) - val-aux/sciknoweval/score/mean@16:np.float64(0.61328125) - val-aux/sciknoweval/score/std@16:np.float64(0.18125889304909915) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6852) - val-aux/sciknoweval/score/best@2/std:np.float64(0.15465207513909812) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5414125) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.14903691503963334) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6131875000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.18035006026777042) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7420375) - val-aux/sciknoweval/score/best@4/std:np.float64(0.12007545620567092) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.482225) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1146577346150142) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6277625) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.14023131776921524) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.78975) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09577876488826616) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.43423750000000005) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07714076320360426) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6336375000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09981662846229708) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8320624999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.07157349089044285) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4047375) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04145535225486681) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6367624999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.0671080518080989) - val-core/sciknoweval/acc/mean@16:np.float64(0.61328125) - val-aux/sciknoweval/acc/std@16:np.float64(0.18125889304909915) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6852) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.15465207513909812) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5414125) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.14903691503963334) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6131875000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.18035006026777042) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7420375) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.12007545620567092) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.482225) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1146577346150142) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6277625) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.14023131776921524) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.78975) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09577876488826616) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.43423750000000005) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07714076320360426) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6336375000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09981662846229708) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8320624999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.07157349089044285) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4047375) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04145535225486681) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6367624999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.0671080518080989) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.990625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.030556568705962538) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9989750000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.010087512303978256) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9817375) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.040950720392224096) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.990325) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.03109914326301299) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9670625000000002) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.050240658088205936) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99635) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.01916044898238421) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9449) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.05467743751202757) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.999125) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.008915794300729528) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.918325) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.047861241896262015) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.999925) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0016371701590179605) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00229323236271739 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00229323236271739 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:440.75 - response_length/max:1682.0 - response_length/min:116.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:440.75 - response_length_non_aborted/max:1682.0 - response_length_non_aborted/min:116.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.8125 - prompt_length/max:355.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.746236562728882e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.339398443698883) - timing_s/agent_loop/generate_sequences/max:np.float64(11.241939969360828) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.378861661745759) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.241939969360828) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:341 - timing_s/agent_loop/slowest/response_length:1682 - timing_s/gen:17.086866164579988 - timing_s/reward:0.061425795778632164 - timing_s/old_log_prob:2.430392887443304 - timing_s/adv:0.2020460832864046 - timing_s/update_actor:9.794211328029633 - timing_s/step:29.6614922080189 - timing_s/testing:97.09009108878672 - timing_s/save_checkpoint:7.344661565497518 - timing_s/stop_profile:0.00012345612049102783 - timing_per_token_ms/update_actor:0.054611313052177 - timing_per_token_ms/gen:0.15143634930321173 - timing_per_token_ms/adv:0.001126584013328601 - perf/total_num_tokens:179344 - perf/time_per_step:29.6614922080189 - perf/throughput:755.7947470336425 (TaskRunner pid=2049544) Training Progress: 20%|██ | 20/100 [14:54<1:29:33, 67.17s/it] (TaskRunner pid=2049544) step:21 - global_seqlen/min:20890 - global_seqlen/max:29372 - global_seqlen/minmax_diff:8482 - global_seqlen/balanced_min:23784 - global_seqlen/balanced_max:24194 - global_seqlen/mean:23842.25 - actor/entropy:0.10979560017585754 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7417163848876953 - training/rollout_probs_diff_mean:0.0029196166433393955 - training/rollout_probs_diff_std:0.01234018336981535 - training/rollout_actor_probs_pearson_corr:0.9974682927131653 - rollout_corr/rollout_is_mean:1.0000228881835938 - rollout_corr/rollout_is_ratio_fraction_high:1.6375992345274426e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010644395661074668 - rollout_corr/rollout_is_max:2.094970703125 - rollout_corr/rollout_is_min:0.07465586066246033 - rollout_corr/rollout_is_std:0.03303755819797516 - rollout_corr/rollout_is_eff_sample_size:0.9989098287246259 - rollout_corr/rollout_is_seq_mean:0.9999141693115234 - rollout_corr/rollout_is_seq_std:0.0017565247835591435 - rollout_corr/rollout_is_seq_max:1.0051144361495972 - rollout_corr/rollout_is_seq_min:0.9947226643562317 - rollout_corr/rollout_is_seq_max_deviation:0.0052773356437683105 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1051466539502144) - rollout_corr/training_log_ppl:np.float64(0.09741823082003975) - rollout_corr/kl:np.float64(0.0006470407322218819) - rollout_corr/k3_kl:np.float64(0.0006291852859376945) - rollout_corr/rollout_ppl:np.float64(1.1044287369586527) - rollout_corr/rollout_log_ppl:np.float64(0.09677118871331913) - rollout_corr/log_ppl_diff:np.float64(0.0006470421067206189) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014577525143977255) - rollout_corr/log_ppl_diff_max:np.float64(0.009566932916641235) - rollout_corr/log_ppl_diff_min:np.float64(-0.005102425813674927) - rollout_corr/ppl_ratio:np.float64(1.0006491616368294) - rollout_corr/chi2_token:np.float64(0.0012274319306015968) - rollout_corr/chi2_seq:np.float64(1.2051882504019886) - actor/pg_loss:np.float64(0.00013444910291582346) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00045100462170921674) - actor/ppo_kl:np.float64(-5.319785113400144e-05) - actor/pg_clipfrac_lower:np.float64(2.38095403801708e-05) - actor/grad_norm:np.float64(0.2096654698252678) - perf/mfu/actor:np.float64(0.0669203788543201) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(105.66231536865234) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004104233346879482 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004104233346879482 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:477.0703125 - response_length/max:2091.0 - response_length/min:152.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:477.0703125 - response_length_non_aborted/max:2091.0 - response_length_non_aborted/min:152.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.0 - prompt_length/max:364.0 - prompt_length/min:195.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015165284276008606 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7176804710179567) - timing_s/agent_loop/generate_sequences/max:np.float64(12.88374364003539) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.737000476088724) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.88374364003539) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:2091 - timing_s/gen:19.051039807498455 - timing_s/reward:0.05670631118118763 - timing_s/old_log_prob:2.6326032262295485 - timing_s/adv:0.14211471565067768 - timing_s/update_actor:10.165599260479212 - timing_s/step:32.148173009976745 - timing_s/stop_profile:0.000135812908411026 - timing_per_token_ms/update_actor:0.053296140572299235 - timing_per_token_ms/gen:0.1559898453082654 - timing_per_token_ms/adv:0.000745078147252659 - perf/total_num_tokens:190738 - perf/time_per_step:32.148173009976745 - perf/throughput:741.6362352100347 (TaskRunner pid=2049544) Training Progress: 21%|██ | 21/100 [15:26<1:14:37, 56.68s/it] (TaskRunner pid=2049544) step:22 - global_seqlen/min:16900 - global_seqlen/max:32996 - global_seqlen/minmax_diff:16096 - global_seqlen/balanced_min:23691 - global_seqlen/balanced_max:23913 - global_seqlen/mean:23732.375 - actor/entropy:0.12188427150249481 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5312178134918213 - training/rollout_probs_diff_mean:0.0033468599431216717 - training/rollout_probs_diff_std:0.013181475922465324 - training/rollout_actor_probs_pearson_corr:0.9972920417785645 - rollout_corr/rollout_is_mean:0.999908983707428 - rollout_corr/rollout_is_ratio_fraction_high:2.4285402105306275e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001942832168424502 - rollout_corr/rollout_is_max:3.412278175354004 - rollout_corr/rollout_is_min:0.14518168568611145 - rollout_corr/rollout_is_std:0.03405214846134186 - rollout_corr/rollout_is_eff_sample_size:0.9988416753032981 - rollout_corr/rollout_is_seq_mean:0.999866247177124 - rollout_corr/rollout_is_seq_std:0.00229041394777596 - rollout_corr/rollout_is_seq_max:1.0077131986618042 - rollout_corr/rollout_is_seq_min:0.9842400550842285 - rollout_corr/rollout_is_seq_max_deviation:0.015759944915771484 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1243417421355844) - rollout_corr/training_log_ppl:np.float64(0.11367632360634161) - rollout_corr/kl:np.float64(0.0010524871724442164) - rollout_corr/k3_kl:np.float64(0.000926966261786788) - rollout_corr/rollout_ppl:np.float64(1.1231554276309907) - rollout_corr/rollout_log_ppl:np.float64(0.11262383543362375) - rollout_corr/log_ppl_diff:np.float64(0.001052488172717858) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018730249648797326) - rollout_corr/log_ppl_diff_max:np.float64(0.0278562530875206) - rollout_corr/log_ppl_diff_min:np.float64(-0.004908844828605652) - rollout_corr/ppl_ratio:np.float64(1.0010584215633571) - rollout_corr/chi2_token:np.float64(0.0015142448246479034) - rollout_corr/chi2_seq:np.float64(0.5919819474220276) - actor/pg_loss:np.float64(-3.6185141652822495e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007750144300189277) - actor/ppo_kl:np.float64(0.00017040614556229627) - actor/pg_clipfrac_lower:np.float64(5.271592272038106e-06) - actor/grad_norm:np.float64(0.4317408576607704) - perf/mfu/actor:np.float64(0.06862934905562151) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(105.66873407363892) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0038765573408454657 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0038765573408454657 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:482.54296875 - response_length/max:1955.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:482.54296875 - response_length_non_aborted/max:1955.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.09375 - prompt_length/max:375.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013677775859832764 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3632201459258795) - timing_s/agent_loop/generate_sequences/max:np.float64(12.304583990946412) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.897737120372767) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.304583990946412) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:1955 - timing_s/gen:18.77624169550836 - timing_s/reward:0.054932184517383575 - timing_s/old_log_prob:2.5277608297765255 - timing_s/adv:0.14852365665137768 - timing_s/update_actor:10.204811798408628 - timing_s/step:31.823464412242174 - timing_s/stop_profile:0.0001289471983909607 - timing_per_token_ms/update_actor:0.05374942351117739 - timing_per_token_ms/gen:0.15199619282211235 - timing_per_token_ms/adv:0.0007822839931284673 - perf/total_num_tokens:189859 - perf/time_per_step:31.823464412242174 - perf/throughput:745.7508300344066 (TaskRunner pid=2049544) Training Progress: 22%|██▏ | 22/100 [15:58<1:04:00, 49.24s/it] (TaskRunner pid=2049544) step:23 - global_seqlen/min:17606 - global_seqlen/max:33708 - global_seqlen/minmax_diff:16102 - global_seqlen/balanced_min:26856 - global_seqlen/balanced_max:26886 - global_seqlen/mean:26877.375 - actor/entropy:0.1653793752193451 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9992972612380981 - training/rollout_probs_diff_mean:0.003593530971556902 - training/rollout_probs_diff_std:0.01221021730452776 - training/rollout_actor_probs_pearson_corr:0.9982529878616333 - rollout_corr/rollout_is_mean:1.0001059770584106 - rollout_corr/rollout_is_ratio_fraction_high:2.7500669602886774e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.18765116087161e-05 - rollout_corr/rollout_is_max:3.486543655395508 - rollout_corr/rollout_is_min:0.00043065936188213527 - rollout_corr/rollout_is_std:0.034376177936792374 - rollout_corr/rollout_is_eff_sample_size:0.9988199109866871 - rollout_corr/rollout_is_seq_mean:1.0000687837600708 - rollout_corr/rollout_is_seq_std:0.0017359366174787283 - rollout_corr/rollout_is_seq_max:1.0058130025863647 - rollout_corr/rollout_is_seq_min:0.9936341047286987 - rollout_corr/rollout_is_seq_max_deviation:0.0063658952713012695 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1735385786741972) - rollout_corr/training_log_ppl:np.float64(0.1555803755109082) - rollout_corr/kl:np.float64(0.0008435647158400172) - rollout_corr/k3_kl:np.float64(0.000810335308230492) - rollout_corr/rollout_ppl:np.float64(1.172548221424222) - rollout_corr/rollout_log_ppl:np.float64(0.15473681054572808) - rollout_corr/log_ppl_diff:np.float64(0.0008435649651801214) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016919265181059018) - rollout_corr/log_ppl_diff_max:np.float64(0.013718657195568085) - rollout_corr/log_ppl_diff_min:np.float64(-0.005128346383571625) - rollout_corr/ppl_ratio:np.float64(1.000846435315907) - rollout_corr/chi2_token:np.float64(0.001545611536130309) - rollout_corr/chi2_seq:np.float64(0.7387706565205008) - actor/pg_loss:np.float64(-7.304479368031025e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014301339112989808) - actor/ppo_kl:np.float64(0.00022386275868435668) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4195772558450699) - perf/mfu/actor:np.float64(0.07791174639008124) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.34968948364258) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:1 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014247066341340542 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014247066341340542 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:568.16796875 - response_length/max:1852.0 - response_length/min:102.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:568.16796875 - response_length_non_aborted/max:1852.0 - response_length_non_aborted/min:102.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.75 - prompt_length/max:375.0 - prompt_length/min:203.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003154464066028595 - timing_s/agent_loop/generate_sequences/min:np.float64(1.197395345196128) - timing_s/agent_loop/generate_sequences/max:np.float64(13.042586963623762) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.787129377022211) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.042586963623762) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:1852 - timing_s/gen:18.867352159693837 - timing_s/reward:0.06311722472310066 - timing_s/old_log_prob:2.6495919916778803 - timing_s/adv:0.15099932067096233 - timing_s/update_actor:9.910499073565006 - timing_s/step:31.743287844583392 - timing_s/stop_profile:0.00010893121361732483 - timing_per_token_ms/update_actor:0.046091271346090375 - timing_per_token_ms/gen:0.12971620793046343 - timing_per_token_ms/adv:0.0007022603615074126 - perf/total_num_tokens:215019 - perf/time_per_step:31.743287844583392 - perf/throughput:846.7104961399359 (TaskRunner pid=2049544) Training Progress: 23%|██▎ | 23/100 [16:30<56:36, 44.11s/it] (TaskRunner pid=2049544) step:24 - global_seqlen/min:18041 - global_seqlen/max:30779 - global_seqlen/minmax_diff:12738 - global_seqlen/balanced_min:24022 - global_seqlen/balanced_max:24069 - global_seqlen/mean:24056.125 - actor/entropy:0.143111914396286 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8866641521453857 - training/rollout_probs_diff_mean:0.003361048409715295 - training/rollout_probs_diff_std:0.012565950863063335 - training/rollout_actor_probs_pearson_corr:0.9978638887405396 - rollout_corr/rollout_is_mean:1.0000112056732178 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00013656703231390566 - rollout_corr/rollout_is_max:1.9401601552963257 - rollout_corr/rollout_is_min:0.01988193951547146 - rollout_corr/rollout_is_std:0.03432759642601013 - rollout_corr/rollout_is_eff_sample_size:0.9988230031268704 - rollout_corr/rollout_is_seq_mean:1.000041127204895 - rollout_corr/rollout_is_seq_std:0.0024783890694379807 - rollout_corr/rollout_is_seq_max:1.0213310718536377 - rollout_corr/rollout_is_seq_min:0.9930093884468079 - rollout_corr/rollout_is_seq_max_deviation:0.021331071853637695 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.144452238921076) - rollout_corr/training_log_ppl:np.float64(0.13059414884628495) - rollout_corr/kl:np.float64(0.00082983940849779) - rollout_corr/k3_kl:np.float64(0.0008679421140129762) - rollout_corr/rollout_ppl:np.float64(1.1434855586849153) - rollout_corr/rollout_log_ppl:np.float64(0.12976430927665206) - rollout_corr/log_ppl_diff:np.float64(0.0008298395696328953) - rollout_corr/log_ppl_abs_diff:np.float64(0.001792395327356644) - rollout_corr/log_ppl_diff_max:np.float64(0.01518431305885315) - rollout_corr/log_ppl_diff_min:np.float64(-0.016911908984184265) - rollout_corr/ppl_ratio:np.float64(1.000833987724036) - rollout_corr/chi2_token:np.float64(0.0022546271793544292) - rollout_corr/chi2_seq:np.float64(1.743649244075641) - actor/pg_loss:np.float64(0.00021742633543908596) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007519598264025262) - actor/ppo_kl:np.float64(9.309058857098762e-05) - actor/pg_clipfrac_lower:np.float64(5.556543328566477e-06) - actor/grad_norm:np.float64(0.3718913719058037) - perf/mfu/actor:np.float64(0.06935155745712235) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.33251953125) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:1 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010280685499310493 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010280685499310493 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:486.25390625 - response_length/max:1963.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:486.25390625 - response_length_non_aborted/max:1963.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.5 - prompt_length/max:383.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013114511966705322 - timing_s/agent_loop/generate_sequences/min:np.float64(1.162827044725418) - timing_s/agent_loop/generate_sequences/max:np.float64(12.901960568502545) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.848180819972185) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.901960568502545) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:257 - timing_s/agent_loop/slowest/response_length:1963 - timing_s/gen:18.82960532233119 - timing_s/reward:0.0641421228647232 - timing_s/old_log_prob:2.5491561722010374 - timing_s/adv:0.1262466385960579 - timing_s/update_actor:10.136537404730916 - timing_s/step:31.79429466277361 - timing_s/stop_profile:0.00012167543172836304 - timing_per_token_ms/update_actor:0.052671291639504056 - timing_per_token_ms/gen:0.15126489442028254 - timing_per_token_ms/adv:0.0006560004915383186 - perf/total_num_tokens:192449 - perf/time_per_step:31.79429466277361 - perf/throughput:756.6176653752331 (TaskRunner pid=2049544) Training Progress: 24%|██▍ | 24/100 [17:02<51:12, 40.43s/it] (TaskRunner pid=2049544) step:25 - global_seqlen/min:17209 - global_seqlen/max:32556 - global_seqlen/minmax_diff:15347 - global_seqlen/balanced_min:25066 - global_seqlen/balanced_max:25097 - global_seqlen/mean:25084.5 - actor/entropy:0.16078905761241913 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7048856616020203 - training/rollout_probs_diff_mean:0.003612553235143423 - training/rollout_probs_diff_std:0.01245187222957611 - training/rollout_actor_probs_pearson_corr:0.9980130791664124 - rollout_corr/rollout_is_mean:1.0001463890075684 - rollout_corr/rollout_is_ratio_fraction_high:3.095304418820888e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.73826104705222e-05 - rollout_corr/rollout_is_max:2.4638538360595703 - rollout_corr/rollout_is_min:0.07782533764839172 - rollout_corr/rollout_is_std:0.03481380268931389 - rollout_corr/rollout_is_eff_sample_size:0.9987899420046099 - rollout_corr/rollout_is_seq_mean:1.0001909732818604 - rollout_corr/rollout_is_seq_std:0.0020113997161388397 - rollout_corr/rollout_is_seq_max:1.0118286609649658 - rollout_corr/rollout_is_seq_min:0.9921811819076538 - rollout_corr/rollout_is_seq_max_deviation:0.01182866096496582 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1733281780034304) - rollout_corr/training_log_ppl:np.float64(0.15433276955445763) - rollout_corr/kl:np.float64(0.0005177396068951623) - rollout_corr/k3_kl:np.float64(0.0007405701547043009) - rollout_corr/rollout_ppl:np.float64(1.172693099360913) - rollout_corr/rollout_log_ppl:np.float64(0.15381502895615995) - rollout_corr/log_ppl_diff:np.float64(0.0005177405982976779) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015419888804899529) - rollout_corr/log_ppl_diff_max:np.float64(0.017087146639823914) - rollout_corr/log_ppl_diff_min:np.float64(-0.010565042495727539) - rollout_corr/ppl_ratio:np.float64(1.0005205699708313) - rollout_corr/chi2_token:np.float64(0.0020056585781276226) - rollout_corr/chi2_seq:np.float64(0.9553266488946974) - actor/pg_loss:np.float64(7.516134064644575e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005029199624004832) - actor/ppo_kl:np.float64(6.852521258160493e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3206869065761566) - perf/mfu/actor:np.float64(0.07269379031279649) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.39129257202148) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005469983443617821 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005469983443617821 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:504.796875 - response_length/max:1635.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:504.796875 - response_length_non_aborted/max:1635.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.09375 - prompt_length/max:365.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001018233597278595 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8177736550569534) - timing_s/agent_loop/generate_sequences/max:np.float64(11.67398145236075) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.060982199414866) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.67398145236075) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:335 - timing_s/agent_loop/slowest/response_length:1635 - timing_s/gen:17.524463137611747 - timing_s/reward:0.05791650153696537 - timing_s/old_log_prob:2.436915224418044 - timing_s/adv:0.13883326761424541 - timing_s/update_actor:9.980426739901304 - timing_s/step:30.245057743042707 - timing_s/stop_profile:0.00012283958494663239 - timing_per_token_ms/update_actor:0.049734032669085014 - timing_per_token_ms/gen:0.13560887065970026 - timing_per_token_ms/adv:0.0006918279595678876 - perf/total_num_tokens:200676 - perf/time_per_step:30.245057743042707 - perf/throughput:829.3751730651004 (TaskRunner pid=2049544) Training Progress: 25%|██▌ | 25/100 [17:32<46:43, 37.39s/it] (TaskRunner pid=2049544) (TaskRunner pid=2049544) step:26 - global_seqlen/min:19588 - global_seqlen/max:26583 - global_seqlen/minmax_diff:6995 - global_seqlen/balanced_min:22329 - global_seqlen/balanced_max:22375 - global_seqlen/mean:22358.5 - actor/entropy:0.14037299156188965 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3580157160758972 - training/rollout_probs_diff_mean:0.003176919650286436 - training/rollout_probs_diff_std:0.011454894207417965 - training/rollout_actor_probs_pearson_corr:0.9981624484062195 - rollout_corr/rollout_is_mean:1.0003305673599243 - rollout_corr/rollout_is_ratio_fraction_high:2.6683745090849698e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.6683745090849698e-05 - rollout_corr/rollout_is_max:3.044771909713745 - rollout_corr/rollout_is_min:0.3202765882015228 - rollout_corr/rollout_is_std:0.03229307755827904 - rollout_corr/rollout_is_eff_sample_size:0.9989589572900974 - rollout_corr/rollout_is_seq_mean:1.0003631114959717 - rollout_corr/rollout_is_seq_std:0.001997264800593257 - rollout_corr/rollout_is_seq_max:1.009385347366333 - rollout_corr/rollout_is_seq_min:0.9931842684745789 - rollout_corr/rollout_is_seq_max_deviation:0.009385347366333008 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.137321101501584) - rollout_corr/training_log_ppl:np.float64(0.12318833442259347) - rollout_corr/kl:np.float64(0.00036465673569097135) - rollout_corr/k3_kl:np.float64(0.0006004141492397252) - rollout_corr/rollout_ppl:np.float64(1.136900127865374) - rollout_corr/rollout_log_ppl:np.float64(0.12282367694933782) - rollout_corr/log_ppl_diff:np.float64(0.00036465747325564735) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014688569026475307) - rollout_corr/log_ppl_diff_max:np.float64(0.011015359312295914) - rollout_corr/log_ppl_diff_min:np.float64(-0.0049578845500946045) - rollout_corr/ppl_ratio:np.float64(1.0003668149001896) - rollout_corr/chi2_token:np.float64(0.001733903307467699) - rollout_corr/chi2_seq:np.float64(0.5194167881272733) - actor/pg_loss:np.float64(8.537911344319582e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003864719383273041) - actor/ppo_kl:np.float64(0.00016390427055512635) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1358119323849678) - perf/mfu/actor:np.float64(0.06507624815359347) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.42486190795898) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:1 - critic/score/mean:0.7578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01127944141626358 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01127944141626358 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:439.171875 - response_length/max:1781.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:439.171875 - response_length_non_aborted/max:1781.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.53125 - prompt_length/max:363.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013301707804203033 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8018740545958281) - timing_s/agent_loop/generate_sequences/max:np.float64(11.262868203222752) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.332270985694777) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.262868203222752) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:316 - timing_s/agent_loop/slowest/response_length:1781 - timing_s/gen:17.137579491361976 - timing_s/reward:0.06755838170647621 - timing_s/old_log_prob:2.513957444578409 - timing_s/adv:0.14904913492500782 - timing_s/update_actor:10.074241016060114 - timing_s/step:30.039264272898436 - timing_s/stop_profile:0.00011388026177883148 - timing_per_token_ms/update_actor:0.05632220976396065 - timing_per_token_ms/gen:0.15243159614475021 - timing_per_token_ms/adv:0.0008332912255127122 - perf/total_num_tokens:178868 - perf/time_per_step:30.039264272898436 - perf/throughput:744.309174714773 (TaskRunner pid=2049544) Training Progress: 26%|██▌ | 26/100 [18:02<43:24, 35.20s/it] (TaskRunner pid=2049544) step:27 - global_seqlen/min:20390 - global_seqlen/max:35494 - global_seqlen/minmax_diff:15104 - global_seqlen/balanced_min:27757 - global_seqlen/balanced_max:27828 - global_seqlen/mean:27799.5 - actor/entropy:0.13566820323467255 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4101554751396179 - training/rollout_probs_diff_mean:0.002812632592394948 - training/rollout_probs_diff_std:0.010530136525630951 - training/rollout_actor_probs_pearson_corr:0.9983336925506592 - rollout_corr/rollout_is_mean:1.0000394582748413 - rollout_corr/rollout_is_ratio_fraction_high:1.9780045477091335e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.615344005287625e-05 - rollout_corr/rollout_is_max:2.5563464164733887 - rollout_corr/rollout_is_min:0.3321341276168823 - rollout_corr/rollout_is_std:0.03021036833524704 - rollout_corr/rollout_is_eff_sample_size:0.9990882848714958 - rollout_corr/rollout_is_seq_mean:1.0000697374343872 - rollout_corr/rollout_is_seq_std:0.0016781107988208532 - rollout_corr/rollout_is_seq_max:1.0076134204864502 - rollout_corr/rollout_is_seq_min:0.9905708432197571 - rollout_corr/rollout_is_seq_max_deviation:0.00942915678024292 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1346074021421373) - rollout_corr/training_log_ppl:np.float64(0.12218492220563348) - rollout_corr/kl:np.float64(0.0005914641850237601) - rollout_corr/k3_kl:np.float64(0.0005361424209464616) - rollout_corr/rollout_ppl:np.float64(1.1339335390366614) - rollout_corr/rollout_log_ppl:np.float64(0.1215934576903237) - rollout_corr/log_ppl_diff:np.float64(0.0005914645153097808) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011890121677424759) - rollout_corr/log_ppl_diff_max:np.float64(0.012660838663578033) - rollout_corr/log_ppl_diff_min:np.float64(-0.006112679839134216) - rollout_corr/ppl_ratio:np.float64(1.0005930345505476) - rollout_corr/chi2_token:np.float64(0.0009534833952784538) - rollout_corr/chi2_seq:np.float64(1.2373452726751566) - actor/pg_loss:np.float64(0.00019052473362535238) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004077799751485145) - actor/ppo_kl:np.float64(0.00015187378462222512) - actor/pg_clipfrac_lower:np.float64(2.895663556046202e-06) - actor/grad_norm:np.float64(0.24880316480994225) - perf/mfu/actor:np.float64(0.08016295921139835) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.3461446762085) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:1 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0011563085718080401 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0011563085718080401 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:592.453125 - response_length/max:2410.0 - response_length/min:89.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:592.453125 - response_length_non_aborted/max:2410.0 - response_length_non_aborted/min:89.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.28125 - prompt_length/max:375.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010118260979652405 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9887365885078907) - timing_s/agent_loop/generate_sequences/max:np.float64(15.905478810891509) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.845138130702253) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.905478810891509) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:233 - timing_s/agent_loop/slowest/response_length:2410 - timing_s/gen:21.729817861691117 - timing_s/reward:0.06810113973915577 - timing_s/old_log_prob:2.5614845752716064 - timing_s/adv:0.1540636345744133 - timing_s/update_actor:10.151232639327645 - timing_s/step:34.75130227394402 - timing_s/stop_profile:0.00013370253145694733 - timing_per_token_ms/update_actor:0.04564485260223945 - timing_per_token_ms/gen:0.1432722648264045 - timing_per_token_ms/adv:0.0006927446292847592 - perf/total_num_tokens:222396 - perf/time_per_step:34.75130227394402 - perf/throughput:799.9556327661317 (TaskRunner pid=2049544) Training Progress: 27%|██▋ | 27/100 [18:37<42:40, 35.08s/it] (TaskRunner pid=2049544) step:28 - global_seqlen/min:17957 - global_seqlen/max:34244 - global_seqlen/minmax_diff:16287 - global_seqlen/balanced_min:25513 - global_seqlen/balanced_max:25550 - global_seqlen/mean:25526.75 - actor/entropy:0.16894282400608063 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.47068703174591064 - training/rollout_probs_diff_mean:0.003456175560131669 - training/rollout_probs_diff_std:0.011879513040184975 - training/rollout_actor_probs_pearson_corr:0.9982811212539673 - rollout_corr/rollout_is_mean:0.9999144077301025 - rollout_corr/rollout_is_ratio_fraction_high:1.4930944416846614e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.972377766738646e-05 - rollout_corr/rollout_is_max:2.6947081089019775 - rollout_corr/rollout_is_min:0.39785826206207275 - rollout_corr/rollout_is_std:0.03413606435060501 - rollout_corr/rollout_is_eff_sample_size:0.9988359665383245 - rollout_corr/rollout_is_seq_mean:0.9999093413352966 - rollout_corr/rollout_is_seq_std:0.002044897060841322 - rollout_corr/rollout_is_seq_max:1.0098328590393066 - rollout_corr/rollout_is_seq_min:0.9918521642684937 - rollout_corr/rollout_is_seq_max_deviation:0.00983285903930664 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1787242372520268) - rollout_corr/training_log_ppl:np.float64(0.15832054920610972) - rollout_corr/kl:np.float64(0.0006220183064016283) - rollout_corr/k3_kl:np.float64(0.0006294625401039866) - rollout_corr/rollout_ppl:np.float64(1.1779772732406855) - rollout_corr/rollout_log_ppl:np.float64(0.15769852932862705) - rollout_corr/log_ppl_diff:np.float64(0.0006220198774826713) - rollout_corr/log_ppl_abs_diff:np.float64(0.001586154270626139) - rollout_corr/log_ppl_diff_max:np.float64(0.009446874260902405) - rollout_corr/log_ppl_diff_min:np.float64(-0.006108276546001434) - rollout_corr/ppl_ratio:np.float64(1.0006244094111025) - rollout_corr/chi2_token:np.float64(0.0012534137349575758) - rollout_corr/chi2_seq:np.float64(0.5095949505921453) - actor/pg_loss:np.float64(0.0001528579741716385) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005431991780824319) - actor/ppo_kl:np.float64(-0.00011134325540240742) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2734988182783127) - perf/mfu/actor:np.float64(0.07474159440863055) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.44351959228516) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:1 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.018798992037773132 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.018798992037773132 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:523.2421875 - response_length/max:1995.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:523.2421875 - response_length_non_aborted/max:1995.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.46875 - prompt_length/max:363.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001253858208656311 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8268284611403942) - timing_s/agent_loop/generate_sequences/max:np.float64(13.178542386740446) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.144354823620233) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.178542386740446) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:302 - timing_s/agent_loop/slowest/response_length:1995 - timing_s/gen:19.040658405050635 - timing_s/reward:0.06457959674298763 - timing_s/old_log_prob:2.444152580574155 - timing_s/adv:0.15133203007280827 - timing_s/update_actor:10.063168801367283 - timing_s/step:31.856078814715147 - timing_s/stop_profile:0.00013601407408714294 - timing_per_token_ms/update_actor:0.04927756569758823 - timing_per_token_ms/gen:0.14214750582344632 - timing_per_token_ms/adv:0.0007410463047235168 - perf/total_num_tokens:204214 - perf/time_per_step:31.856078814715147 - perf/throughput:801.3148808574813 (TaskRunner pid=2049544) Training Progress: 28%|██▊ | 28/100 [19:09<40:57, 34.13s/it] (TaskRunner pid=2049544) step:29 - global_seqlen/min:20947 - global_seqlen/max:33732 - global_seqlen/minmax_diff:12785 - global_seqlen/balanced_min:25386 - global_seqlen/balanced_max:25685 - global_seqlen/mean:25470.375 - actor/entropy:0.13968856632709503 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5444822311401367 - training/rollout_probs_diff_mean:0.0031616955529898405 - training/rollout_probs_diff_std:0.011410399340093136 - training/rollout_actor_probs_pearson_corr:0.9981111884117126 - rollout_corr/rollout_is_mean:1.000052809715271 - rollout_corr/rollout_is_ratio_fraction_high:7.336810995184351e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.8694487961474806e-05 - rollout_corr/rollout_is_max:3.6677300930023193 - rollout_corr/rollout_is_min:0.22905343770980835 - rollout_corr/rollout_is_std:0.0324532613158226 - rollout_corr/rollout_is_eff_sample_size:0.9989480129782379 - rollout_corr/rollout_is_seq_mean:1.0001238584518433 - rollout_corr/rollout_is_seq_std:0.002332444768399 - rollout_corr/rollout_is_seq_max:1.0094679594039917 - rollout_corr/rollout_is_seq_min:0.9920924305915833 - rollout_corr/rollout_is_seq_max_deviation:0.0094679594039917 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.140123389661312) - rollout_corr/training_log_ppl:np.float64(0.12807749381863687) - rollout_corr/kl:np.float64(0.000638476259126719) - rollout_corr/k3_kl:np.float64(0.0006935356665564996) - rollout_corr/rollout_ppl:np.float64(1.1393897044472396) - rollout_corr/rollout_log_ppl:np.float64(0.1274390167054662) - rollout_corr/log_ppl_diff:np.float64(0.0006384771131706657) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017721431377140107) - rollout_corr/log_ppl_diff_max:np.float64(0.011614277958869934) - rollout_corr/log_ppl_diff_min:np.float64(-0.011640861630439758) - rollout_corr/ppl_ratio:np.float64(1.0006417338736355) - rollout_corr/chi2_token:np.float64(0.0015386808663606644) - rollout_corr/chi2_seq:np.float64(0.6557973187882453) - actor/pg_loss:np.float64(8.236418943852186e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006260939494495688) - actor/ppo_kl:np.float64(0.00012735309167744013) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2651471309363842) - perf/mfu/actor:np.float64(0.07338363198697957) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.38971328735352) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:1 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009649740532040596 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009649740532040596 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:532.41796875 - response_length/max:2748.0 - response_length/min:73.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:532.41796875 - response_length_non_aborted/max:2748.0 - response_length_non_aborted/min:73.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.53125 - prompt_length/max:375.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.792375981807709e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8251552730798721) - timing_s/agent_loop/generate_sequences/max:np.float64(16.657433584332466) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.999644991774403) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.657433584332466) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:265 - timing_s/agent_loop/slowest/response_length:2748 - timing_s/gen:22.55767191387713 - timing_s/reward:0.06376934424042702 - timing_s/old_log_prob:2.4361399989575148 - timing_s/adv:0.1466422975063324 - timing_s/update_actor:10.275281056761742 - timing_s/step:35.56420364230871 - timing_s/stop_profile:0.00011993944644927979 - timing_per_token_ms/update_actor:0.050427609805321585 - timing_per_token_ms/gen:0.16550137502019185 - timing_per_token_ms/adv:0.000719670879925857 - perf/total_num_tokens:203763 - perf/time_per_step:35.56420364230871 - perf/throughput:716.1801022222058 (TaskRunner pid=2049544) Training Progress: 29%|██▉ | 29/100 [19:45<40:54, 34.57s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We need to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20/actor (TaskRunner pid=2049544) step:30 - global_seqlen/min:16681 - global_seqlen/max:26264 - global_seqlen/minmax_diff:9583 - global_seqlen/balanced_min:22916 - global_seqlen/balanced_max:22938 - global_seqlen/mean:22931.25 - actor/entropy:0.1647506058216095 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3170890808105469 - training/rollout_probs_diff_mean:0.0035670390352606773 - training/rollout_probs_diff_std:0.01208495907485485 - training/rollout_actor_probs_pearson_corr:0.9981399178504944 - rollout_corr/rollout_is_mean:0.9999049305915833 - rollout_corr/rollout_is_ratio_fraction_high:8.544525371689815e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.00011962335702264681 - rollout_corr/rollout_is_max:2.2645792961120605 - rollout_corr/rollout_is_min:0.1764460951089859 - rollout_corr/rollout_is_std:0.03402237966656685 - rollout_corr/rollout_is_eff_sample_size:0.9988435782394575 - rollout_corr/rollout_is_seq_mean:0.9999919533729553 - rollout_corr/rollout_is_seq_std:0.0021591568365693092 - rollout_corr/rollout_is_seq_max:1.0069876909255981 - rollout_corr/rollout_is_seq_min:0.9930654764175415 - rollout_corr/rollout_is_seq_max_deviation:0.0069876909255981445 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1598813771270216) - rollout_corr/training_log_ppl:np.float64(0.14310977509740042) - rollout_corr/kl:np.float64(0.0006737301786081673) - rollout_corr/k3_kl:np.float64(0.0006828237883240718) - rollout_corr/rollout_ppl:np.float64(1.1591023579239845) - rollout_corr/rollout_log_ppl:np.float64(0.14243604317016434) - rollout_corr/log_ppl_diff:np.float64(0.0006737319272360764) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017324861764791422) - rollout_corr/log_ppl_diff_max:np.float64(0.0082569420337677) - rollout_corr/log_ppl_diff_min:np.float64(-0.005524754524230957) - rollout_corr/ppl_ratio:np.float64(1.000676429597661) - rollout_corr/chi2_token:np.float64(0.0013968497514724731) - rollout_corr/chi2_seq:np.float64(0.42728084372356534) - actor/pg_loss:np.float64(2.553162630647421e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004172229162122676) - actor/ppo_kl:np.float64(2.4439394231023925e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29012734070420265) - perf/mfu/actor:np.float64(0.06612954074679292) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.5312614440918) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.63125) - val-aux/sciknoweval/reward/std@16:np.float64(0.16827281213257073) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.69945) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1358224817804255) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5622375) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14543653725309813) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.630625) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1674322657572851) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7523875) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09759004740494832) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5039875) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11512292056447274) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.648175) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13191413368892382) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7910625) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06526407309863623) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.45775000000000005) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08015696464235675) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6597000000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09227713650829415) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.81775) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.039141593967153915) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.42647500000000005) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04889979668850037) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6632125) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06499777900917662) - val-aux/sciknoweval/score/mean@16:np.float64(0.63125) - val-aux/sciknoweval/score/std@16:np.float64(0.16827281213257073) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.69945) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1358224817804255) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5622375) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.14543653725309813) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.630625) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1674322657572851) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7523875) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09759004740494832) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5039875) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11512292056447274) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.648175) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13191413368892382) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7910625) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06526407309863623) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.45775000000000005) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08015696464235675) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6597000000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09227713650829415) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.81775) - val-aux/sciknoweval/score/best@16/std:np.float64(0.039141593967153915) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.42647500000000005) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04889979668850037) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6632125) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06499777900917662) - val-core/sciknoweval/acc/mean@16:np.float64(0.63125) - val-aux/sciknoweval/acc/std@16:np.float64(0.16827281213257073) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.69945) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1358224817804255) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5622375) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.14543653725309813) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.630625) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1674322657572851) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7523875) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09759004740494832) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5039875) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11512292056447274) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.648175) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13191413368892382) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7910625) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06526407309863623) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.45775000000000005) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08015696464235675) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6597000000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09227713650829415) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.81775) - val-core/sciknoweval/acc/best@16/std:np.float64(0.039141593967153915) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.42647500000000005) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04889979668850037) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6632125) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06499777900917662) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98359375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.04653599667599353) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.997425) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.01734892838049699) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9696374999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.05990914656773903) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9836) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.046352514622771394) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.999875) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.00190372690402996) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9472625000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0713750183504594) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9930749999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.029627882526401583) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9146000000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.07555851957352466) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9977499999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.014792280126255261) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8794875000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.06529402915117104) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9997250000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0034755281407842023) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:1 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012676871381700039 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012676871381700039 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:457.1640625 - response_length/max:1567.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:457.1640625 - response_length_non_aborted/max:1567.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.4375 - prompt_length/max:330.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010706298053264618 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7780425660312176) - timing_s/agent_loop/generate_sequences/max:np.float64(10.887974064797163) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.538448243663879) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.887974064797163) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:1567 - timing_s/gen:16.679681358858943 - timing_s/reward:0.06917422637343407 - timing_s/old_log_prob:2.485142918303609 - timing_s/adv:0.12575341202318668 - timing_s/update_actor:10.15482790954411 - timing_s/step:29.608009880408645 - timing_s/testing:103.1428439784795 - timing_s/save_checkpoint:6.939728271216154 - timing_s/stop_profile:0.0001257825642824173 - timing_per_token_ms/update_actor:0.05535474466908755 - timing_per_token_ms/gen:0.14251996307789996 - timing_per_token_ms/adv:0.0006854914800936859 - perf/total_num_tokens:183450 - perf/time_per_step:29.608009880408645 - perf/throughput:774.4948104456491 (TaskRunner pid=2049544) Training Progress: 30%|███ | 30/100 [22:05<1:17:09, 66.13s/it] (TaskRunner pid=2049544) step:31 - global_seqlen/min:19206 - global_seqlen/max:40437 - global_seqlen/minmax_diff:21231 - global_seqlen/balanced_min:26988 - global_seqlen/balanced_max:27006 - global_seqlen/mean:26994.75 - actor/entropy:0.15669088065624237 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2584149241447449 - training/rollout_probs_diff_mean:0.003061925061047077 - training/rollout_probs_diff_std:0.010589882731437683 - training/rollout_actor_probs_pearson_corr:0.9985439777374268 - rollout_corr/rollout_is_mean:1.0000404119491577 - rollout_corr/rollout_is_ratio_fraction_high:1.3810819837090094e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.7621639674180187e-05 - rollout_corr/rollout_is_max:2.6365303993225098 - rollout_corr/rollout_is_min:0.22984786331653595 - rollout_corr/rollout_is_std:0.030734580010175705 - rollout_corr/rollout_is_eff_sample_size:0.9990563960283122 - rollout_corr/rollout_is_seq_mean:1.0000174045562744 - rollout_corr/rollout_is_seq_std:0.0015726868296042085 - rollout_corr/rollout_is_seq_max:1.0063960552215576 - rollout_corr/rollout_is_seq_min:0.9952640533447266 - rollout_corr/rollout_is_seq_max_deviation:0.006396055221557617 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1623710673302412) - rollout_corr/training_log_ppl:np.float64(0.14397141717927298) - rollout_corr/kl:np.float64(0.0004526557635822215) - rollout_corr/k3_kl:np.float64(0.0005084346583146271) - rollout_corr/rollout_ppl:np.float64(1.1618167613632977) - rollout_corr/rollout_log_ppl:np.float64(0.1435187609968125) - rollout_corr/log_ppl_diff:np.float64(0.0004526561824604869) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012202383077237755) - rollout_corr/log_ppl_diff_max:np.float64(0.005235195159912109) - rollout_corr/log_ppl_diff_min:np.float64(-0.004862427711486816) - rollout_corr/ppl_ratio:np.float64(1.0004539461806417) - rollout_corr/chi2_token:np.float64(0.0011370060965418816) - rollout_corr/chi2_seq:np.float64(0.43574777944013476) - actor/pg_loss:np.float64(1.602328848093748e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027079670348939544) - actor/ppo_kl:np.float64(-1.6756775252702028e-05) - actor/pg_clipfrac_lower:np.float64(2.4157391180779086e-06) - actor/grad_norm:np.float64(0.20888379216194153) - perf/mfu/actor:np.float64(0.07792531348505069) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.11697006225586) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:1 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006074171047657728 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.006074171047657728 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:565.6796875 - response_length/max:1784.0 - response_length/min:147.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:565.6796875 - response_length_non_aborted/max:1784.0 - response_length_non_aborted/min:147.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.90625 - prompt_length/max:364.0 - prompt_length/min:188.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.553879499435425e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7227725125849247) - timing_s/agent_loop/generate_sequences/max:np.float64(13.10448818653822) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.673034100342193) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.10448818653822) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:284 - timing_s/agent_loop/slowest/response_length:1784 - timing_s/gen:18.842551900073886 - timing_s/reward:0.07317895255982876 - timing_s/old_log_prob:2.6255607213824987 - timing_s/adv:0.14376060664653778 - timing_s/update_actor:10.008040392771363 - timing_s/step:31.798675939440727 - timing_s/stop_profile:0.00011486746370792389 - timing_per_token_ms/update_actor:0.0463425313846737 - timing_per_token_ms/gen:0.13011554062503547 - timing_per_token_ms/adv:0.000665687803399447 - perf/total_num_tokens:215958 - perf/time_per_step:31.798675939440727 - perf/throughput:848.9268563071744 (TaskRunner pid=2049544) Training Progress: 31%|███ | 31/100 [22:36<1:04:13, 55.85s/it] (TaskRunner pid=2049544) step:32 - global_seqlen/min:15472 - global_seqlen/max:26624 - global_seqlen/minmax_diff:11152 - global_seqlen/balanced_min:21157 - global_seqlen/balanced_max:21170 - global_seqlen/mean:21164.125 - actor/entropy:0.16215530037879944 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35835716128349304 - training/rollout_probs_diff_mean:0.0033622696064412594 - training/rollout_probs_diff_std:0.011649704538285732 - training/rollout_actor_probs_pearson_corr:0.9983426928520203 - rollout_corr/rollout_is_mean:0.9998896718025208 - rollout_corr/rollout_is_ratio_fraction_high:1.9218758097849786e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.765627429354936e-05 - rollout_corr/rollout_is_max:2.1169991493225098 - rollout_corr/rollout_is_min:0.016281770542263985 - rollout_corr/rollout_is_std:0.033417847007513046 - rollout_corr/rollout_is_eff_sample_size:0.9988841362904098 - rollout_corr/rollout_is_seq_mean:0.9999381303787231 - rollout_corr/rollout_is_seq_std:0.00247986800968647 - rollout_corr/rollout_is_seq_max:1.011742353439331 - rollout_corr/rollout_is_seq_min:0.9914096593856812 - rollout_corr/rollout_is_seq_max_deviation:0.011742353439331055 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1614093114621937) - rollout_corr/training_log_ppl:np.float64(0.1426902024759329) - rollout_corr/kl:np.float64(0.0009178641532645582) - rollout_corr/k3_kl:np.float64(0.0007639187314083529) - rollout_corr/rollout_ppl:np.float64(1.1602927432395518) - rollout_corr/rollout_log_ppl:np.float64(0.1417723370686872) - rollout_corr/log_ppl_diff:np.float64(0.0009178654072456993) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020935917491442524) - rollout_corr/log_ppl_diff_max:np.float64(0.03533336520195007) - rollout_corr/log_ppl_diff_min:np.float64(-0.009082436561584473) - rollout_corr/ppl_ratio:np.float64(1.000924628926441) - rollout_corr/chi2_token:np.float64(0.00115761230699718) - rollout_corr/chi2_seq:np.float64(0.4135602100286633) - actor/pg_loss:np.float64(-7.768673822283745e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003810818443525932) - actor/ppo_kl:np.float64(0.00014974098553555137) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.26316603645682335) - perf/mfu/actor:np.float64(0.0603751772224912) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.58000946044922) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003006534418091178 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003006534418091178 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:406.50390625 - response_length/max:1261.0 - response_length/min:50.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:406.50390625 - response_length_non_aborted/max:1261.0 - response_length_non_aborted/min:50.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.875 - prompt_length/max:375.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013849511742591858 - timing_s/agent_loop/generate_sequences/min:np.float64(0.584519986063242) - timing_s/agent_loop/generate_sequences/max:np.float64(9.110312020406127) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.030619656012277) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.110312020406127) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:1261 - timing_s/gen:14.926933655515313 - timing_s/reward:0.05845559947192669 - timing_s/old_log_prob:2.34299498423934 - timing_s/adv:0.13051477633416653 - timing_s/update_actor:10.037976827472448 - timing_s/step:27.581528935581446 - timing_s/stop_profile:0.00012824125587940216 - timing_per_token_ms/update_actor:0.05928650976282063 - timing_per_token_ms/gen:0.14343855912665462 - timing_per_token_ms/adv:0.000770849115745197 - perf/total_num_tokens:169313 - perf/time_per_step:27.581528935581446 - perf/throughput:767.3296520084244 (TaskRunner pid=2049544) Training Progress: 32%|███▏ | 32/100 [23:04<53:42, 47.39s/it] (TaskRunner pid=2049544) step:33 - global_seqlen/min:21009 - global_seqlen/max:35793 - global_seqlen/minmax_diff:14784 - global_seqlen/balanced_min:26523 - global_seqlen/balanced_max:26619 - global_seqlen/mean:26546.375 - actor/entropy:0.1607956439256668 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4141917824745178 - training/rollout_probs_diff_mean:0.0031411326490342617 - training/rollout_probs_diff_std:0.01075077150017023 - training/rollout_actor_probs_pearson_corr:0.998579204082489 - rollout_corr/rollout_is_mean:1.0000613927841187 - rollout_corr/rollout_is_ratio_fraction_high:2.17348788282834e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.0714716053334996e-05 - rollout_corr/rollout_is_max:5.24385929107666 - rollout_corr/rollout_is_min:0.3405860662460327 - rollout_corr/rollout_is_std:0.0315140075981617 - rollout_corr/rollout_is_eff_sample_size:0.9990078527425291 - rollout_corr/rollout_is_seq_mean:1.0000804662704468 - rollout_corr/rollout_is_seq_std:0.0017564821755513549 - rollout_corr/rollout_is_seq_max:1.006698489189148 - rollout_corr/rollout_is_seq_min:0.995316743850708 - rollout_corr/rollout_is_seq_max_deviation:0.006698489189147949 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.176677621435374) - rollout_corr/training_log_ppl:np.float64(0.15545551202376373) - rollout_corr/kl:np.float64(0.000548688222613336) - rollout_corr/k3_kl:np.float64(0.0006223271676191189) - rollout_corr/rollout_ppl:np.float64(1.1760094286873937) - rollout_corr/rollout_log_ppl:np.float64(0.1549068232634454) - rollout_corr/log_ppl_diff:np.float64(0.0005486887603183277) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014138846672722138) - rollout_corr/log_ppl_diff_max:np.float64(0.008264932781457901) - rollout_corr/log_ppl_diff_min:np.float64(-0.005941852927207947) - rollout_corr/ppl_ratio:np.float64(1.0005506034940481) - rollout_corr/chi2_token:np.float64(0.0014625547919422388) - rollout_corr/chi2_seq:np.float64(0.45368791883811355) - actor/pg_loss:np.float64(5.0174305215477943e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003640777423470354) - actor/ppo_kl:np.float64(6.6643967455704e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2803301140666008) - perf/mfu/actor:np.float64(0.07602373904107226) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.56497955322266) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:1 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006237910129129887 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006237910129129887 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:539.16796875 - response_length/max:2340.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:539.16796875 - response_length_non_aborted/max:2340.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.40625 - prompt_length/max:437.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011204741895198822 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2724983282387257) - timing_s/agent_loop/generate_sequences/max:np.float64(14.548386441543698) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.296186965904781) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.548386441543698) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:2340 - timing_s/gen:20.489616226404905 - timing_s/reward:0.06211276724934578 - timing_s/old_log_prob:2.5394117888063192 - timing_s/adv:0.15707547590136528 - timing_s/update_actor:10.15939660742879 - timing_s/step:33.50294020399451 - timing_s/stop_profile:0.00015977025032043457 - timing_per_token_ms/update_actor:0.04783796567058962 - timing_per_token_ms/gen:0.14844643603356522 - timing_per_token_ms/adv:0.0007396277076501277 - perf/total_num_tokens:212371 - perf/time_per_step:33.50294020399451 - perf/throughput:792.3595612314322 (TaskRunner pid=2049544) Training Progress: 33%|███▎ | 33/100 [23:38<48:17, 43.24s/it] (TaskRunner pid=2049544) step:34 - global_seqlen/min:19210 - global_seqlen/max:32362 - global_seqlen/minmax_diff:13152 - global_seqlen/balanced_min:24672 - global_seqlen/balanced_max:24722 - global_seqlen/mean:24706.625 - actor/entropy:0.1634763479232788 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967529773712158 - training/rollout_probs_diff_mean:0.003025032114237547 - training/rollout_probs_diff_std:0.010739969089627266 - training/rollout_actor_probs_pearson_corr:0.998622715473175 - rollout_corr/rollout_is_mean:1.0000046491622925 - rollout_corr/rollout_is_ratio_fraction_high:2.2830181478639133e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.84905462549068e-05 - rollout_corr/rollout_is_max:3.96623158454895 - rollout_corr/rollout_is_min:0.17613501846790314 - rollout_corr/rollout_is_std:0.03082752786576748 - rollout_corr/rollout_is_eff_sample_size:0.9990505658256099 - rollout_corr/rollout_is_seq_mean:1.0001435279846191 - rollout_corr/rollout_is_seq_std:0.001950009143911302 - rollout_corr/rollout_is_seq_max:1.0084269046783447 - rollout_corr/rollout_is_seq_min:0.9940277338027954 - rollout_corr/rollout_is_seq_max_deviation:0.008426904678344727 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1493624411523342) - rollout_corr/training_log_ppl:np.float64(0.13525041530374438) - rollout_corr/kl:np.float64(0.0005815880843990096) - rollout_corr/k3_kl:np.float64(0.0006954628924091821) - rollout_corr/rollout_ppl:np.float64(1.1486861389130354) - rollout_corr/rollout_log_ppl:np.float64(0.13466882749344222) - rollout_corr/log_ppl_diff:np.float64(0.0005815878103021532) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016140752850333229) - rollout_corr/log_ppl_diff_max:np.float64(0.00731133297085762) - rollout_corr/log_ppl_diff_min:np.float64(-0.007330030202865601) - rollout_corr/ppl_ratio:np.float64(1.0005840898957103) - rollout_corr/chi2_token:np.float64(0.0018483358435332775) - rollout_corr/chi2_seq:np.float64(0.8693230981007218) - actor/pg_loss:np.float64(-7.774424739181995e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004309596388338832) - actor/ppo_kl:np.float64(0.00016126566817398214) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.24917255342006683) - perf/mfu/actor:np.float64(0.07059737130291072) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.48730087280273) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:1 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002149841981008649 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002149841981008649 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:513.30078125 - response_length/max:2220.0 - response_length/min:80.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:513.30078125 - response_length_non_aborted/max:2220.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.78125 - prompt_length/max:382.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001704581081867218 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9642345681786537) - timing_s/agent_loop/generate_sequences/max:np.float64(14.358597544953227) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.930665408472123) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.358597544953227) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:2220 - timing_s/gen:20.079416058957577 - timing_s/reward:0.055915795266628265 - timing_s/old_log_prob:2.5313415098935366 - timing_s/adv:0.14141708984971046 - timing_s/update_actor:10.167155608534813 - timing_s/step:33.08425388485193 - timing_s/stop_profile:0.0001313425600528717 - timing_per_token_ms/update_actor:0.05143941963205625 - timing_per_token_ms/gen:0.15280557101295672 - timing_per_token_ms/adv:0.000715481626131202 - perf/total_num_tokens:197653 - perf/time_per_step:33.08425388485193 - perf/throughput:746.7789688106661 (TaskRunner pid=2049544) Training Progress: 34%|███▍ | 34/100 [24:11<44:13, 40.21s/it] (TaskRunner pid=2049544) step:35 - global_seqlen/min:18610 - global_seqlen/max:31076 - global_seqlen/minmax_diff:12466 - global_seqlen/balanced_min:24486 - global_seqlen/balanced_max:24516 - global_seqlen/mean:24505.375 - actor/entropy:0.18308240175247192 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6351485848426819 - training/rollout_probs_diff_mean:0.003324724268168211 - training/rollout_probs_diff_std:0.011287217028439045 - training/rollout_actor_probs_pearson_corr:0.998609185218811 - rollout_corr/rollout_is_mean:0.9998504519462585 - rollout_corr/rollout_is_ratio_fraction_high:1.5409863408422098e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.852466034004465e-05 - rollout_corr/rollout_is_max:3.141425609588623 - rollout_corr/rollout_is_min:0.22313041985034943 - rollout_corr/rollout_is_std:0.03237971290946007 - rollout_corr/rollout_is_eff_sample_size:0.9989524144661357 - rollout_corr/rollout_is_seq_mean:0.999904215335846 - rollout_corr/rollout_is_seq_std:0.002002937952056527 - rollout_corr/rollout_is_seq_max:1.0084916353225708 - rollout_corr/rollout_is_seq_min:0.9936511516571045 - rollout_corr/rollout_is_seq_max_deviation:0.0084916353225708 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1920651025138795) - rollout_corr/training_log_ppl:np.float64(0.16792560920293909) - rollout_corr/kl:np.float64(0.0008154499521539904) - rollout_corr/k3_kl:np.float64(0.0007078099062596266) - rollout_corr/rollout_ppl:np.float64(1.1910648443736136) - rollout_corr/rollout_log_ppl:np.float64(0.1671101573156193) - rollout_corr/log_ppl_diff:np.float64(0.0008154518873197958) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017030747985700145) - rollout_corr/log_ppl_diff_max:np.float64(0.010384440422058105) - rollout_corr/log_ppl_diff_min:np.float64(-0.007758818566799164) - rollout_corr/ppl_ratio:np.float64(1.0008183948229998) - rollout_corr/chi2_token:np.float64(0.0012777894735336304) - rollout_corr/chi2_seq:np.float64(1.3958141808398068) - actor/pg_loss:np.float64(1.1038966476917267e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003179428899784398) - actor/ppo_kl:np.float64(0.00012983981464032013) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2326635681092739) - perf/mfu/actor:np.float64(0.07068896275098592) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.57692337036133) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004130806773900986 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004130806773900986 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:506.98046875 - response_length/max:1617.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:506.98046875 - response_length_non_aborted/max:1617.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.8125 - prompt_length/max:365.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.149631321430206e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9240005984902382) - timing_s/agent_loop/generate_sequences/max:np.float64(11.936645423993468) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.090721758017025) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.936645423993468) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:1617 - timing_s/gen:17.894150488078594 - timing_s/reward:0.06287588179111481 - timing_s/old_log_prob:2.4778820667415857 - timing_s/adv:0.1439337357878685 - timing_s/update_actor:10.013244837522507 - timing_s/step:30.680053589865565 - timing_s/stop_profile:0.00010856986045837402 - timing_per_token_ms/update_actor:0.05107677824519369 - timing_per_token_ms/gen:0.13787321140082284 - timing_per_token_ms/adv:0.0007341947215043052 - perf/total_num_tokens:196043 - perf/time_per_step:30.680053589865565 - perf/throughput:798.7396413184485 (TaskRunner pid=2049544) Training Progress: 35%|███▌ | 35/100 [24:41<40:28, 37.37s/it] (TaskRunner pid=2049544) step:36 - global_seqlen/min:24110 - global_seqlen/max:37995 - global_seqlen/minmax_diff:13885 - global_seqlen/balanced_min:31143 - global_seqlen/balanced_max:31201 - global_seqlen/mean:31181.25 - actor/entropy:0.2063475251197815 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35195159912109375 - training/rollout_probs_diff_mean:0.003259577788412571 - training/rollout_probs_diff_std:0.010389592498540878 - training/rollout_actor_probs_pearson_corr:0.9988572597503662 - rollout_corr/rollout_is_mean:0.9999135732650757 - rollout_corr/rollout_is_ratio_fraction_high:5.54268399355351e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.217073597421404e-05 - rollout_corr/rollout_is_max:5.167435169219971 - rollout_corr/rollout_is_min:0.08687802404165268 - rollout_corr/rollout_is_std:0.03102603740990162 - rollout_corr/rollout_is_eff_sample_size:0.9990380727632393 - rollout_corr/rollout_is_seq_mean:0.9999022483825684 - rollout_corr/rollout_is_seq_std:0.0015286069829016924 - rollout_corr/rollout_is_seq_max:1.0051732063293457 - rollout_corr/rollout_is_seq_min:0.9951693415641785 - rollout_corr/rollout_is_seq_max_deviation:0.005173206329345703 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2262746798805892) - rollout_corr/training_log_ppl:np.float64(0.19696971234225202) - rollout_corr/kl:np.float64(0.0006086873604012055) - rollout_corr/k3_kl:np.float64(0.0005889998031136656) - rollout_corr/rollout_ppl:np.float64(1.2254916024394333) - rollout_corr/rollout_log_ppl:np.float64(0.19636102448566817) - rollout_corr/log_ppl_diff:np.float64(0.000608687856583856) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012519802548922598) - rollout_corr/log_ppl_diff_max:np.float64(0.006760932505130768) - rollout_corr/log_ppl_diff_min:np.float64(-0.005945771932601929) - rollout_corr/ppl_ratio:np.float64(1.0006101776380092) - rollout_corr/chi2_token:np.float64(0.001163150416687131) - rollout_corr/chi2_seq:np.float64(0.9006679092999548) - actor/pg_loss:np.float64(6.59312354400754e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00041949365038362885) - actor/ppo_kl:np.float64(-4.479620235997572e-05) - actor/pg_clipfrac_lower:np.float64(3.426535158723709e-06) - actor/grad_norm:np.float64(0.21542762964963913) - perf/mfu/actor:np.float64(0.08866368508784078) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.68338394165039) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:1 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03192170336842537 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.03192170336842537 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:704.7578125 - response_length/max:2399.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:704.7578125 - response_length_non_aborted/max:2399.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.65625 - prompt_length/max:375.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.613484144210815e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1882787812501192) - timing_s/agent_loop/generate_sequences/max:np.float64(16.427959153428674) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.1280951405206) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.427959153428674) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:263 - timing_s/agent_loop/slowest/response_length:2399 - timing_s/gen:22.231742713600397 - timing_s/reward:0.06625362113118172 - timing_s/old_log_prob:2.5154502913355827 - timing_s/adv:0.1720115952193737 - timing_s/update_actor:10.132180897518992 - timing_s/step:35.22234913893044 - timing_s/stop_profile:0.00012091919779777527 - timing_per_token_ms/update_actor:0.04061808337349766 - timing_per_token_ms/gen:0.12322352932412729 - timing_per_token_ms/adv:0.00068956342040238 - perf/total_num_tokens:249450 - perf/time_per_step:35.22234913893044 - perf/throughput:885.2688921175928 (TaskRunner pid=2049544) Training Progress: 36%|███▌ | 36/100 [25:17<39:11, 36.74s/it] (TaskRunner pid=2049544) step:37 - global_seqlen/min:20543 - global_seqlen/max:33018 - global_seqlen/minmax_diff:12475 - global_seqlen/balanced_min:26755 - global_seqlen/balanced_max:26801 - global_seqlen/mean:26782.75 - actor/entropy:0.1873542070388794 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3535153567790985 - training/rollout_probs_diff_mean:0.003138389438390732 - training/rollout_probs_diff_std:0.010506313294172287 - training/rollout_actor_probs_pearson_corr:0.9987807273864746 - rollout_corr/rollout_is_mean:0.9999997019767761 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.388498225831427e-05 - rollout_corr/rollout_is_max:1.984167218208313 - rollout_corr/rollout_is_min:0.38167211413383484 - rollout_corr/rollout_is_std:0.031033720821142197 - rollout_corr/rollout_is_eff_sample_size:0.9990378348031789 - rollout_corr/rollout_is_seq_mean:0.9999757409095764 - rollout_corr/rollout_is_seq_std:0.0020235180854797363 - rollout_corr/rollout_is_seq_max:1.007972240447998 - rollout_corr/rollout_is_seq_min:0.991079568862915 - rollout_corr/rollout_is_seq_max_deviation:0.008920431137084961 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.17386184213683) - rollout_corr/training_log_ppl:np.float64(0.15399431145488052) - rollout_corr/kl:np.float64(0.000585384931017785) - rollout_corr/k3_kl:np.float64(0.0005648392770467581) - rollout_corr/rollout_ppl:np.float64(1.1731591601856053) - rollout_corr/rollout_log_ppl:np.float64(0.15340892578387866) - rollout_corr/log_ppl_diff:np.float64(0.000585385671001859) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015394459333037958) - rollout_corr/log_ppl_diff_max:np.float64(0.009715557098388672) - rollout_corr/log_ppl_diff_min:np.float64(-0.006565690040588379) - rollout_corr/ppl_ratio:np.float64(1.0005877201911062) - rollout_corr/chi2_token:np.float64(0.0010866590309888124) - rollout_corr/chi2_seq:np.float64(1.2919703791849315) - actor/pg_loss:np.float64(-7.207388989627361e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027252964378021716) - actor/ppo_kl:np.float64(2.565641079144143e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2241645734757185) - perf/mfu/actor:np.float64(0.07756524385091812) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.62493515014648) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:1 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012340063229203224 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012340063229203224 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:576.3984375 - response_length/max:2563.0 - response_length/min:72.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:576.3984375 - response_length_non_aborted/max:2563.0 - response_length_non_aborted/min:72.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:260.5625 - prompt_length/max:350.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.650814950466156e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7121520359069109) - timing_s/agent_loop/generate_sequences/max:np.float64(16.06510110758245) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.399860835161235) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.06510110758245) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:344 - timing_s/agent_loop/slowest/response_length:2563 - timing_s/gen:21.823639852926135 - timing_s/reward:0.07287336513400078 - timing_s/old_log_prob:2.61249627918005 - timing_s/adv:0.16705027595162392 - timing_s/update_actor:10.120526600629091 - timing_s/step:34.892802860587835 - timing_s/stop_profile:0.00011398270726203918 - timing_per_token_ms/update_actor:0.04723435140449119 - timing_per_token_ms/gen:0.14789872357260286 - timing_per_token_ms/adv:0.0007796542361763818 - perf/total_num_tokens:214262 - perf/time_per_step:34.892802860587835 - perf/throughput:767.5723302312205 (TaskRunner pid=2049544) Training Progress: 37%|███▋ | 37/100 [25:52<38:00, 36.20s/it] (TaskRunner pid=2049544) step:38 - global_seqlen/min:20896 - global_seqlen/max:36221 - global_seqlen/minmax_diff:15325 - global_seqlen/balanced_min:26629 - global_seqlen/balanced_max:26693 - global_seqlen/mean:26673.0 - actor/entropy:0.17777901887893677 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311151742935181 - training/rollout_probs_diff_mean:0.003014863934367895 - training/rollout_probs_diff_std:0.01037665270268917 - training/rollout_actor_probs_pearson_corr:0.998752772808075 - rollout_corr/rollout_is_mean:1.0000866651535034 - rollout_corr/rollout_is_ratio_fraction_high:2.75694746960653e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.446184564381838e-05 - rollout_corr/rollout_is_max:4.722990036010742 - rollout_corr/rollout_is_min:0.217980295419693 - rollout_corr/rollout_is_std:0.03060242161154747 - rollout_corr/rollout_is_eff_sample_size:0.9990646060209646 - rollout_corr/rollout_is_seq_mean:1.0000120401382446 - rollout_corr/rollout_is_seq_std:0.0017710411921143532 - rollout_corr/rollout_is_seq_max:1.005569338798523 - rollout_corr/rollout_is_seq_min:0.9887515902519226 - rollout_corr/rollout_is_seq_max_deviation:0.011248409748077393 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1705434750765562) - rollout_corr/training_log_ppl:np.float64(0.15039988774515223) - rollout_corr/kl:np.float64(0.0003758689842143781) - rollout_corr/k3_kl:np.float64(0.0005343400661388387) - rollout_corr/rollout_ppl:np.float64(1.170116110239178) - rollout_corr/rollout_log_ppl:np.float64(0.1500240175955696) - rollout_corr/log_ppl_diff:np.float64(0.00037587014958262444) - rollout_corr/log_ppl_abs_diff:np.float64(0.00122680542699527) - rollout_corr/log_ppl_diff_max:np.float64(0.012495182454586029) - rollout_corr/log_ppl_diff_min:np.float64(-0.005860418081283569) - rollout_corr/ppl_ratio:np.float64(1.000377441989258) - rollout_corr/chi2_token:np.float64(0.0015310256276279688) - rollout_corr/chi2_seq:np.float64(0.8941571062896401) - actor/pg_loss:np.float64(7.285538595169783e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002832129719081422) - actor/ppo_kl:np.float64(-9.715641379060003e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20185822248458862) - perf/mfu/actor:np.float64(0.07681988024557596) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.53835678100586) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:1 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0021598960738629103 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0021598960738629103 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:566.75 - response_length/max:2166.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:566.75 - response_length_non_aborted/max:2166.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.78125 - prompt_length/max:328.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.976459503173828e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1030525546520948) - timing_s/agent_loop/generate_sequences/max:np.float64(14.58826527185738) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.473840224818559) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.58826527185738) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:328 - timing_s/agent_loop/slowest/response_length:2166 - timing_s/gen:20.538045482710004 - timing_s/reward:0.06479933671653271 - timing_s/old_log_prob:2.464496372267604 - timing_s/adv:0.16802607849240303 - timing_s/update_actor:10.160526245832443 - timing_s/step:33.478492356836796 - timing_s/stop_profile:0.00017846189439296722 - timing_per_token_ms/update_actor:0.04761615793982887 - timing_per_token_ms/gen:0.1415557832674653 - timing_per_token_ms/adv:0.0007874352270667109 - perf/total_num_tokens:213384 - perf/time_per_step:33.478492356836796 - perf/throughput:796.720465058606 (TaskRunner pid=2049544) Training Progress: 38%|███▊ | 38/100 [26:25<36:34, 35.40s/it] (TaskRunner pid=2049544) step:39 - global_seqlen/min:15255 - global_seqlen/max:27450 - global_seqlen/minmax_diff:12195 - global_seqlen/balanced_min:22238 - global_seqlen/balanced_max:22271 - global_seqlen/mean:22258.375 - actor/entropy:0.1585860699415207 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.403981477022171 - training/rollout_probs_diff_mean:0.003201692830771208 - training/rollout_probs_diff_std:0.011194097809493542 - training/rollout_actor_probs_pearson_corr:0.9984230995178223 - rollout_corr/rollout_is_mean:1.0000526905059814 - rollout_corr/rollout_is_ratio_fraction_high:4.465760866878554e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.252065213629976e-05 - rollout_corr/rollout_is_max:3.0618841648101807 - rollout_corr/rollout_is_min:0.25154978036880493 - rollout_corr/rollout_is_std:0.032896533608436584 - rollout_corr/rollout_is_eff_sample_size:0.9989191068729061 - rollout_corr/rollout_is_seq_mean:1.0000567436218262 - rollout_corr/rollout_is_seq_std:0.0022173086181282997 - rollout_corr/rollout_is_seq_max:1.0083491802215576 - rollout_corr/rollout_is_seq_min:0.9902656674385071 - rollout_corr/rollout_is_seq_max_deviation:0.00973433256149292 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1577396178618073) - rollout_corr/training_log_ppl:np.float64(0.13937123074720148) - rollout_corr/kl:np.float64(0.00047606502702546294) - rollout_corr/k3_kl:np.float64(0.0006253823398196801) - rollout_corr/rollout_ppl:np.float64(1.157200226560235) - rollout_corr/rollout_log_ppl:np.float64(0.1388951658955193) - rollout_corr/log_ppl_diff:np.float64(0.0004760648516821675) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017094570139306597) - rollout_corr/log_ppl_diff_max:np.float64(0.008976280689239502) - rollout_corr/log_ppl_diff_min:np.float64(-0.007652416825294495) - rollout_corr/ppl_ratio:np.float64(1.0004788676742464) - rollout_corr/chi2_token:np.float64(0.001593738328665495) - rollout_corr/chi2_seq:np.float64(1.153579653473571) - actor/pg_loss:np.float64(-4.086887929588556e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021084232048451668) - actor/ppo_kl:np.float64(-5.276580297675082e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.22345420345664024) - perf/mfu/actor:np.float64(0.06472844108846296) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.6031494140625) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:1 - critic/score/mean:0.79296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.79296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0039901575073599815 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0039901575073599815 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:437.35546875 - response_length/max:1579.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:437.35546875 - response_length_non_aborted/max:1579.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.21875 - prompt_length/max:326.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015556812286376953 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7400109507143497) - timing_s/agent_loop/generate_sequences/max:np.float64(10.687160091474652) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.344982730093761) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.687160091474652) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:300 - timing_s/agent_loop/slowest/response_length:1564 - timing_s/gen:16.393727527931333 - timing_s/reward:0.060859134420752525 - timing_s/old_log_prob:2.4244355112314224 - timing_s/adv:0.15019422583281994 - timing_s/update_actor:9.942341027781367 - timing_s/step:29.05965017527342 - timing_s/stop_profile:0.00014510564506053925 - timing_per_token_ms/update_actor:0.05583483198897812 - timing_per_token_ms/gen:0.14642093841654236 - timing_per_token_ms/adv:0.0008434702995660057 - perf/total_num_tokens:178067 - perf/time_per_step:29.05965017527342 - perf/throughput:765.954678247966 (TaskRunner pid=2049544) Training Progress: 39%|███▉ | 39/100 [26:54<34:04, 33.52s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2049544) (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. We need to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30/actor (TaskRunner pid=2049544) step:40 - global_seqlen/min:20667 - global_seqlen/max:43425 - global_seqlen/minmax_diff:22758 - global_seqlen/balanced_min:28994 - global_seqlen/balanced_max:29096 - global_seqlen/mean:29057.375 - actor/entropy:0.16825498640537262 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.38080501556396484 - training/rollout_probs_diff_mean:0.002904911758378148 - training/rollout_probs_diff_std:0.010255875997245312 - training/rollout_actor_probs_pearson_corr:0.9987383484840393 - rollout_corr/rollout_is_mean:0.9999452829360962 - rollout_corr/rollout_is_ratio_fraction_high:1.811298898246605e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.62259779649321e-05 - rollout_corr/rollout_is_max:2.355332374572754 - rollout_corr/rollout_is_min:0.382403701543808 - rollout_corr/rollout_is_std:0.029906976968050003 - rollout_corr/rollout_is_eff_sample_size:0.9991061339931943 - rollout_corr/rollout_is_seq_mean:0.9999656081199646 - rollout_corr/rollout_is_seq_std:0.0016488394467160106 - rollout_corr/rollout_is_seq_max:1.009485125541687 - rollout_corr/rollout_is_seq_min:0.9944038391113281 - rollout_corr/rollout_is_seq_max_deviation:0.009485125541687012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.157492823433131) - rollout_corr/training_log_ppl:np.float64(0.14261198020540178) - rollout_corr/kl:np.float64(0.0005399936895650326) - rollout_corr/k3_kl:np.float64(0.0005378889725164981) - rollout_corr/rollout_ppl:np.float64(1.156866628676653) - rollout_corr/rollout_log_ppl:np.float64(0.1420719847228611) - rollout_corr/log_ppl_diff:np.float64(0.0005399954825406894) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012537419388536364) - rollout_corr/log_ppl_diff_max:np.float64(0.009929120540618896) - rollout_corr/log_ppl_diff_min:np.float64(-0.007549874484539032) - rollout_corr/ppl_ratio:np.float64(1.0005416304338723) - rollout_corr/chi2_token:np.float64(0.0010685438755899668) - rollout_corr/chi2_seq:np.float64(1.119046370498836) - actor/pg_loss:np.float64(-2.1900166757404804e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024728815151320305) - actor/ppo_kl:np.float64(-7.47417178104115e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1894925720989704) - perf/mfu/actor:np.float64(0.08261510768114952) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.61338424682617) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.62421875) - val-aux/sciknoweval/reward/std@16:np.float64(0.16084149793846786) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6912874999999999) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13357408072652655) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5557125) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14105131014378522) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.623125) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1618698771783546) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7462375) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.0922266430818953) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5021) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10738293756009418) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6433249999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13041149154323356) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.78285) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.05523922352229523) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.45817499999999994) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07435729339376719) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.653275) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.0950390140033621) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8010999999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.02694306919694267) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.42795000000000005) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04192728514733711) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.654725) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07283536359808326) - val-aux/sciknoweval/score/mean@16:np.float64(0.62421875) - val-aux/sciknoweval/score/std@16:np.float64(0.16084149793846786) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6912874999999999) - val-aux/sciknoweval/score/best@2/std:np.float64(0.13357408072652655) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5557125) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.14105131014378522) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.623125) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1618698771783546) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7462375) - val-aux/sciknoweval/score/best@4/std:np.float64(0.0922266430818953) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5021) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10738293756009418) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6433249999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13041149154323356) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.78285) - val-aux/sciknoweval/score/best@8/std:np.float64(0.05523922352229523) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.45817499999999994) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07435729339376719) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.653275) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.0950390140033621) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8010999999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.02694306919694267) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.42795000000000005) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04192728514733711) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.654725) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07283536359808326) - val-core/sciknoweval/acc/mean@16:np.float64(0.62421875) - val-aux/sciknoweval/acc/std@16:np.float64(0.16084149793846786) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6912874999999999) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.13357408072652655) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5557125) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.14105131014378522) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.623125) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1618698771783546) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7462375) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.0922266430818953) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5021) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10738293756009418) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6433249999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13041149154323356) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.78285) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.05523922352229523) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.45817499999999994) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07435729339376719) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.653275) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.0950390140033621) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8010999999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.02694306919694267) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.42795000000000005) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04192728514733711) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.654725) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07283536359808326) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9609375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.08980726114743592) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9929124999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.038290550480143085) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9296875) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.11086911079562345) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9614625) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.08902370132140851) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9996375000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0060893990521829235) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.8815249999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.12288515049099524) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9828125) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.060772813601514916) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8237249999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.11143746284761198) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9951625) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.03001908595873743) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.7747124999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0752713841460063) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99925) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.008297615319292063) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:1 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004764470737427473 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004764470737427473 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:646.98046875 - response_length/max:3478.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:646.98046875 - response_length_non_aborted/max:3478.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.0625 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.821169078350067e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9843795355409384) - timing_s/agent_loop/generate_sequences/max:np.float64(20.97976647876203) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.125959783450526) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.97976647876203) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:313 - timing_s/agent_loop/slowest/response_length:3478 - timing_s/gen:26.76995438337326 - timing_s/reward:0.06942468881607056 - timing_s/old_log_prob:2.522913720458746 - timing_s/adv:0.12163343653082848 - timing_s/update_actor:10.376626841723919 - timing_s/step:39.94687524065375 - timing_s/testing:129.93085425533354 - timing_s/save_checkpoint:6.843135168775916 - timing_s/stop_profile:9.070523083209991e-05 - timing_per_token_ms/update_actor:0.044638524822544705 - timing_per_token_ms/gen:0.1616279615242277 - timing_per_token_ms/adv:0.0005232468372092648 - perf/total_num_tokens:232459 - perf/time_per_step:39.94687524065375 - perf/throughput:727.4004493454958 (TaskRunner pid=2049544) Training Progress: 40%|████ | 40/100 [29:51<1:16:29, 76.50s/it] (TaskRunner pid=2049544) step:41 - global_seqlen/min:24357 - global_seqlen/max:35343 - global_seqlen/minmax_diff:10986 - global_seqlen/balanced_min:29480 - global_seqlen/balanced_max:29507 - global_seqlen/mean:29501.125 - actor/entropy:0.1756914258003235 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44652730226516724 - training/rollout_probs_diff_mean:0.00312806130386889 - training/rollout_probs_diff_std:0.010616622865200043 - training/rollout_actor_probs_pearson_corr:0.9987709522247314 - rollout_corr/rollout_is_mean:0.9999130368232727 - rollout_corr/rollout_is_ratio_fraction_high:1.81861159944674e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.424815647827927e-05 - rollout_corr/rollout_is_max:3.465665340423584 - rollout_corr/rollout_is_min:0.21781140565872192 - rollout_corr/rollout_is_std:0.031423088163137436 - rollout_corr/rollout_is_eff_sample_size:0.9990134444982739 - rollout_corr/rollout_is_seq_mean:0.9999098777770996 - rollout_corr/rollout_is_seq_std:0.0014667019713670015 - rollout_corr/rollout_is_seq_max:1.0048595666885376 - rollout_corr/rollout_is_seq_min:0.9947068691253662 - rollout_corr/rollout_is_seq_max_deviation:0.005293130874633789 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1796441837213933) - rollout_corr/training_log_ppl:np.float64(0.16063264349941164) - rollout_corr/kl:np.float64(0.0006431586379722631) - rollout_corr/k3_kl:np.float64(0.0005324390123746525) - rollout_corr/rollout_ppl:np.float64(1.178866429720074) - rollout_corr/rollout_log_ppl:np.float64(0.15998948468040908) - rollout_corr/log_ppl_diff:np.float64(0.000643158819002565) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012713416290353052) - rollout_corr/log_ppl_diff_max:np.float64(0.007460832595825195) - rollout_corr/log_ppl_diff_min:np.float64(-0.0031778663396835327) - rollout_corr/ppl_ratio:np.float64(1.0006445937324315) - rollout_corr/chi2_token:np.float64(0.0008295027073472738) - rollout_corr/chi2_seq:np.float64(0.9563508839346468) - actor/pg_loss:np.float64(9.131035767495632e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002985205437653349) - actor/ppo_kl:np.float64(3.691702790131757e-05) - actor/pg_clipfrac_lower:np.float64(5.99118084210204e-06) - actor/grad_norm:np.float64(0.2258460782468319) - perf/mfu/actor:np.float64(0.08484757615781298) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.7967414855957) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:1 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01819217950105667 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01819217950105667 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:644.37890625 - response_length/max:1915.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:644.37890625 - response_length_non_aborted/max:1915.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:361.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.878321826457977e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2621874269098043) - timing_s/agent_loop/generate_sequences/max:np.float64(14.428985564038157) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.6339426440390525) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.428985564038157) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1915 - timing_s/gen:20.114043543115258 - timing_s/reward:0.06382922269403934 - timing_s/old_log_prob:2.600179996341467 - timing_s/adv:0.1322335172444582 - timing_s/update_actor:9.928239937871695 - timing_s/step:32.94103864394128 - timing_s/stop_profile:0.00011893920600414276 - timing_per_token_ms/update_actor:0.04206720903809471 - timing_per_token_ms/gen:0.12193211451867568 - timing_per_token_ms/adv:0.0005602901467505824 - perf/total_num_tokens:236009 - perf/time_per_step:32.94103864394128 - perf/throughput:895.5736131722133 (TaskRunner pid=2049544) Training Progress: 41%|████ | 41/100 [30:24<1:02:23, 63.45s/it] (TaskRunner pid=2049544) step:42 - global_seqlen/min:25542 - global_seqlen/max:34694 - global_seqlen/minmax_diff:9152 - global_seqlen/balanced_min:30320 - global_seqlen/balanced_max:30380 - global_seqlen/mean:30366.25 - actor/entropy:0.19629985094070435 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31755661964416504 - training/rollout_probs_diff_mean:0.0030814865604043007 - training/rollout_probs_diff_std:0.010210192762315273 - training/rollout_actor_probs_pearson_corr:0.9988877177238464 - rollout_corr/rollout_is_mean:0.9999521374702454 - rollout_corr/rollout_is_ratio_fraction_high:1.7170919818454422e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.7170919818454422e-05 - rollout_corr/rollout_is_max:2.3355937004089355 - rollout_corr/rollout_is_min:0.20813822746276855 - rollout_corr/rollout_is_std:0.03025965206325054 - rollout_corr/rollout_is_eff_sample_size:0.9990850720973192 - rollout_corr/rollout_is_seq_mean:1.0000097751617432 - rollout_corr/rollout_is_seq_std:0.0012307165889069438 - rollout_corr/rollout_is_seq_max:1.006462574005127 - rollout_corr/rollout_is_seq_min:0.9960255026817322 - rollout_corr/rollout_is_seq_max_deviation:0.006462574005126953 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1838456057012081) - rollout_corr/training_log_ppl:np.float64(0.161605034765671) - rollout_corr/kl:np.float64(0.00040539363182290344) - rollout_corr/k3_kl:np.float64(0.0005141745206458381) - rollout_corr/rollout_ppl:np.float64(1.1833340604789555) - rollout_corr/rollout_log_ppl:np.float64(0.1611996397114126) - rollout_corr/log_ppl_diff:np.float64(0.00040539505425840616) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010806201898958534) - rollout_corr/log_ppl_diff_max:np.float64(0.005684956908226013) - rollout_corr/log_ppl_diff_min:np.float64(-0.0034256726503372192) - rollout_corr/ppl_ratio:np.float64(1.000406396575272) - rollout_corr/chi2_token:np.float64(0.0012630913406610489) - rollout_corr/chi2_seq:np.float64(1.0963045465759933) - actor/pg_loss:np.float64(-1.263362355530262e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005835516549268505) - actor/ppo_kl:np.float64(-5.1322627289351885e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21340569853782654) - perf/mfu/actor:np.float64(0.0874389401304758) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.24371719360352) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:1 - critic/score/mean:0.5703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0002196446730522439 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0002196446730522439 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:682.4765625 - response_length/max:2365.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:682.4765625 - response_length_non_aborted/max:2365.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:355.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014196522533893585 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5289256758987904) - timing_s/agent_loop/generate_sequences/max:np.float64(16.370171174407005) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.762006698663754) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.370171174407005) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:2365 - timing_s/gen:22.206629561260343 - timing_s/reward:0.06503966823220253 - timing_s/old_log_prob:2.5655108485370874 - timing_s/adv:0.14332416281104088 - timing_s/update_actor:10.220189092680812 - timing_s/step:35.30416659079492 - timing_s/stop_profile:0.00013269484043121338 - timing_per_token_ms/update_actor:0.04207051040497597 - timing_per_token_ms/gen:0.12710274827008908 - timing_per_token_ms/adv:0.000589981323060309 - perf/total_num_tokens:242930 - perf/time_per_step:35.30416659079492 - perf/throughput:860.1321864348324 (TaskRunner pid=2049544) Training Progress: 42%|████▏ | 42/100 [31:00<53:11, 55.03s/it] (TaskRunner pid=2049544) step:43 - global_seqlen/min:22214 - global_seqlen/max:34661 - global_seqlen/minmax_diff:12447 - global_seqlen/balanced_min:28283 - global_seqlen/balanced_max:28875 - global_seqlen/mean:28428.25 - actor/entropy:0.17894256114959717 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9968691468238831 - training/rollout_probs_diff_mean:0.0030561003368347883 - training/rollout_probs_diff_std:0.010964183136820793 - training/rollout_actor_probs_pearson_corr:0.9986260533332825 - rollout_corr/rollout_is_mean:0.9999217391014099 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.6834687711670995e-05 - rollout_corr/rollout_is_max:1.9516299962997437 - rollout_corr/rollout_is_min:3.1283660064218566e-05 - rollout_corr/rollout_is_std:0.02998857945203781 - rollout_corr/rollout_is_eff_sample_size:0.9991012551699266 - rollout_corr/rollout_is_seq_mean:0.9999701976776123 - rollout_corr/rollout_is_seq_std:0.0015368059976026416 - rollout_corr/rollout_is_seq_max:1.007220983505249 - rollout_corr/rollout_is_seq_min:0.9942285418510437 - rollout_corr/rollout_is_seq_max_deviation:0.0072209835052490234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1693742866627872) - rollout_corr/training_log_ppl:np.float64(0.15083149004203733) - rollout_corr/kl:np.float64(0.0007233850172774225) - rollout_corr/k3_kl:np.float64(0.0006608226789381888) - rollout_corr/rollout_ppl:np.float64(1.168502108193934) - rollout_corr/rollout_log_ppl:np.float64(0.15010810326202773) - rollout_corr/log_ppl_diff:np.float64(0.000723386780009605) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014408983988687396) - rollout_corr/log_ppl_diff_max:np.float64(0.025284215807914734) - rollout_corr/log_ppl_diff_min:np.float64(-0.005987722426652908) - rollout_corr/ppl_ratio:np.float64(1.0007265941239893) - rollout_corr/chi2_token:np.float64(0.0009258103091269732) - rollout_corr/chi2_seq:np.float64(1.824508806457743) - actor/pg_loss:np.float64(0.0001151182223111391) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000508402825516896) - actor/ppo_kl:np.float64(5.8405713696974004e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2429964765906334) - perf/mfu/actor:np.float64(0.08157217656976501) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.25969696044922) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:1 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.018738869577646255 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.018738869577646255 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:618.5703125 - response_length/max:2843.0 - response_length/min:133.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:618.5703125 - response_length_non_aborted/max:2843.0 - response_length_non_aborted/min:133.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.8125 - prompt_length/max:365.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.98380172252655e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5165256280452013) - timing_s/agent_loop/generate_sequences/max:np.float64(18.731039391830564) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.20822462200158) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.731039391830564) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:2843 - timing_s/gen:24.405709667131305 - timing_s/reward:0.0670168325304985 - timing_s/old_log_prob:2.534782351925969 - timing_s/adv:0.14802170917391777 - timing_s/update_actor:10.064561607316136 - timing_s/step:37.307200910523534 - timing_s/stop_profile:0.00010807998478412628 - timing_per_token_ms/update_actor:0.044254226022161655 - timing_per_token_ms/gen:0.15412120734008175 - timing_per_token_ms/adv:0.0006508565826858748 - perf/total_num_tokens:227426 - perf/time_per_step:37.307200910523534 - perf/throughput:762.0043666149454 (TaskRunner pid=2049544) Training Progress: 43%|████▎ | 43/100 [31:37<47:14, 49.73s/it] (TaskRunner pid=2049544) step:44 - global_seqlen/min:22462 - global_seqlen/max:40198 - global_seqlen/minmax_diff:17736 - global_seqlen/balanced_min:31295 - global_seqlen/balanced_max:31374 - global_seqlen/mean:31349.5 - actor/entropy:0.17300812900066376 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6893051266670227 - training/rollout_probs_diff_mean:0.002969349967315793 - training/rollout_probs_diff_std:0.010275507345795631 - training/rollout_actor_probs_pearson_corr:0.9987791776657104 - rollout_corr/rollout_is_mean:0.9999350309371948 - rollout_corr/rollout_is_ratio_fraction_high:1.112074914999539e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.448299659998156e-05 - rollout_corr/rollout_is_max:2.314483165740967 - rollout_corr/rollout_is_min:0.03194784000515938 - rollout_corr/rollout_is_std:0.030064012855291367 - rollout_corr/rollout_is_eff_sample_size:0.9990967333762735 - rollout_corr/rollout_is_seq_mean:0.9999725222587585 - rollout_corr/rollout_is_seq_std:0.0014090986223891377 - rollout_corr/rollout_is_seq_max:1.0068600177764893 - rollout_corr/rollout_is_seq_min:0.9924980998039246 - rollout_corr/rollout_is_seq_max_deviation:0.0075019001960754395 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1756953983567655) - rollout_corr/training_log_ppl:np.float64(0.15532817154598888) - rollout_corr/kl:np.float64(0.00048252152897987344) - rollout_corr/k3_kl:np.float64(0.0005270084845747647) - rollout_corr/rollout_ppl:np.float64(1.1751229297369719) - rollout_corr/rollout_log_ppl:np.float64(0.15484564797225175) - rollout_corr/log_ppl_diff:np.float64(0.00048252357373712584) - rollout_corr/log_ppl_abs_diff:np.float64(0.00115150140482001) - rollout_corr/log_ppl_diff_max:np.float64(0.01116035133600235) - rollout_corr/log_ppl_diff_min:np.float64(-0.005915746092796326) - rollout_corr/ppl_ratio:np.float64(1.0004838448949158) - rollout_corr/chi2_token:np.float64(0.0010843609925359488) - rollout_corr/chi2_seq:np.float64(0.9741545785218477) - actor/pg_loss:np.float64(-0.00012030208017677069) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002879910933870633) - actor/ppo_kl:np.float64(-3.409391269992845e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15622974559664726) - perf/mfu/actor:np.float64(0.08872996761202245) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.24708938598633) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:1 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017787639051675797 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017787639051675797 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:702.515625 - response_length/max:2613.0 - response_length/min:172.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:702.515625 - response_length_non_aborted/max:2613.0 - response_length_non_aborted/min:172.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.15625 - prompt_length/max:460.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001371428370475769 - timing_s/agent_loop/generate_sequences/min:np.float64(2.046190181747079) - timing_s/agent_loop/generate_sequences/max:np.float64(18.486025908961892) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.959546060519642) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.486025908961892) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:2613 - timing_s/gen:24.597592102363706 - timing_s/reward:0.07184095494449139 - timing_s/old_log_prob:2.5473712403327227 - timing_s/adv:0.13307522423565388 - timing_s/update_actor:10.182054167613387 - timing_s/step:37.62097364850342 - timing_s/stop_profile:0.0001139417290687561 - timing_per_token_ms/update_actor:0.040598949614879774 - timing_per_token_ms/gen:0.13677182503927685 - timing_per_token_ms/adv:0.0005306114301490211 - perf/total_num_tokens:250796 - perf/time_per_step:37.62097364850342 - perf/throughput:833.2984758156863 (TaskRunner pid=2049544) Training Progress: 44%|████▍ | 44/100 [32:15<43:02, 46.11s/it] (TaskRunner pid=2049544) step:45 - global_seqlen/min:20785 - global_seqlen/max:36095 - global_seqlen/minmax_diff:15310 - global_seqlen/balanced_min:29569 - global_seqlen/balanced_max:29783 - global_seqlen/mean:29630.5 - actor/entropy:0.2007351815700531 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102352559566498 - training/rollout_probs_diff_mean:0.0033135332632809877 - training/rollout_probs_diff_std:0.01088031567633152 - training/rollout_actor_probs_pearson_corr:0.998772919178009 - rollout_corr/rollout_is_mean:1.0001227855682373 - rollout_corr/rollout_is_ratio_fraction_high:6.0338379626045935e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.033837780705653e-05 - rollout_corr/rollout_is_max:5.347679138183594 - rollout_corr/rollout_is_min:0.0047262562438845634 - rollout_corr/rollout_is_std:0.03222286328673363 - rollout_corr/rollout_is_eff_sample_size:0.998963001987778 - rollout_corr/rollout_is_seq_mean:1.00006902217865 - rollout_corr/rollout_is_seq_std:0.001625648234039545 - rollout_corr/rollout_is_seq_max:1.0073453187942505 - rollout_corr/rollout_is_seq_min:0.9958909749984741 - rollout_corr/rollout_is_seq_max_deviation:0.007345318794250488 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.208143400028348) - rollout_corr/training_log_ppl:np.float64(0.17995586975303013) - rollout_corr/kl:np.float64(0.0005010489769254889) - rollout_corr/k3_kl:np.float64(0.0006270660993550337) - rollout_corr/rollout_ppl:np.float64(1.20752296410501) - rollout_corr/rollout_log_ppl:np.float64(0.1794548208345077) - rollout_corr/log_ppl_diff:np.float64(0.0005010489185224287) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013183121991460212) - rollout_corr/log_ppl_diff_max:np.float64(0.006140023469924927) - rollout_corr/log_ppl_diff_min:np.float64(-0.006050072610378265) - rollout_corr/ppl_ratio:np.float64(1.000502604758367) - rollout_corr/chi2_token:np.float64(0.0014851994346827269) - rollout_corr/chi2_seq:np.float64(1.3357745304238051) - actor/pg_loss:np.float64(-6.521574687212706e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00033872173025883967) - actor/ppo_kl:np.float64(-1.556063463681312e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20266040414571762) - perf/mfu/actor:np.float64(0.08389738845778953) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.0823974609375) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:2 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005806060507893562 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005806060507893562 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:647.390625 - response_length/max:2551.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:647.390625 - response_length_non_aborted/max:2551.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.5625 - prompt_length/max:437.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00031680427491664886 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9299852326512337) - timing_s/agent_loop/generate_sequences/max:np.float64(16.61781131848693) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.397257668249949) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.61781131848693) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:324 - timing_s/agent_loop/slowest/response_length:2551 - timing_s/gen:22.67037677206099 - timing_s/reward:0.06372027657926083 - timing_s/old_log_prob:2.7545996010303497 - timing_s/adv:0.11587526835501194 - timing_s/update_actor:10.353036934509873 - timing_s/step:36.06465018354356 - timing_s/stop_profile:0.00011578015983104706 - timing_per_token_ms/update_actor:0.04367559159696037 - timing_per_token_ms/gen:0.13678937544988892 - timing_per_token_ms/adv:0.0004888344288613588 - perf/total_num_tokens:237044 - perf/time_per_step:36.06465018354356 - perf/throughput:821.5939943740397 (TaskRunner pid=2049544) Training Progress: 45%|████▌ | 45/100 [32:51<39:37, 43.23s/it] (TaskRunner pid=2049544) step:46 - global_seqlen/min:16300 - global_seqlen/max:40451 - global_seqlen/minmax_diff:24151 - global_seqlen/balanced_min:26008 - global_seqlen/balanced_max:26038 - global_seqlen/mean:26025.75 - actor/entropy:0.16465333104133606 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.42921197414398193 - training/rollout_probs_diff_mean:0.0029608916956931353 - training/rollout_probs_diff_std:0.01070943009108305 - training/rollout_actor_probs_pearson_corr:0.9986132979393005 - rollout_corr/rollout_is_mean:0.9999878406524658 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.257039058255032e-05 - rollout_corr/rollout_is_max:1.9571115970611572 - rollout_corr/rollout_is_min:0.060181617736816406 - rollout_corr/rollout_is_std:0.030463820323348045 - rollout_corr/rollout_is_eff_sample_size:0.9990728161485534 - rollout_corr/rollout_is_seq_mean:1.000013828277588 - rollout_corr/rollout_is_seq_std:0.0019006438087671995 - rollout_corr/rollout_is_seq_max:1.008240818977356 - rollout_corr/rollout_is_seq_min:0.993937611579895 - rollout_corr/rollout_is_seq_max_deviation:0.008240818977355957 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.153148335404694) - rollout_corr/training_log_ppl:np.float64(0.136645665843389) - rollout_corr/kl:np.float64(0.0006094530078737392) - rollout_corr/k3_kl:np.float64(0.0006117465548527434) - rollout_corr/rollout_ppl:np.float64(1.152437952812761) - rollout_corr/rollout_log_ppl:np.float64(0.13603621201764327) - rollout_corr/log_ppl_diff:np.float64(0.0006094538257457316) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014239892771001905) - rollout_corr/log_ppl_diff_max:np.float64(0.014275498688220978) - rollout_corr/log_ppl_diff_min:np.float64(-0.006409347057342529) - rollout_corr/ppl_ratio:np.float64(1.0006118323653936) - rollout_corr/chi2_token:np.float64(0.0012153126299381256) - rollout_corr/chi2_seq:np.float64(0.35479475394822657) - actor/pg_loss:np.float64(-5.846214480698109e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001732110890770855) - actor/ppo_kl:np.float64(5.357556297269639e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1466388632543385) - perf/mfu/actor:np.float64(0.07518990372656345) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.19413375854492) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:2 - critic/score/mean:0.7578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.023664643988013268 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.023664643988013268 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:561.8671875 - response_length/max:2403.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:561.8671875 - response_length_non_aborted/max:2403.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.4375 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002136826515197754 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7386901136487722) - timing_s/agent_loop/generate_sequences/max:np.float64(15.293935690075159) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.345510819919582) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.293935690075159) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:2403 - timing_s/gen:21.075742207467556 - timing_s/reward:0.06618223898112774 - timing_s/old_log_prob:2.503137046471238 - timing_s/adv:0.11277000606060028 - timing_s/update_actor:10.213601699098945 - timing_s/step:34.05827110260725 - timing_s/stop_profile:0.00012739188969135284 - timing_per_token_ms/update_actor:0.04905527073714948 - timing_per_token_ms/gen:0.14652416056582793 - timing_per_token_ms/adv:0.000541627071557017 - perf/total_num_tokens:208206 - perf/time_per_step:34.05827110260725 - perf/throughput:764.1535861169318 (TaskRunner pid=2049544) Training Progress: 46%|████▌ | 46/100 [33:25<36:26, 40.50s/it] (TaskRunner pid=2049544) step:47 - global_seqlen/min:23716 - global_seqlen/max:42058 - global_seqlen/minmax_diff:18342 - global_seqlen/balanced_min:31374 - global_seqlen/balanced_max:31661 - global_seqlen/mean:31456.375 - actor/entropy:0.14881354570388794 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4682484567165375 - training/rollout_probs_diff_mean:0.002767270663753152 - training/rollout_probs_diff_std:0.010147105902433395 - training/rollout_actor_probs_pearson_corr:0.9986124634742737 - rollout_corr/rollout_is_mean:1.0000232458114624 - rollout_corr/rollout_is_ratio_fraction_high:5.438477273855824e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.350781819084659e-05 - rollout_corr/rollout_is_max:2.263822555541992 - rollout_corr/rollout_is_min:0.43962690234184265 - rollout_corr/rollout_is_std:0.02890361100435257 - rollout_corr/rollout_is_eff_sample_size:0.999165278640662 - rollout_corr/rollout_is_seq_mean:1.0001194477081299 - rollout_corr/rollout_is_seq_std:0.0015050210058689117 - rollout_corr/rollout_is_seq_max:1.005627155303955 - rollout_corr/rollout_is_seq_min:0.993998646736145 - rollout_corr/rollout_is_seq_max_deviation:0.0060013532638549805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1467792866751552) - rollout_corr/training_log_ppl:np.float64(0.13201592379482463) - rollout_corr/kl:np.float64(0.0004759397659170883) - rollout_corr/k3_kl:np.float64(0.0005555009257420807) - rollout_corr/rollout_ppl:np.float64(1.146229465957731) - rollout_corr/rollout_log_ppl:np.float64(0.13153998326015426) - rollout_corr/log_ppl_diff:np.float64(0.00047594053467037156) - rollout_corr/log_ppl_abs_diff:np.float64(0.001261575052922126) - rollout_corr/log_ppl_diff_max:np.float64(0.008257180452346802) - rollout_corr/log_ppl_diff_min:np.float64(-0.0055009350180625916) - rollout_corr/ppl_ratio:np.float64(1.000477674184367) - rollout_corr/chi2_token:np.float64(0.0013230419717729092) - rollout_corr/chi2_seq:np.float64(0.636790500022471) - actor/pg_loss:np.float64(-0.00010656809899955988) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00031795622612662555) - actor/ppo_kl:np.float64(0.00011273976376457995) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1950441598892212) - perf/mfu/actor:np.float64(0.0887384677499009) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.28681945800781) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:2 - critic/score/mean:0.78125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011680489405989647 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011680489405989647 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:718.26171875 - response_length/max:4213.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:718.26171875 - response_length_non_aborted/max:4213.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.75 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017112679779529572 - timing_s/agent_loop/generate_sequences/min:np.float64(0.828915087506175) - timing_s/agent_loop/generate_sequences/max:np.float64(25.74352060444653) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.723139668560179) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(25.74352060444653) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:4213 - timing_s/gen:31.583294926211238 - timing_s/reward:0.0717991292476654 - timing_s/old_log_prob:2.5992581117898226 - timing_s/adv:0.12912281975150108 - timing_s/update_actor:10.13827026821673 - timing_s/step:44.60779097676277 - timing_s/stop_profile:0.00012019835412502289 - timing_per_token_ms/update_actor:0.040287025556094466 - timing_per_token_ms/gen:0.1717650301901359 - timing_per_token_ms/adv:0.0005131027484552061 - perf/total_num_tokens:251651 - perf/time_per_step:44.60779097676277 - perf/throughput:705.1767036925535 (TaskRunner pid=2049544) Training Progress: 47%|████▋ | 47/100 [34:10<36:52, 41.75s/it] (TaskRunner pid=2049544) step:48 - global_seqlen/min:18590 - global_seqlen/max:34823 - global_seqlen/minmax_diff:16233 - global_seqlen/balanced_min:28237 - global_seqlen/balanced_max:28801 - global_seqlen/mean:28407.875 - actor/entropy:0.14581364393234253 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.47068536281585693 - training/rollout_probs_diff_mean:0.00291546992957592 - training/rollout_probs_diff_std:0.01052320096641779 - training/rollout_actor_probs_pearson_corr:0.9985108971595764 - rollout_corr/rollout_is_mean:0.9999670386314392 - rollout_corr/rollout_is_ratio_fraction_high:1.2597076420206577e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.779122926061973e-05 - rollout_corr/rollout_is_max:2.1663451194763184 - rollout_corr/rollout_is_min:0.35977837443351746 - rollout_corr/rollout_is_std:0.030446205288171768 - rollout_corr/rollout_is_eff_sample_size:0.9990738870447088 - rollout_corr/rollout_is_seq_mean:0.9999531507492065 - rollout_corr/rollout_is_seq_std:0.0016499547054991126 - rollout_corr/rollout_is_seq_max:1.0073652267456055 - rollout_corr/rollout_is_seq_min:0.9916918873786926 - rollout_corr/rollout_is_seq_max_deviation:0.008308112621307373 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1378675876185298) - rollout_corr/training_log_ppl:np.float64(0.12675013709667837) - rollout_corr/kl:np.float64(0.000753273961016987) - rollout_corr/k3_kl:np.float64(0.0006820533353391056) - rollout_corr/rollout_ppl:np.float64(1.1369990683160722) - rollout_corr/rollout_log_ppl:np.float64(0.12599686199246207) - rollout_corr/log_ppl_diff:np.float64(0.0007532751042163) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014732455310877413) - rollout_corr/log_ppl_diff_max:np.float64(0.009199053049087524) - rollout_corr/log_ppl_diff_min:np.float64(-0.006944321095943451) - rollout_corr/ppl_ratio:np.float64(1.0007553710602224) - rollout_corr/chi2_token:np.float64(0.0013299805577844381) - rollout_corr/chi2_seq:np.float64(0.6524860400240868) - actor/pg_loss:np.float64(-2.4408800527453423e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007820084601917188) - actor/ppo_kl:np.float64(0.00018927762639009416) - actor/pg_clipfrac_lower:np.float64(1.1097301467088982e-05) - actor/grad_norm:np.float64(0.2395218200981617) - perf/mfu/actor:np.float64(0.08095804530180152) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.21407699584961) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:2 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01809648796916008 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01809648796916008 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:620.18359375 - response_length/max:3666.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:620.18359375 - response_length_non_aborted/max:3666.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.5625 - prompt_length/max:365.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014730170369148254 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9394192546606064) - timing_s/agent_loop/generate_sequences/max:np.float64(21.90096348337829) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.9833613064474775) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.90096348337829) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:309 - timing_s/agent_loop/slowest/response_length:3666 - timing_s/gen:28.045954892411828 - timing_s/reward:0.0676717646420002 - timing_s/old_log_prob:2.6066879238933325 - timing_s/adv:0.12023183144629002 - timing_s/update_actor:10.316766967996955 - timing_s/step:41.24971695430577 - timing_s/stop_profile:0.00013453327119350433 - timing_per_token_ms/update_actor:0.04539571759589971 - timing_per_token_ms/gen:0.17664851570170015 - timing_per_token_ms/adv:0.0005290427013912956 - perf/total_num_tokens:227263 - perf/time_per_step:41.24971695430577 - perf/throughput:688.680483104132 (TaskRunner pid=2049544) Training Progress: 48%|████▊ | 48/100 [34:51<36:03, 41.61s/it] (TaskRunner pid=2049544) step:49 - global_seqlen/min:18126 - global_seqlen/max:38899 - global_seqlen/minmax_diff:20773 - global_seqlen/balanced_min:26338 - global_seqlen/balanced_max:26710 - global_seqlen/mean:26451.75 - actor/entropy:0.18256059288978577 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3656392991542816 - training/rollout_probs_diff_mean:0.0032566494774073362 - training/rollout_probs_diff_std:0.010963589884340763 - training/rollout_actor_probs_pearson_corr:0.9987143278121948 - rollout_corr/rollout_is_mean:1.0001035928726196 - rollout_corr/rollout_is_ratio_fraction_high:6.82920153849409e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.146281521068886e-05 - rollout_corr/rollout_is_max:2.4023168087005615 - rollout_corr/rollout_is_min:0.14749553799629211 - rollout_corr/rollout_is_std:0.03197216987609863 - rollout_corr/rollout_is_eff_sample_size:0.9989789431740315 - rollout_corr/rollout_is_seq_mean:1.0002081394195557 - rollout_corr/rollout_is_seq_std:0.001892627333290875 - rollout_corr/rollout_is_seq_max:1.008901834487915 - rollout_corr/rollout_is_seq_min:0.9917373061180115 - rollout_corr/rollout_is_seq_max_deviation:0.008901834487915039 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1714685205370188) - rollout_corr/training_log_ppl:np.float64(0.15115834452444687) - rollout_corr/kl:np.float64(0.0004743365148804912) - rollout_corr/k3_kl:np.float64(0.0006857500739982925) - rollout_corr/rollout_ppl:np.float64(1.1708842911757529) - rollout_corr/rollout_log_ppl:np.float64(0.15068400763266254) - rollout_corr/log_ppl_diff:np.float64(0.0004743368917843327) - rollout_corr/log_ppl_abs_diff:np.float64(0.001488212525146082) - rollout_corr/log_ppl_diff_max:np.float64(0.010617084801197052) - rollout_corr/log_ppl_diff_min:np.float64(-0.015819266438484192) - rollout_corr/ppl_ratio:np.float64(1.0004769298247993) - rollout_corr/chi2_token:np.float64(0.0018537091091275215) - rollout_corr/chi2_seq:np.float64(0.6309683343861252) - actor/pg_loss:np.float64(3.3562071621418e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005490989437930693) - actor/ppo_kl:np.float64(8.648514501530258e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2604895681142807) - perf/mfu/actor:np.float64(0.07564944214215971) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.27551984786987) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:2 - critic/score/mean:0.77734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.77734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015421191230416298 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015421191230416298 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:571.9921875 - response_length/max:3613.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:571.9921875 - response_length_non_aborted/max:3613.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.625 - prompt_length/max:362.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014311634004116058 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9619488045573235) - timing_s/agent_loop/generate_sequences/max:np.float64(21.043767675757408) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.410715061574592) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.043767675757408) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:3586 - timing_s/gen:27.190765019506216 - timing_s/reward:0.06571215577423573 - timing_s/old_log_prob:2.4588573779910803 - timing_s/adv:0.11915759928524494 - timing_s/update_actor:10.26756145618856 - timing_s/step:40.190888445824385 - timing_s/stop_profile:0.00011413171887397766 - timing_per_token_ms/update_actor:0.048520237111857246 - timing_per_token_ms/gen:0.185691217779869 - timing_per_token_ms/adv:0.0005630893952443834 - perf/total_num_tokens:211614 - perf/time_per_step:40.190888445824385 - perf/throughput:658.1529053694804 (TaskRunner pid=2049544) Training Progress: 49%|████▉ | 49/100 [35:31<35:01, 41.20s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) The electric field $ E $ due to a point charge $ q $ at a distance $ r $ is given by: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) - $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2049544) - $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2049544) - $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We need to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs ($ 1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest option is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40/actor (TaskRunner pid=2049544) step:50 - global_seqlen/min:22621 - global_seqlen/max:38182 - global_seqlen/minmax_diff:15561 - global_seqlen/balanced_min:31307 - global_seqlen/balanced_max:32242 - global_seqlen/mean:31432.0 - actor/entropy:0.20185792446136475 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3018432855606079 - training/rollout_probs_diff_mean:0.003106999211013317 - training/rollout_probs_diff_std:0.010115313343703747 - training/rollout_actor_probs_pearson_corr:0.9989494681358337 - rollout_corr/rollout_is_mean:1.0000050067901611 - rollout_corr/rollout_is_ratio_fraction_high:5.458515261125285e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.458515261125285e-06 - rollout_corr/rollout_is_max:2.0969016551971436 - rollout_corr/rollout_is_min:0.33815088868141174 - rollout_corr/rollout_is_std:0.030253741890192032 - rollout_corr/rollout_is_eff_sample_size:0.9990853100798829 - rollout_corr/rollout_is_seq_mean:0.9999303817749023 - rollout_corr/rollout_is_seq_std:0.001567038125358522 - rollout_corr/rollout_is_seq_max:1.0050349235534668 - rollout_corr/rollout_is_seq_min:0.993672788143158 - rollout_corr/rollout_is_seq_max_deviation:0.006327211856842041 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.191426008939743) - rollout_corr/training_log_ppl:np.float64(0.16859979891887633) - rollout_corr/kl:np.float64(0.0005692955341736194) - rollout_corr/k3_kl:np.float64(0.0005476311655883137) - rollout_corr/rollout_ppl:np.float64(1.1907334956340492) - rollout_corr/rollout_log_ppl:np.float64(0.16803050318412716) - rollout_corr/log_ppl_diff:np.float64(0.0005692957347491756) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012659797939704731) - rollout_corr/log_ppl_diff_max:np.float64(0.008102566003799438) - rollout_corr/log_ppl_diff_min:np.float64(-0.00437454879283905) - rollout_corr/ppl_ratio:np.float64(1.0005708439275622) - rollout_corr/chi2_token:np.float64(0.0010575931519269943) - rollout_corr/chi2_seq:np.float64(1.1911898551043123) - actor/pg_loss:np.float64(6.677815690636635e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036017004458699375) - actor/ppo_kl:np.float64(2.6112116451315615e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17873636819422245) - perf/mfu/actor:np.float64(0.0885810936633662) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.26837921142578) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6453125) - val-aux/sciknoweval/reward/std@16:np.float64(0.17485248493267602) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7153499999999999) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13834881246475225) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.57635) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1582729828765162) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6450625000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.17474490419708083) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7678750000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09578158475998025) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5130625) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12920767926064702) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6637124999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13847719028530886) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8036125000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06368274289112907) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.458575) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09684683458113633) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.674125) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09966534252010995) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8278625) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03910595287238987) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4183625) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06058065805524353) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6779125) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06833156224087977) - val-aux/sciknoweval/score/mean@16:np.float64(0.6453125) - val-aux/sciknoweval/score/std@16:np.float64(0.17485248493267602) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7153499999999999) - val-aux/sciknoweval/score/best@2/std:np.float64(0.13834881246475225) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.57635) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1582729828765162) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6450625000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.17474490419708083) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7678750000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09578158475998025) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5130625) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12920767926064702) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6637124999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13847719028530886) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8036125000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06368274289112907) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.458575) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09684683458113633) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.674125) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09966534252010995) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8278625) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03910595287238987) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4183625) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06058065805524353) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6779125) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06833156224087977) - val-core/sciknoweval/acc/mean@16:np.float64(0.6453125) - val-aux/sciknoweval/acc/std@16:np.float64(0.17485248493267602) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7153499999999999) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.13834881246475225) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.57635) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1582729828765162) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6450625000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.17474490419708083) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7678750000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09578158475998025) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5130625) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12920767926064702) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6637124999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13847719028530886) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8036125000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06368274289112907) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.458575) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09684683458113633) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.674125) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09966534252010995) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8278625) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03910595287238987) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4183625) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06058065805524353) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6779125) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06833156224087977) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.978125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.05947208208072883) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9967624999999998) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.02126663775002429) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9593875000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0761225622474269) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9781625) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.059091934676165636) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999375) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0017437187447881502) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9295375) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.08943279499033274) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9918500000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.03609564352248361) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.890025) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.09147875280510498) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9984124999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.014058383946768527) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8474) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.07545396833360063) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99985) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.002693900935493915) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:2 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.016236353665590286 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.016236353665590286 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:715.625 - response_length/max:3780.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:715.625 - response_length_non_aborted/max:3780.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.625 - prompt_length/max:355.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014733336865901947 - timing_s/agent_loop/generate_sequences/min:np.float64(0.98030811175704) - timing_s/agent_loop/generate_sequences/max:np.float64(23.301443008705974) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.870220143442566) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.301443008705974) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:3780 - timing_s/gen:29.260056165978312 - timing_s/reward:0.07223350368440151 - timing_s/old_log_prob:2.538018411025405 - timing_s/adv:0.11729281768202782 - timing_s/update_actor:10.420197248458862 - timing_s/step:42.49460377730429 - timing_s/testing:117.82088489830494 - timing_s/save_checkpoint:6.728960711508989 - timing_s/stop_profile:0.00011482089757919312 - timing_per_token_ms/update_actor:0.04143944566229822 - timing_per_token_ms/gen:0.15971646378809123 - timing_per_token_ms/adv:0.0004664546389110931 - perf/total_num_tokens:251456 - perf/time_per_step:42.49460377730429 - perf/throughput:739.6703864971049 (TaskRunner pid=2049544) Training Progress: 50%|█████ | 50/100 [38:18<1:05:48, 78.97s/it] (TaskRunner pid=2049544) step:51 - global_seqlen/min:18785 - global_seqlen/max:33808 - global_seqlen/minmax_diff:15023 - global_seqlen/balanced_min:26399 - global_seqlen/balanced_max:26471 - global_seqlen/mean:26450.125 - actor/entropy:0.24430422484874725 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4564788341522217 - training/rollout_probs_diff_mean:0.003582285949960351 - training/rollout_probs_diff_std:0.010883934795856476 - training/rollout_actor_probs_pearson_corr:0.999004602432251 - rollout_corr/rollout_is_mean:0.999866783618927 - rollout_corr/rollout_is_ratio_fraction_high:1.396638253936544e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.189914761809632e-05 - rollout_corr/rollout_is_max:2.2966716289520264 - rollout_corr/rollout_is_min:0.327895849943161 - rollout_corr/rollout_is_std:0.033104244619607925 - rollout_corr/rollout_is_eff_sample_size:0.9989051897151183 - rollout_corr/rollout_is_seq_mean:0.9998487830162048 - rollout_corr/rollout_is_seq_std:0.0018385471776127815 - rollout_corr/rollout_is_seq_max:1.0075657367706299 - rollout_corr/rollout_is_seq_min:0.9936858415603638 - rollout_corr/rollout_is_seq_max_deviation:0.007565736770629883 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.222205103840679) - rollout_corr/training_log_ppl:np.float64(0.18761249752424192) - rollout_corr/kl:np.float64(0.0007189881607398974) - rollout_corr/k3_kl:np.float64(0.0005682342473676272) - rollout_corr/rollout_ppl:np.float64(1.2213212260976434) - rollout_corr/rollout_log_ppl:np.float64(0.1868935083039105) - rollout_corr/log_ppl_diff:np.float64(0.000718989220331423) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014408860151888803) - rollout_corr/log_ppl_diff_max:np.float64(0.007139421999454498) - rollout_corr/log_ppl_diff_min:np.float64(-0.005957402288913727) - rollout_corr/ppl_ratio:np.float64(1.0007210047915578) - rollout_corr/chi2_token:np.float64(0.0008357972837984562) - rollout_corr/chi2_seq:np.float64(1.0313024234492332) - actor/pg_loss:np.float64(4.522060044109821e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003210717095498694) - actor/ppo_kl:np.float64(-9.128155866733323e-07) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.14451957494020462) - perf/mfu/actor:np.float64(0.07523053933355461) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.30613327026367) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:2 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011186200194060802 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011186200194060802 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:559.37890625 - response_length/max:2702.0 - response_length/min:100.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:559.37890625 - response_length_non_aborted/max:2702.0 - response_length_non_aborted/min:100.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.1875 - prompt_length/max:356.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.59448516368866e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9070874620229006) - timing_s/agent_loop/generate_sequences/max:np.float64(16.53190640732646) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.214220573740022) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.53190640732646) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:356 - timing_s/agent_loop/slowest/response_length:2702 - timing_s/gen:22.698202308267355 - timing_s/reward:0.06384330801665783 - timing_s/old_log_prob:2.640516960993409 - timing_s/adv:0.15581872314214706 - timing_s/update_actor:10.303586572408676 - timing_s/step:35.9570118393749 - timing_s/stop_profile:0.00012137554585933685 - timing_per_token_ms/update_actor:0.048693468236958595 - timing_per_token_ms/gen:0.1585058924746849 - timing_per_token_ms/adv:0.0007363798996325493 - perf/total_num_tokens:211601 - perf/time_per_step:35.9570118393749 - perf/throughput:735.6040907447059 (TaskRunner pid=2049544) Training Progress: 51%|█████ | 51/100 [38:54<53:58, 66.09s/it] (TaskRunner pid=2049544) step:52 - global_seqlen/min:23429 - global_seqlen/max:36160 - global_seqlen/minmax_diff:12731 - global_seqlen/balanced_min:30668 - global_seqlen/balanced_max:30765 - global_seqlen/mean:30739.125 - actor/entropy:0.22692662477493286 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3938141167163849 - training/rollout_probs_diff_mean:0.0033940812572836876 - training/rollout_probs_diff_std:0.010467931628227234 - training/rollout_actor_probs_pearson_corr:0.9990243315696716 - rollout_corr/rollout_is_mean:1.000043272972107 - rollout_corr/rollout_is_ratio_fraction_high:1.1359441487002186e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.2718882974004373e-05 - rollout_corr/rollout_is_max:2.0554285049438477 - rollout_corr/rollout_is_min:0.29720351099967957 - rollout_corr/rollout_is_std:0.03246244043111801 - rollout_corr/rollout_is_eff_sample_size:0.9989472992270986 - rollout_corr/rollout_is_seq_mean:1.0000196695327759 - rollout_corr/rollout_is_seq_std:0.001915202010422945 - rollout_corr/rollout_is_seq_max:1.0059826374053955 - rollout_corr/rollout_is_seq_min:0.9903362989425659 - rollout_corr/rollout_is_seq_max_deviation:0.009663701057434082 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.231195269152522) - rollout_corr/training_log_ppl:np.float64(0.19834982651809696) - rollout_corr/kl:np.float64(0.0008127379676388102) - rollout_corr/k3_kl:np.float64(0.0006191496557619303) - rollout_corr/rollout_ppl:np.float64(1.230196955613792) - rollout_corr/rollout_log_ppl:np.float64(0.19753708717325935) - rollout_corr/log_ppl_diff:np.float64(0.0008127393448376097) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014782494035898708) - rollout_corr/log_ppl_diff_max:np.float64(0.011166423559188843) - rollout_corr/log_ppl_diff_min:np.float64(-0.00570167601108551) - rollout_corr/ppl_ratio:np.float64(1.0008151046931744) - rollout_corr/chi2_token:np.float64(0.0008716660086065531) - rollout_corr/chi2_seq:np.float64(0.30706389155238867) - actor/pg_loss:np.float64(2.5613000616431236e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015267897435933264) - actor/ppo_kl:np.float64(0.00024156836637168055) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18445459008216858) - perf/mfu/actor:np.float64(0.08808419007285372) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.37739562988281) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:2 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007727969903498888 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007727969903498888 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:687.75390625 - response_length/max:2764.0 - response_length/min:70.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:687.75390625 - response_length_non_aborted/max:2764.0 - response_length_non_aborted/min:70.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.84375 - prompt_length/max:365.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011474080383777618 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7448907978832722) - timing_s/agent_loop/generate_sequences/max:np.float64(18.379133939743042) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.580681977618951) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.379133939743042) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:344 - timing_s/agent_loop/slowest/response_length:2764 - timing_s/gen:24.19850930571556 - timing_s/reward:0.07030453719198704 - timing_s/old_log_prob:2.519341915845871 - timing_s/adv:0.11268062144517899 - timing_s/update_actor:10.240152211859822 - timing_s/step:37.230506498366594 - timing_s/stop_profile:0.00011861883103847504 - timing_per_token_ms/update_actor:0.0416413618306467 - timing_per_token_ms/gen:0.13744077077054248 - timing_per_token_ms/adv:0.0004582133577532663 - perf/total_num_tokens:245913 - perf/time_per_step:37.230506498366594 - perf/throughput:825.643481410832 (TaskRunner pid=2049544) Training Progress: 52%|█████▏ | 52/100 [39:32<45:57, 57.45s/it] (TaskRunner pid=2049544) step:53 - global_seqlen/min:19572 - global_seqlen/max:34404 - global_seqlen/minmax_diff:14832 - global_seqlen/balanced_min:27920 - global_seqlen/balanced_max:29013 - global_seqlen/mean:28188.25 - actor/entropy:0.22438420355319977 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30675768852233887 - training/rollout_probs_diff_mean:0.0035303884651511908 - training/rollout_probs_diff_std:0.010939665138721466 - training/rollout_actor_probs_pearson_corr:0.9988002181053162 - rollout_corr/rollout_is_mean:1.000083327293396 - rollout_corr/rollout_is_ratio_fraction_high:1.938861169037409e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.524009636952542e-05 - rollout_corr/rollout_is_max:2.4050400257110596 - rollout_corr/rollout_is_min:0.11861640214920044 - rollout_corr/rollout_is_std:0.03258340060710907 - rollout_corr/rollout_is_eff_sample_size:0.9989399238583818 - rollout_corr/rollout_is_seq_mean:1.0001201629638672 - rollout_corr/rollout_is_seq_std:0.001550388871692121 - rollout_corr/rollout_is_seq_max:1.0046480894088745 - rollout_corr/rollout_is_seq_min:0.994028627872467 - rollout_corr/rollout_is_seq_max_deviation:0.005971372127532959 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2066223253495991) - rollout_corr/training_log_ppl:np.float64(0.17772597989824135) - rollout_corr/kl:np.float64(0.0005446721567352597) - rollout_corr/k3_kl:np.float64(0.0006030060052921726) - rollout_corr/rollout_ppl:np.float64(1.2059485423378646) - rollout_corr/rollout_log_ppl:np.float64(0.17718130597495474) - rollout_corr/log_ppl_diff:np.float64(0.0005446739232866094) - rollout_corr/log_ppl_abs_diff:np.float64(0.001306041274801828) - rollout_corr/log_ppl_diff_max:np.float64(0.006498947739601135) - rollout_corr/log_ppl_diff_min:np.float64(-0.005862616002559662) - rollout_corr/ppl_ratio:np.float64(1.0005462081171572) - rollout_corr/chi2_token:np.float64(0.001310169231146574) - rollout_corr/chi2_seq:np.float64(0.6170898247510195) - actor/pg_loss:np.float64(5.588738713413477e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00040137838948339777) - actor/ppo_kl:np.float64(0.00010251342614253645) - actor/pg_clipfrac_lower:np.float64(1.1289739632047713e-06) - actor/grad_norm:np.float64(0.2714943513274193) - perf/mfu/actor:np.float64(0.0789630435605575) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.31622695922852) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:2 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02330026589334011 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02330026589334011 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:604.4140625 - response_length/max:3739.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:604.4140625 - response_length_non_aborted/max:3739.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.46875 - prompt_length/max:375.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001577865332365036 - timing_s/agent_loop/generate_sequences/min:np.float64(1.263271963223815) - timing_s/agent_loop/generate_sequences/max:np.float64(22.0587509367615) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.729596399491129) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.0587509367615) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:3739 - timing_s/gen:27.900284377858043 - timing_s/reward:0.0937537681311369 - timing_s/old_log_prob:2.7064144033938646 - timing_s/adv:0.1434136312454939 - timing_s/update_actor:10.502349453046918 - timing_s/step:41.45142296142876 - timing_s/stop_profile:0.00011624768376350403 - timing_per_token_ms/update_actor:0.04657237258896401 - timing_per_token_ms/gen:0.18031593341858748 - timing_per_token_ms/adv:0.0006359637049368704 - perf/total_num_tokens:225506 - perf/time_per_step:41.45142296142876 - perf/throughput:680.030937085794 (TaskRunner pid=2049544) Training Progress: 53%|█████▎ | 53/100 [40:13<41:15, 52.67s/it] (TaskRunner pid=2049544) step:54 - global_seqlen/min:21471 - global_seqlen/max:31367 - global_seqlen/minmax_diff:9896 - global_seqlen/balanced_min:26297 - global_seqlen/balanced_max:26328 - global_seqlen/mean:26318.0 - actor/entropy:0.21818441152572632 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6292486190795898 - training/rollout_probs_diff_mean:0.0032305221538990736 - training/rollout_probs_diff_std:0.010491388849914074 - training/rollout_actor_probs_pearson_corr:0.9989482760429382 - rollout_corr/rollout_is_mean:1.0000078678131104 - rollout_corr/rollout_is_ratio_fraction_high:2.0765270164702088e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.3843512533640023e-05 - rollout_corr/rollout_is_max:2.25616192817688 - rollout_corr/rollout_is_min:0.26139989495277405 - rollout_corr/rollout_is_std:0.03054003044962883 - rollout_corr/rollout_is_eff_sample_size:0.999068175625075 - rollout_corr/rollout_is_seq_mean:0.9999727010726929 - rollout_corr/rollout_is_seq_std:0.001957092434167862 - rollout_corr/rollout_is_seq_max:1.007343053817749 - rollout_corr/rollout_is_seq_min:0.9894382953643799 - rollout_corr/rollout_is_seq_max_deviation:0.010561704635620117 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1926028821617365) - rollout_corr/training_log_ppl:np.float64(0.16857526991225313) - rollout_corr/kl:np.float64(0.0005841684274781045) - rollout_corr/k3_kl:np.float64(0.0005794948536674838) - rollout_corr/rollout_ppl:np.float64(1.1918872729875147) - rollout_corr/rollout_log_ppl:np.float64(0.1679911007740884) - rollout_corr/log_ppl_diff:np.float64(0.0005841691381647252) - rollout_corr/log_ppl_abs_diff:np.float64(0.001538367178000044) - rollout_corr/log_ppl_diff_max:np.float64(0.011501207947731018) - rollout_corr/log_ppl_diff_min:np.float64(-0.005205966532230377) - rollout_corr/ppl_ratio:np.float64(1.0005866582505405) - rollout_corr/chi2_token:np.float64(0.0011341231875121593) - rollout_corr/chi2_seq:np.float64(1.4627508698031306) - actor/pg_loss:np.float64(0.00015633960720151663) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00042431396536812827) - actor/ppo_kl:np.float64(3.235917171728886e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16759618930518627) - perf/mfu/actor:np.float64(0.07638956882275981) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.35180282592773) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:2 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008146907202899456 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008146907202899456 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:564.34375 - response_length/max:2093.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:564.34375 - response_length_non_aborted/max:2093.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.09375 - prompt_length/max:363.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015513785183429718 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6626291759312153) - timing_s/agent_loop/generate_sequences/max:np.float64(14.199840543791652) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.107884573328192) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.199840543791652) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:326 - timing_s/agent_loop/slowest/response_length:2093 - timing_s/gen:19.986812436953187 - timing_s/reward:0.06590797752141953 - timing_s/old_log_prob:2.560090212151408 - timing_s/adv:0.13934014923870564 - timing_s/update_actor:10.21920620650053 - timing_s/step:33.05696338787675 - timing_s/stop_profile:0.00011862441897392273 - timing_per_token_ms/update_actor:0.04853715236007927 - timing_per_token_ms/gen:0.13834384819863493 - timing_per_token_ms/adv:0.0006618101168340378 - perf/total_num_tokens:210544 - perf/time_per_step:33.05696338787675 - perf/throughput:796.1408823670663 (TaskRunner pid=2049544) Training Progress: 54%|█████▍ | 54/100 [40:46<35:52, 46.80s/it] (TaskRunner pid=2049544) step:55 - global_seqlen/min:18191 - global_seqlen/max:33987 - global_seqlen/minmax_diff:15796 - global_seqlen/balanced_min:26610 - global_seqlen/balanced_max:26684 - global_seqlen/mean:26660.125 - actor/entropy:0.1736723929643631 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105892539024353 - training/rollout_probs_diff_mean:0.0030401991680264473 - training/rollout_probs_diff_std:0.01039278693497181 - training/rollout_actor_probs_pearson_corr:0.9987287521362305 - rollout_corr/rollout_is_mean:1.0000579357147217 - rollout_corr/rollout_is_ratio_fraction_high:1.375430656480603e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.375430656480603e-05 - rollout_corr/rollout_is_max:2.335618734359741 - rollout_corr/rollout_is_min:0.392137348651886 - rollout_corr/rollout_is_std:0.030324585735797882 - rollout_corr/rollout_is_eff_sample_size:0.999081502372468 - rollout_corr/rollout_is_seq_mean:1.000092625617981 - rollout_corr/rollout_is_seq_std:0.00183792260941118 - rollout_corr/rollout_is_seq_max:1.0104801654815674 - rollout_corr/rollout_is_seq_min:0.9918577671051025 - rollout_corr/rollout_is_seq_max_deviation:0.010480165481567383 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1669259667396545) - rollout_corr/training_log_ppl:np.float64(0.14700542857826804) - rollout_corr/kl:np.float64(0.0007857797350844242) - rollout_corr/k3_kl:np.float64(0.0007165672559068526) - rollout_corr/rollout_ppl:np.float64(1.1659750957041979) - rollout_corr/rollout_log_ppl:np.float64(0.14621964823891176) - rollout_corr/log_ppl_diff:np.float64(0.000785780339356279) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016149269104062114) - rollout_corr/log_ppl_diff_max:np.float64(0.058644767850637436) - rollout_corr/log_ppl_diff_min:np.float64(-0.004740942269563675) - rollout_corr/ppl_ratio:np.float64(1.00079559488222) - rollout_corr/chi2_token:np.float64(0.0010453276336193085) - rollout_corr/chi2_seq:np.float64(0.6579223412554711) - actor/pg_loss:np.float64(-0.00018631992861628532) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005601525168685839) - actor/ppo_kl:np.float64(0.00037079197026113064) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.25370683521032333) - perf/mfu/actor:np.float64(0.0767427855529022) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.34751892089844) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:2 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0002467178856022656 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0002467178856022656 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:568.00390625 - response_length/max:2618.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:568.00390625 - response_length_non_aborted/max:2618.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.125 - prompt_length/max:383.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001354403793811798 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8230457752943039) - timing_s/agent_loop/generate_sequences/max:np.float64(16.547815980389714) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.079917931871023) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.547815980389714) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:308 - timing_s/agent_loop/slowest/response_length:2618 - timing_s/gen:22.890313317999244 - timing_s/reward:0.06661977060139179 - timing_s/old_log_prob:2.436913600191474 - timing_s/adv:0.1175360195338726 - timing_s/update_actor:10.17019310221076 - timing_s/step:35.7683734241873 - timing_s/stop_profile:0.0001267455518245697 - timing_per_token_ms/update_actor:0.04768447776506468 - timing_per_token_ms/gen:0.15742019626019876 - timing_per_token_ms/adv:0.0005510852796726975 - perf/total_num_tokens:213281 - perf/time_per_step:35.7683734241873 - perf/throughput:745.3546932042451 (TaskRunner pid=2049544) Training Progress: 55%|█████▌ | 55/100 [41:22<32:37, 43.51s/it] (TaskRunner pid=2049544) step:56 - global_seqlen/min:22164 - global_seqlen/max:48288 - global_seqlen/minmax_diff:26124 - global_seqlen/balanced_min:28371 - global_seqlen/balanced_max:28431 - global_seqlen/mean:28409.375 - actor/entropy:0.21692201495170593 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3174453377723694 - training/rollout_probs_diff_mean:0.003212681272998452 - training/rollout_probs_diff_std:0.010248677805066109 - training/rollout_actor_probs_pearson_corr:0.9989055395126343 - rollout_corr/rollout_is_mean:1.0000497102737427 - rollout_corr/rollout_is_ratio_fraction_high:1.2312617400311865e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.309416181058623e-05 - rollout_corr/rollout_is_max:2.551107883453369 - rollout_corr/rollout_is_min:0.3708029091358185 - rollout_corr/rollout_is_std:0.031356628984212875 - rollout_corr/rollout_is_eff_sample_size:0.9990177275875637 - rollout_corr/rollout_is_seq_mean:1.0000814199447632 - rollout_corr/rollout_is_seq_std:0.002115494105964899 - rollout_corr/rollout_is_seq_max:1.0103763341903687 - rollout_corr/rollout_is_seq_min:0.9925733208656311 - rollout_corr/rollout_is_seq_max_deviation:0.010376334190368652 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2074726885184646) - rollout_corr/training_log_ppl:np.float64(0.18105585960438475) - rollout_corr/kl:np.float64(0.0007089410478542391) - rollout_corr/k3_kl:np.float64(0.0007255429266592728) - rollout_corr/rollout_ppl:np.float64(1.2066095937043428) - rollout_corr/rollout_log_ppl:np.float64(0.18034691788489) - rollout_corr/log_ppl_diff:np.float64(0.00070894171949476) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016144337132573128) - rollout_corr/log_ppl_diff_max:np.float64(0.014080479741096497) - rollout_corr/log_ppl_diff_min:np.float64(-0.00835387408733368) - rollout_corr/ppl_ratio:np.float64(1.0007117404602468) - rollout_corr/chi2_token:np.float64(0.0016396085266023874) - rollout_corr/chi2_seq:np.float64(0.7452940770890564) - actor/pg_loss:np.float64(0.00023605907335877419) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00041431330055274884) - actor/ppo_kl:np.float64(0.00019277200499256963) - actor/pg_clipfrac_lower:np.float64(1.5439723938470706e-05) - actor/grad_norm:np.float64(0.27160290628671646) - perf/mfu/actor:np.float64(0.08149403640203684) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.39385986328125) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:2 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0020954536739736795 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0020954536739736795 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:634.51171875 - response_length/max:2697.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:634.51171875 - response_length_non_aborted/max:2697.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:253.28125 - prompt_length/max:355.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.834898471832275e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9801836125552654) - timing_s/agent_loop/generate_sequences/max:np.float64(17.489308523014188) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.853927971802477) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.489308523014188) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:315 - timing_s/agent_loop/slowest/response_length:2659 - timing_s/gen:23.2441786210984 - timing_s/reward:0.08269395306706429 - timing_s/old_log_prob:2.530390776693821 - timing_s/adv:0.136021314188838 - timing_s/update_actor:10.16875221952796 - timing_s/step:36.25585407577455 - timing_s/stop_profile:0.00013040564954280853 - timing_per_token_ms/update_actor:0.04474206234529957 - timing_per_token_ms/gen:0.14309833854217624 - timing_per_token_ms/adv:0.0005984877975529117 - perf/total_num_tokens:227275 - perf/time_per_step:36.25585407577455 - perf/throughput:783.5803547924854 (TaskRunner pid=2049544) Training Progress: 56%|█████▌ | 56/100 [41:59<30:19, 41.35s/it] (TaskRunner pid=2049544) step:57 - global_seqlen/min:25396 - global_seqlen/max:39989 - global_seqlen/minmax_diff:14593 - global_seqlen/balanced_min:32645 - global_seqlen/balanced_max:32684 - global_seqlen/mean:32673.25 - actor/entropy:0.20889751613140106 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23668746650218964 - training/rollout_probs_diff_mean:0.0030729498248547316 - training/rollout_probs_diff_std:0.009970693849027157 - training/rollout_actor_probs_pearson_corr:0.9990144371986389 - rollout_corr/rollout_is_mean:0.9999489188194275 - rollout_corr/rollout_is_ratio_fraction_high:5.2733157644979656e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.691321035148576e-05 - rollout_corr/rollout_is_max:2.685802698135376 - rollout_corr/rollout_is_min:0.4615940749645233 - rollout_corr/rollout_is_std:0.03016890585422516 - rollout_corr/rollout_is_eff_sample_size:0.9990906647175409 - rollout_corr/rollout_is_seq_mean:0.9998156428337097 - rollout_corr/rollout_is_seq_std:0.0015230864519253373 - rollout_corr/rollout_is_seq_max:1.0045884847640991 - rollout_corr/rollout_is_seq_min:0.9938079118728638 - rollout_corr/rollout_is_seq_max_deviation:0.0061920881271362305 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2169137438759208) - rollout_corr/training_log_ppl:np.float64(0.18850286441738717) - rollout_corr/kl:np.float64(0.0008272747242870082) - rollout_corr/k3_kl:np.float64(0.0005567002334601057) - rollout_corr/rollout_ppl:np.float64(1.2158889258280396) - rollout_corr/rollout_log_ppl:np.float64(0.18767558728723088) - rollout_corr/log_ppl_diff:np.float64(0.0008272771301562898) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012787482482963242) - rollout_corr/log_ppl_diff_max:np.float64(0.00949886441230774) - rollout_corr/log_ppl_diff_min:np.float64(-0.003404974937438965) - rollout_corr/ppl_ratio:np.float64(1.000829064520076) - rollout_corr/chi2_token:np.float64(0.0005316368769854307) - rollout_corr/chi2_seq:np.float64(0.33462699479423463) - actor/pg_loss:np.float64(4.014407750219107e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00038230796246807586) - actor/ppo_kl:np.float64(0.00015057841544852124) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21619294956326485) - perf/mfu/actor:np.float64(0.09399730782746046) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.41035079956055) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:2 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012718578800559044 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012718578800559044 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:740.7578125 - response_length/max:2402.0 - response_length/min:102.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:740.7578125 - response_length_non_aborted/max:2402.0 - response_length_non_aborted/min:102.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.28125 - prompt_length/max:382.0 - prompt_length/min:209.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010364875197410583 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0282678361982107) - timing_s/agent_loop/generate_sequences/max:np.float64(16.842207921668887) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.293985813957988) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.842207921668887) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:361 - timing_s/agent_loop/slowest/response_length:2402 - timing_s/gen:22.68095701187849 - timing_s/reward:0.07416719943284988 - timing_s/old_log_prob:2.519518541172147 - timing_s/adv:0.15157914347946644 - timing_s/update_actor:10.281581053510308 - timing_s/step:35.79420953243971 - timing_s/stop_profile:0.00011889636516571045 - timing_per_token_ms/update_actor:0.03933485746562673 - timing_per_token_ms/gen:0.11960385274728419 - timing_per_token_ms/adv:0.0005799053640189851 - perf/total_num_tokens:261386 - perf/time_per_step:35.79420953243971 - perf/throughput:912.8082566089012 (TaskRunner pid=2049544) Training Progress: 57%|█████▋ | 57/100 [42:34<28:26, 39.69s/it] (TaskRunner pid=2049544) step:58 - global_seqlen/min:19068 - global_seqlen/max:41428 - global_seqlen/minmax_diff:22360 - global_seqlen/balanced_min:32655 - global_seqlen/balanced_max:33260 - global_seqlen/mean:32761.875 - actor/entropy:0.25287628173828125 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.42262715101242065 - training/rollout_probs_diff_mean:0.0032485839910805225 - training/rollout_probs_diff_std:0.009784280322492123 - training/rollout_actor_probs_pearson_corr:0.9992220997810364 - rollout_corr/rollout_is_mean:0.9999250769615173 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.117093729088083e-05 - rollout_corr/rollout_is_max:1.704056739807129 - rollout_corr/rollout_is_min:0.30506160855293274 - rollout_corr/rollout_is_std:0.030359944328665733 - rollout_corr/rollout_is_eff_sample_size:0.9990791225700723 - rollout_corr/rollout_is_seq_mean:0.9998817443847656 - rollout_corr/rollout_is_seq_std:0.0017060940153896809 - rollout_corr/rollout_is_seq_max:1.0051213502883911 - rollout_corr/rollout_is_seq_min:0.9917200803756714 - rollout_corr/rollout_is_seq_max_deviation:0.008279919624328613 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.248608773574233) - rollout_corr/training_log_ppl:np.float64(0.2106571579060983) - rollout_corr/kl:np.float64(0.0006302413064700474) - rollout_corr/k3_kl:np.float64(0.0005439822076880318) - rollout_corr/rollout_ppl:np.float64(1.2478208504617214) - rollout_corr/rollout_log_ppl:np.float64(0.21002691524336115) - rollout_corr/log_ppl_diff:np.float64(0.000630242662737146) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013482350041158497) - rollout_corr/log_ppl_diff_max:np.float64(0.007139943540096283) - rollout_corr/log_ppl_diff_min:np.float64(-0.009232334792613983) - rollout_corr/ppl_ratio:np.float64(1.0006320539396256) - rollout_corr/chi2_token:np.float64(0.0009030492510646582) - rollout_corr/chi2_seq:np.float64(2.417929224204272) - actor/pg_loss:np.float64(-9.389116894453764e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003193014416638107) - actor/ppo_kl:np.float64(-1.695539558355108e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21667541190981865) - perf/mfu/actor:np.float64(0.09071205098155613) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.45818328857422) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:2 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013659363612532616 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013659363612532616 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:751.90234375 - response_length/max:4193.0 - response_length/min:62.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:751.90234375 - response_length_non_aborted/max:4193.0 - response_length_non_aborted/min:62.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.90625 - prompt_length/max:341.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.30633395910263e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7540517617017031) - timing_s/agent_loop/generate_sequences/max:np.float64(25.78281068429351) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.984791398346715) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(25.78281068429351) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:4193 - timing_s/gen:31.48099230416119 - timing_s/reward:0.07489402964711189 - timing_s/old_log_prob:2.5185533091425896 - timing_s/adv:0.12432781234383583 - timing_s/update_actor:10.674950456246734 - timing_s/step:44.96027087979019 - timing_s/stop_profile:0.00012622401118278503 - timing_per_token_ms/update_actor:0.04072931744690564 - timing_per_token_ms/gen:0.16354866720433686 - timing_per_token_ms/adv:0.00047436163354446225 - perf/total_num_tokens:262095 - perf/time_per_step:44.96027087979019 - perf/throughput:728.6850003104983 (TaskRunner pid=2049544) Training Progress: 58%|█████▊ | 58/100 [43:19<28:54, 41.29s/it] (TaskRunner pid=2049544) step:59 - global_seqlen/min:23178 - global_seqlen/max:36252 - global_seqlen/minmax_diff:13074 - global_seqlen/balanced_min:28566 - global_seqlen/balanced_max:28612 - global_seqlen/mean:28596.625 - actor/entropy:0.24240879714488983 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3797503709793091 - training/rollout_probs_diff_mean:0.0033322961535304785 - training/rollout_probs_diff_std:0.010173420421779156 - training/rollout_actor_probs_pearson_corr:0.9990172982215881 - rollout_corr/rollout_is_mean:1.0001167058944702 - rollout_corr/rollout_is_ratio_fraction_high:6.2438730310532264e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.873162000265438e-05 - rollout_corr/rollout_is_max:2.413045644760132 - rollout_corr/rollout_is_min:0.38240286707878113 - rollout_corr/rollout_is_std:0.03103756159543991 - rollout_corr/rollout_is_eff_sample_size:0.9990379537831949 - rollout_corr/rollout_is_seq_mean:1.0002449750900269 - rollout_corr/rollout_is_seq_std:0.0013697257963940501 - rollout_corr/rollout_is_seq_max:1.003889560699463 - rollout_corr/rollout_is_seq_min:0.9955812692642212 - rollout_corr/rollout_is_seq_max_deviation:0.004418730735778809 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2532247374765575) - rollout_corr/training_log_ppl:np.float64(0.21026221416104818) - rollout_corr/kl:np.float64(0.00036410490782223803) - rollout_corr/k3_kl:np.float64(0.0005213920420086282) - rollout_corr/rollout_ppl:np.float64(1.2527254964224994) - rollout_corr/rollout_log_ppl:np.float64(0.20989810848550405) - rollout_corr/log_ppl_diff:np.float64(0.0003641056755441241) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011843454485642724) - rollout_corr/log_ppl_diff_max:np.float64(0.0051978230476379395) - rollout_corr/log_ppl_diff_min:np.float64(-0.004664614796638489) - rollout_corr/ppl_ratio:np.float64(1.000365313142538) - rollout_corr/chi2_token:np.float64(0.0013532442972064018) - rollout_corr/chi2_seq:np.float64(1.167969731381163) - actor/pg_loss:np.float64(5.0825648941099644e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027953678352332645) - actor/ppo_kl:np.float64(0.0001204940012513589) - actor/pg_clipfrac_lower:np.float64(2.0624340777430916e-06) - actor/grad_norm:np.float64(0.2315312847495079) - perf/mfu/actor:np.float64(0.08222550993023878) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.56365966796875) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:2 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0007726792828179896 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0007726792828179896 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:625.61328125 - response_length/max:2175.0 - response_length/min:80.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:625.61328125 - response_length_non_aborted/max:2175.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.03125 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.861107587814331e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0195303186774254) - timing_s/agent_loop/generate_sequences/max:np.float64(15.18119196780026) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.312842677085428) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.18119196780026) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:274 - timing_s/agent_loop/slowest/response_length:2156 - timing_s/gen:21.4505631364882 - timing_s/reward:0.06381673738360405 - timing_s/old_log_prob:2.4540651980787516 - timing_s/adv:0.12210911512374878 - timing_s/update_actor:10.219576759263873 - timing_s/step:34.34806403145194 - timing_s/stop_profile:0.00012100301682949066 - timing_per_token_ms/update_actor:0.04467125385978185 - timing_per_token_ms/gen:0.13393459628045104 - timing_per_token_ms/adv:0.0005337566720012797 - perf/total_num_tokens:228773 - perf/time_per_step:34.34806403145194 - perf/throughput:832.5542008368959 (TaskRunner pid=2049544) Training Progress: 59%|█████▉ | 59/100 [43:54<26:48, 39.22s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) (TaskRunner pid=2049544) - Electric field $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2049544) - Distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2049544) - Coulomb's constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We are to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the known values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0) \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.5}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest option is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50/actor (TaskRunner pid=2049544) step:60 - global_seqlen/min:19702 - global_seqlen/max:41022 - global_seqlen/minmax_diff:21320 - global_seqlen/balanced_min:29482 - global_seqlen/balanced_max:29782 - global_seqlen/mean:29534.0 - actor/entropy:0.2471179962158203 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2885955274105072 - training/rollout_probs_diff_mean:0.0032362197525799274 - training/rollout_probs_diff_std:0.009918652474880219 - training/rollout_actor_probs_pearson_corr:0.9991265535354614 - rollout_corr/rollout_is_mean:1.0000226497650146 - rollout_corr/rollout_is_ratio_fraction_high:1.1968021681241225e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1968021681241225e-05 - rollout_corr/rollout_is_max:2.9771621227264404 - rollout_corr/rollout_is_min:0.3293769657611847 - rollout_corr/rollout_is_std:0.03091634064912796 - rollout_corr/rollout_is_eff_sample_size:0.999045211617763 - rollout_corr/rollout_is_seq_mean:0.9999569654464722 - rollout_corr/rollout_is_seq_std:0.0014956871746107936 - rollout_corr/rollout_is_seq_max:1.0054192543029785 - rollout_corr/rollout_is_seq_min:0.9950917959213257 - rollout_corr/rollout_is_seq_max_deviation:0.005419254302978516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2401416734792292) - rollout_corr/training_log_ppl:np.float64(0.19959818758798065) - rollout_corr/kl:np.float64(0.0005349192640053957) - rollout_corr/k3_kl:np.float64(0.0004940067541099324) - rollout_corr/rollout_ppl:np.float64(1.2394655891694129) - rollout_corr/rollout_log_ppl:np.float64(0.19906326734781032) - rollout_corr/log_ppl_diff:np.float64(0.0005349202401703224) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011775598395615816) - rollout_corr/log_ppl_diff_max:np.float64(0.005937367677688599) - rollout_corr/log_ppl_diff_min:np.float64(-0.004511777311563492) - rollout_corr/ppl_ratio:np.float64(1.000536180799827) - rollout_corr/chi2_token:np.float64(0.0009251013398170471) - rollout_corr/chi2_seq:np.float64(1.390581222018227) - actor/pg_loss:np.float64(4.893657751381397e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018130719581677113) - actor/ppo_kl:np.float64(-2.2069814953340483e-07) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15835927985608578) - perf/mfu/actor:np.float64(0.08403507336330182) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.51084518432617) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6375) - val-aux/sciknoweval/reward/std@16:np.float64(0.1808793393521641) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.71125) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.14531694283181013) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5644) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15827951046753036) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6389125) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.18130674840602382) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.766225) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1033040821613348) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.49958749999999996) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12475642320545952) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6604125) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1416569094891499) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8033625000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07008229463050868) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.449375) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08561471923061226) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6742) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10267672443906181) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8312625) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04984537510833955) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.41555) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.052249733074818304) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6821875000000001) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.076862250314467) - val-aux/sciknoweval/score/mean@16:np.float64(0.6375) - val-aux/sciknoweval/score/std@16:np.float64(0.1808793393521641) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.71125) - val-aux/sciknoweval/score/best@2/std:np.float64(0.14531694283181013) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5644) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15827951046753036) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6389125) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.18130674840602382) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.766225) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1033040821613348) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.49958749999999996) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12475642320545952) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6604125) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1416569094891499) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8033625000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07008229463050868) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.449375) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08561471923061226) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6742) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10267672443906181) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8312625) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04984537510833955) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.41555) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.052249733074818304) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6821875000000001) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.076862250314467) - val-core/sciknoweval/acc/mean@16:np.float64(0.6375) - val-aux/sciknoweval/acc/std@16:np.float64(0.1808793393521641) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.71125) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.14531694283181013) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5644) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15827951046753036) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6389125) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.18130674840602382) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.766225) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1033040821613348) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.49958749999999996) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12475642320545952) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6604125) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1416569094891499) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8033625000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07008229463050868) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.449375) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08561471923061226) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6742) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10267672443906181) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8312625) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04984537510833955) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.41555) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.052249733074818304) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6821875000000001) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.076862250314467) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9765625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0574668098874074) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9954875000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.022722407353494737) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9575250000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.07239005753165481) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9770249999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.05686531821374918) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9995499999999999) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.004893080696751963) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9285500000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.08215960170640288) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9901500000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.03655386464803364) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9999750000000001) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0005584576975922169) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8929375) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.08171927324120949) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9971125000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.017114100398696087) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8558125000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.06612062868202828) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9996750000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.00411263922012108) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:2 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00966417696326971 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.00966417696326971 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:652.78125 - response_length/max:3139.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:652.78125 - response_length_non_aborted/max:3139.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.15625 - prompt_length/max:340.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.66247171163559e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2835856825113297) - timing_s/agent_loop/generate_sequences/max:np.float64(20.483323162421584) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.35434487216844) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.483323162421584) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:197 - timing_s/agent_loop/slowest/response_length:3139 - timing_s/gen:26.403195202350616 - timing_s/reward:0.06402513571083546 - timing_s/old_log_prob:2.5449625328183174 - timing_s/adv:0.11957781203091145 - timing_s/update_actor:10.040485488250852 - timing_s/step:39.238921258598566 - timing_s/testing:114.50520241633058 - timing_s/save_checkpoint:6.870979869738221 - timing_s/stop_profile:0.00015047751367092133 - timing_per_token_ms/update_actor:0.04249545222561646 - timing_per_token_ms/gen:0.15799700322149587 - timing_per_token_ms/adv:0.0005061023398071351 - perf/total_num_tokens:236272 - perf/time_per_step:39.238921258598566 - perf/throughput:752.6710483542691 (TaskRunner pid=2049544) Training Progress: 60%|██████ | 60/100 [46:34<50:26, 75.65s/it] (TaskRunner pid=2049544) step:61 - global_seqlen/min:22238 - global_seqlen/max:39486 - global_seqlen/minmax_diff:17248 - global_seqlen/balanced_min:29924 - global_seqlen/balanced_max:30089 - global_seqlen/mean:29970.875 - actor/entropy:0.23904968798160553 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5477294921875 - training/rollout_probs_diff_mean:0.003164143767207861 - training/rollout_probs_diff_std:0.00985527690500021 - training/rollout_actor_probs_pearson_corr:0.9991676211357117 - rollout_corr/rollout_is_mean:1.0000258684158325 - rollout_corr/rollout_is_ratio_fraction_high:1.1471639481897e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.720745967759285e-05 - rollout_corr/rollout_is_max:4.02189826965332 - rollout_corr/rollout_is_min:0.26447275280952454 - rollout_corr/rollout_is_std:0.030506834387779236 - rollout_corr/rollout_is_eff_sample_size:0.9990700794243639 - rollout_corr/rollout_is_seq_mean:1.0000176429748535 - rollout_corr/rollout_is_seq_std:0.0016575142508372664 - rollout_corr/rollout_is_seq_max:1.0063189268112183 - rollout_corr/rollout_is_seq_min:0.9920255541801453 - rollout_corr/rollout_is_seq_max_deviation:0.007974445819854736 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.25040641753003) - rollout_corr/training_log_ppl:np.float64(0.20799311240261886) - rollout_corr/kl:np.float64(0.0005623800304297433) - rollout_corr/k3_kl:np.float64(0.0005192289710862497) - rollout_corr/rollout_ppl:np.float64(1.2496482259593904) - rollout_corr/rollout_log_ppl:np.float64(0.20743073029007064) - rollout_corr/log_ppl_diff:np.float64(0.0005623821125482209) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012774796996382065) - rollout_corr/log_ppl_diff_max:np.float64(0.009329631924629211) - rollout_corr/log_ppl_diff_min:np.float64(-0.007050305604934692) - rollout_corr/ppl_ratio:np.float64(1.0005640257149935) - rollout_corr/chi2_token:np.float64(0.0009777885861694813) - rollout_corr/chi2_seq:np.float64(0.3470004736445844) - actor/pg_loss:np.float64(-0.0002099055564031005) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002631939267985217) - actor/ppo_kl:np.float64(6.142829495781044e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.23834596201777458) - perf/mfu/actor:np.float64(0.08600884737044563) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.00771045684814) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:2 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007454414386302233 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.007454414386302233 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:681.02734375 - response_length/max:3128.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:681.02734375 - response_length_non_aborted/max:3128.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.5625 - prompt_length/max:328.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015644170343875885 - timing_s/agent_loop/generate_sequences/min:np.float64(0.93056246265769) - timing_s/agent_loop/generate_sequences/max:np.float64(20.83841021731496) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.5293213194963755) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.83841021731496) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:246 - timing_s/agent_loop/slowest/response_length:3128 - timing_s/gen:26.53979305177927 - timing_s/reward:0.08145093731582165 - timing_s/old_log_prob:2.588765997439623 - timing_s/adv:0.11384812742471695 - timing_s/update_actor:10.279441229999065 - timing_s/step:39.70467999763787 - timing_s/stop_profile:0.0001134183257818222 - timing_per_token_ms/update_actor:0.04287262730066717 - timing_per_token_ms/gen:0.1522274656956647 - timing_per_token_ms/adv:0.0004748281766244602 - perf/total_num_tokens:239767 - perf/time_per_step:39.70467999763787 - perf/throughput:754.8448949036497 (TaskRunner pid=2049544) Training Progress: 61%|██████ | 61/100 [47:14<42:10, 64.88s/it] (TaskRunner pid=2049544) step:62 - global_seqlen/min:24330 - global_seqlen/max:43162 - global_seqlen/minmax_diff:18832 - global_seqlen/balanced_min:34742 - global_seqlen/balanced_max:34794 - global_seqlen/mean:34771.375 - actor/entropy:0.2732798457145691 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4058573246002197 - training/rollout_probs_diff_mean:0.00327512389048934 - training/rollout_probs_diff_std:0.009542555548250675 - training/rollout_actor_probs_pearson_corr:0.9991864562034607 - rollout_corr/rollout_is_mean:0.9999688267707825 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.467516540287761e-05 - rollout_corr/rollout_is_max:1.9362671375274658 - rollout_corr/rollout_is_min:0.37468504905700684 - rollout_corr/rollout_is_std:0.030330481007695198 - rollout_corr/rollout_is_eff_sample_size:0.9990807884305588 - rollout_corr/rollout_is_seq_mean:0.9999213814735413 - rollout_corr/rollout_is_seq_std:0.0014323026407510042 - rollout_corr/rollout_is_seq_max:1.0065138339996338 - rollout_corr/rollout_is_seq_min:0.9906108975410461 - rollout_corr/rollout_is_seq_max_deviation:0.009389102458953857 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2884436114691198) - rollout_corr/training_log_ppl:np.float64(0.23855835630820366) - rollout_corr/kl:np.float64(0.0005924100263499099) - rollout_corr/k3_kl:np.float64(0.0005057925414178044) - rollout_corr/rollout_ppl:np.float64(1.2876463397406042) - rollout_corr/rollout_log_ppl:np.float64(0.23796594546729466) - rollout_corr/log_ppl_diff:np.float64(0.0005924108409089968) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011562938307179138) - rollout_corr/log_ppl_diff_max:np.float64(0.009115807712078094) - rollout_corr/log_ppl_diff_min:np.float64(-0.0043695345520973206) - rollout_corr/ppl_ratio:np.float64(1.0005937188398093) - rollout_corr/chi2_token:np.float64(0.0008554272353649139) - rollout_corr/chi2_seq:np.float64(0.8644126462750137) - actor/pg_loss:np.float64(-5.7160970754921436e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00029845362780633877) - actor/ppo_kl:np.float64(3.140159503445261e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.22896912693977356) - perf/mfu/actor:np.float64(0.09839975519482527) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.0562973022461) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:2 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.022515371441841125 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.022515371441841125 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:798.54296875 - response_length/max:3135.0 - response_length/min:72.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:798.54296875 - response_length_non_aborted/max:3135.0 - response_length_non_aborted/min:72.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:288.0625 - prompt_length/max:460.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014179199934005737 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8601270448416471) - timing_s/agent_loop/generate_sequences/max:np.float64(20.932036239653826) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.75760180906218) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.932036239653826) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:3135 - timing_s/gen:27.040857832878828 - timing_s/reward:0.07787876017391682 - timing_s/old_log_prob:2.5947242341935635 - timing_s/adv:0.12395207956433296 - timing_s/update_actor:10.449679136276245 - timing_s/step:40.37802067771554 - timing_s/stop_profile:0.0001217275857925415 - timing_per_token_ms/update_actor:0.03756566693248486 - timing_per_token_ms/gen:0.1322763521104298 - timing_per_token_ms/adv:0.0004455966997434418 - perf/total_num_tokens:278171 - perf/time_per_step:40.37802067771554 - perf/throughput:861.1460991992154 (TaskRunner pid=2049544) Training Progress: 62%|██████▏ | 62/100 [47:55<36:26, 57.55s/it] (TaskRunner pid=2049544) step:63 - global_seqlen/min:21359 - global_seqlen/max:35870 - global_seqlen/minmax_diff:14511 - global_seqlen/balanced_min:26722 - global_seqlen/balanced_max:27254 - global_seqlen/mean:26812.0 - actor/entropy:0.27676644921302795 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9831006526947021 - training/rollout_probs_diff_mean:0.0035199327394366264 - training/rollout_probs_diff_std:0.010530827566981316 - training/rollout_actor_probs_pearson_corr:0.9990900158882141 - rollout_corr/rollout_is_mean:0.9999789595603943 - rollout_corr/rollout_is_ratio_fraction_high:1.3679142284672707e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.419785571168177e-05 - rollout_corr/rollout_is_max:2.8213930130004883 - rollout_corr/rollout_is_min:0.011052249930799007 - rollout_corr/rollout_is_std:0.031423088163137436 - rollout_corr/rollout_is_eff_sample_size:0.9990135634724805 - rollout_corr/rollout_is_seq_mean:1.0000073909759521 - rollout_corr/rollout_is_seq_std:0.001668434706516564 - rollout_corr/rollout_is_seq_max:1.0121815204620361 - rollout_corr/rollout_is_seq_min:0.9929383993148804 - rollout_corr/rollout_is_seq_max_deviation:0.012181520462036133 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2705106027424335) - rollout_corr/training_log_ppl:np.float64(0.22211689058167394) - rollout_corr/kl:np.float64(0.0004739185238458177) - rollout_corr/k3_kl:np.float64(0.0005098591802266128) - rollout_corr/rollout_ppl:np.float64(1.2698905346915126) - rollout_corr/rollout_log_ppl:np.float64(0.2216429725958733) - rollout_corr/log_ppl_diff:np.float64(0.0004739179858006537) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012847277976106852) - rollout_corr/log_ppl_diff_max:np.float64(0.0076948776841163635) - rollout_corr/log_ppl_diff_min:np.float64(-0.010769776999950409) - rollout_corr/ppl_ratio:np.float64(1.0004755130503327) - rollout_corr/chi2_token:np.float64(0.001110320445150137) - rollout_corr/chi2_seq:np.float64(0.7298937705345452) - actor/pg_loss:np.float64(-6.4569758251309395e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017891856134610862) - actor/ppo_kl:np.float64(-4.981012998683809e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.19124767929315567) - perf/mfu/actor:np.float64(0.07698739538780973) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.1104507446289) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:2 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018338599475100636 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0018338599475100636 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:571.125 - response_length/max:3010.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:571.125 - response_length_non_aborted/max:3010.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.75 - prompt_length/max:350.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001445487141609192 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8224132973700762) - timing_s/agent_loop/generate_sequences/max:np.float64(18.348940243944526) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.530061706544075) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.348940243944526) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:268 - timing_s/agent_loop/slowest/response_length:3010 - timing_s/gen:25.11938327550888 - timing_s/reward:0.06868074834346771 - timing_s/old_log_prob:2.4906132221221924 - timing_s/adv:0.12919375859200954 - timing_s/update_actor:10.244162807241082 - timing_s/step:38.14616190828383 - timing_s/stop_profile:0.00011005811393260956 - timing_per_token_ms/update_actor:0.047759225380618206 - timing_per_token_ms/gen:0.17180580594433192 - timing_per_token_ms/adv:0.0006023131368044604 - perf/total_num_tokens:214496 - perf/time_per_step:38.14616190828383 - perf/throughput:702.8754312023591 (TaskRunner pid=2049544) Training Progress: 63%|██████▎ | 63/100 [48:33<31:54, 51.74s/it] (TaskRunner pid=2049544) step:64 - global_seqlen/min:26498 - global_seqlen/max:40322 - global_seqlen/minmax_diff:13824 - global_seqlen/balanced_min:31563 - global_seqlen/balanced_max:31618 - global_seqlen/mean:31602.875 - actor/entropy:0.28700023889541626 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3535178303718567 - training/rollout_probs_diff_mean:0.0033518956042826176 - training/rollout_probs_diff_std:0.009729590266942978 - training/rollout_actor_probs_pearson_corr:0.9992855787277222 - rollout_corr/rollout_is_mean:1.0000327825546265 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0833825399458874e-05 - rollout_corr/rollout_is_max:1.9838535785675049 - rollout_corr/rollout_is_min:0.4158313572406769 - rollout_corr/rollout_is_std:0.03056929260492325 - rollout_corr/rollout_is_eff_sample_size:0.9990665098066491 - rollout_corr/rollout_is_seq_mean:0.9999979734420776 - rollout_corr/rollout_is_seq_std:0.0013619516976177692 - rollout_corr/rollout_is_seq_max:1.0039030313491821 - rollout_corr/rollout_is_seq_min:0.9954256415367126 - rollout_corr/rollout_is_seq_max_deviation:0.0045743584632873535 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2764493175782263) - rollout_corr/training_log_ppl:np.float64(0.22693806317693088) - rollout_corr/kl:np.float64(0.00046152420509582015) - rollout_corr/k3_kl:np.float64(0.0005074531937054871) - rollout_corr/rollout_ppl:np.float64(1.275848121382296) - rollout_corr/rollout_log_ppl:np.float64(0.22647653929016087) - rollout_corr/log_ppl_diff:np.float64(0.00046152388677001) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011273344280198216) - rollout_corr/log_ppl_diff_max:np.float64(0.00530514121055603) - rollout_corr/log_ppl_diff_min:np.float64(-0.005483843386173248) - rollout_corr/ppl_ratio:np.float64(1.0004626805894077) - rollout_corr/chi2_token:np.float64(0.0011098254472017288) - rollout_corr/chi2_seq:np.float64(1.8452809148002416) - actor/pg_loss:np.float64(-0.00011544267181307077) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002127377759961746) - actor/ppo_kl:np.float64(-4.476623224114462e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18739718943834305) - perf/mfu/actor:np.float64(0.08994342294998352) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.98906707763672) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:2 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013745415955781937 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013745415955781937 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:721.12109375 - response_length/max:2725.0 - response_length/min:175.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:721.12109375 - response_length_non_aborted/max:2725.0 - response_length_non_aborted/min:175.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:349.0 - prompt_length/min:192.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.722448885440826e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.080433465540409) - timing_s/agent_loop/generate_sequences/max:np.float64(17.78822729177773) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.979315571334155) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.78822729177773) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:2560 - timing_s/gen:23.619467133656144 - timing_s/reward:0.07297545671463013 - timing_s/old_log_prob:2.6043873876333237 - timing_s/adv:0.14066699519753456 - timing_s/update_actor:10.391787450760603 - timing_s/step:36.91719730384648 - timing_s/stop_profile:0.0001245010644197464 - timing_per_token_ms/update_actor:0.041103014562601514 - timing_per_token_ms/gen:0.127944591124151 - timing_per_token_ms/adv:0.0005563852782283833 - perf/total_num_tokens:252823 - perf/time_per_step:36.91719730384648 - perf/throughput:856.047514655378 (TaskRunner pid=2049544) Training Progress: 64%|██████▍ | 64/100 [49:10<28:23, 47.32s/it] (TaskRunner pid=2049544) step:65 - global_seqlen/min:13817 - global_seqlen/max:44402 - global_seqlen/minmax_diff:30585 - global_seqlen/balanced_min:27087 - global_seqlen/balanced_max:27398 - global_seqlen/mean:27214.25 - actor/entropy:0.2703191936016083 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27730000019073486 - training/rollout_probs_diff_mean:0.003337751142680645 - training/rollout_probs_diff_std:0.009712927974760532 - training/rollout_actor_probs_pearson_corr:0.999212384223938 - rollout_corr/rollout_is_mean:0.9999141097068787 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.3306897017173469e-05 - rollout_corr/rollout_is_max:1.8171900510787964 - rollout_corr/rollout_is_min:0.37625378370285034 - rollout_corr/rollout_is_std:0.030340304598212242 - rollout_corr/rollout_is_eff_sample_size:0.999079955499621 - rollout_corr/rollout_is_seq_mean:1.0000232458114624 - rollout_corr/rollout_is_seq_std:0.001954729435965419 - rollout_corr/rollout_is_seq_max:1.0170458555221558 - rollout_corr/rollout_is_seq_min:0.9947062730789185 - rollout_corr/rollout_is_seq_max_deviation:0.01704585552215576 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2311511896550655) - rollout_corr/training_log_ppl:np.float64(0.1959650574644911) - rollout_corr/kl:np.float64(0.0005068062779431948) - rollout_corr/k3_kl:np.float64(0.0005566824616494159) - rollout_corr/rollout_ppl:np.float64(1.2305298056453466) - rollout_corr/rollout_log_ppl:np.float64(0.1954582512989873) - rollout_corr/log_ppl_diff:np.float64(0.0005068061655038036) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013243403445812874) - rollout_corr/log_ppl_diff_max:np.float64(0.007357165217399597) - rollout_corr/log_ppl_diff_min:np.float64(-0.020653128623962402) - rollout_corr/ppl_ratio:np.float64(1.0005090332124382) - rollout_corr/chi2_token:np.float64(0.0013379347510635853) - rollout_corr/chi2_seq:np.float64(0.5852078427560627) - actor/pg_loss:np.float64(5.363975651562214e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001393496023638363) - actor/ppo_kl:np.float64(4.77582690940892e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17264313623309135) - perf/mfu/actor:np.float64(0.07762399981838528) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.82825469970703) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:2 - critic/score/mean:0.890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0034032389521598816 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.0034032389521598816 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:587.1015625 - response_length/max:3738.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:587.1015625 - response_length_non_aborted/max:3738.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.34375 - prompt_length/max:364.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.999820470809937e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0192465409636497) - timing_s/agent_loop/generate_sequences/max:np.float64(20.993049459531903) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.240891806744912) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.993049459531903) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:3738 - timing_s/gen:26.803196189925075 - timing_s/reward:0.06635245122015476 - timing_s/old_log_prob:2.5201703682541847 - timing_s/adv:0.12346276640892029 - timing_s/update_actor:10.165898667648435 - timing_s/step:39.771548522636294 - timing_s/stop_profile:0.00011387281119823456 - timing_per_token_ms/update_actor:0.04669382156245549 - timing_per_token_ms/gen:0.17833368501194344 - timing_per_token_ms/adv:0.0005670869416248853 - perf/total_num_tokens:217714 - perf/time_per_step:39.771548522636294 - perf/throughput:684.2642796397729 (TaskRunner pid=2049544) Training Progress: 65%|██████▌ | 65/100 [49:50<26:17, 45.07s/it] (TaskRunner pid=2049544) step:66 - global_seqlen/min:26398 - global_seqlen/max:47072 - global_seqlen/minmax_diff:20674 - global_seqlen/balanced_min:34031 - global_seqlen/balanced_max:34100 - global_seqlen/mean:34083.75 - actor/entropy:0.32043859362602234 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4526241421699524 - training/rollout_probs_diff_mean:0.003443863708525896 - training/rollout_probs_diff_std:0.009328007698059082 - training/rollout_actor_probs_pearson_corr:0.999345064163208 - rollout_corr/rollout_is_mean:0.999977171421051 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9896735466318205e-05 - rollout_corr/rollout_is_max:1.9577182531356812 - rollout_corr/rollout_is_min:0.13161711394786835 - rollout_corr/rollout_is_std:0.030255712568759918 - rollout_corr/rollout_is_eff_sample_size:0.9990853100798829 - rollout_corr/rollout_is_seq_mean:1.0000357627868652 - rollout_corr/rollout_is_seq_std:0.0014041649410501122 - rollout_corr/rollout_is_seq_max:1.0082844495773315 - rollout_corr/rollout_is_seq_min:0.9953852295875549 - rollout_corr/rollout_is_seq_max_deviation:0.008284449577331543 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3104135780595243) - rollout_corr/training_log_ppl:np.float64(0.25584780907956883) - rollout_corr/kl:np.float64(0.0005620165548441136) - rollout_corr/k3_kl:np.float64(0.0004995921321437891) - rollout_corr/rollout_ppl:np.float64(1.309668519999832) - rollout_corr/rollout_log_ppl:np.float64(0.2552857908885926) - rollout_corr/log_ppl_diff:np.float64(0.0005620181909762323) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011583589657675475) - rollout_corr/log_ppl_diff_max:np.float64(0.005684643983840942) - rollout_corr/log_ppl_diff_min:np.float64(-0.006929904222488403) - rollout_corr/ppl_ratio:np.float64(1.000563226873055) - rollout_corr/chi2_token:np.float64(0.0008636738639324903) - rollout_corr/chi2_seq:np.float64(0.8563104090280831) - actor/pg_loss:np.float64(0.0001647728495299816) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017283655381561402) - actor/ppo_kl:np.float64(0.00013175434309165013) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1924263872206211) - perf/mfu/actor:np.float64(0.09599912196819169) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.91796875) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:2 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.000762293697334826 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.000762293697334826 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:785.3046875 - response_length/max:3216.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:785.3046875 - response_length_non_aborted/max:3216.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.8125 - prompt_length/max:365.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.582083344459534e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2348940391093493) - timing_s/agent_loop/generate_sequences/max:np.float64(21.131182158365846) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.555415921531676) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.131182158365846) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:3216 - timing_s/gen:26.812409596517682 - timing_s/reward:0.11229918338358402 - timing_s/old_log_prob:2.537730697542429 - timing_s/adv:0.12344273924827576 - timing_s/update_actor:10.477383820340037 - timing_s/step:40.15382357686758 - timing_s/stop_profile:0.00011853314936161041 - timing_per_token_ms/update_actor:0.03842514328800395 - timing_per_token_ms/gen:0.1333698584174021 - timing_per_token_ms/adv:0.00045271844811778253 - perf/total_num_tokens:272670 - perf/time_per_step:40.15382357686758 - perf/throughput:848.8295002530091 (TaskRunner pid=2049544) Training Progress: 66%|██████▌ | 66/100 [50:30<24:42, 43.61s/it] (TaskRunner pid=2049544) step:67 - global_seqlen/min:14820 - global_seqlen/max:39555 - global_seqlen/minmax_diff:24735 - global_seqlen/balanced_min:25556 - global_seqlen/balanced_max:26212 - global_seqlen/mean:25672.375 - actor/entropy:0.27359628677368164 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.46210977435112 - training/rollout_probs_diff_mean:0.0033997194841504097 - training/rollout_probs_diff_std:0.009928795509040356 - training/rollout_actor_probs_pearson_corr:0.9991419315338135 - rollout_corr/rollout_is_mean:1.0000401735305786 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.1833584469277412e-05 - rollout_corr/rollout_is_max:1.9785306453704834 - rollout_corr/rollout_is_min:0.3678695261478424 - rollout_corr/rollout_is_std:0.031190816313028336 - rollout_corr/rollout_is_eff_sample_size:0.9990281975159966 - rollout_corr/rollout_is_seq_mean:1.0001834630966187 - rollout_corr/rollout_is_seq_std:0.002221266273409128 - rollout_corr/rollout_is_seq_max:1.0159006118774414 - rollout_corr/rollout_is_seq_min:0.9894429445266724 - rollout_corr/rollout_is_seq_max_deviation:0.015900611877441406 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3023830708116293) - rollout_corr/training_log_ppl:np.float64(0.2422099069954129) - rollout_corr/kl:np.float64(0.00026683149832429365) - rollout_corr/k3_kl:np.float64(0.000541509634160775) - rollout_corr/rollout_ppl:np.float64(1.301996021065861) - rollout_corr/rollout_log_ppl:np.float64(0.241943074419396) - rollout_corr/log_ppl_diff:np.float64(0.0002668325760168955) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014319048641482368) - rollout_corr/log_ppl_diff_max:np.float64(0.009274959564208984) - rollout_corr/log_ppl_diff_min:np.float64(-0.0138968825340271) - rollout_corr/ppl_ratio:np.float64(1.0002689892426133) - rollout_corr/chi2_token:np.float64(0.0016502379439771175) - rollout_corr/chi2_seq:np.float64(1.019609775627032) - actor/pg_loss:np.float64(-5.0529371947050095e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016554813828406623) - actor/ppo_kl:np.float64(-7.827897409384832e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17443475499749184) - perf/mfu/actor:np.float64(0.07407053273139065) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.82710266113281) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:3 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01963839866220951 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01963839866220951 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:536.73046875 - response_length/max:3340.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:536.73046875 - response_length_non_aborted/max:3340.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.53125 - prompt_length/max:361.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00023074820637702942 - timing_s/agent_loop/generate_sequences/min:np.float64(1.033557141199708) - timing_s/agent_loop/generate_sequences/max:np.float64(19.718276670202613) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.309584286827885) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.718276670202613) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:314 - timing_s/agent_loop/slowest/response_length:3340 - timing_s/gen:25.747549824416637 - timing_s/reward:0.05820099823176861 - timing_s/old_log_prob:2.587189307436347 - timing_s/adv:0.1540521327406168 - timing_s/update_actor:10.0051255710423 - timing_s/step:38.650182999670506 - timing_s/stop_profile:0.0001270286738872528 - timing_per_token_ms/update_actor:0.04871542646055487 - timing_per_token_ms/gen:0.18738710089602584 - timing_per_token_ms/adv:0.00075008707190422 - perf/total_num_tokens:205379 - perf/time_per_step:38.650182999670506 - perf/throughput:664.2238925548905 (TaskRunner pid=2049544) Training Progress: 67%|██████▋ | 67/100 [51:09<23:14, 42.26s/it] (TaskRunner pid=2049544) step:68 - global_seqlen/min:21420 - global_seqlen/max:36370 - global_seqlen/minmax_diff:14950 - global_seqlen/balanced_min:26998 - global_seqlen/balanced_max:27019 - global_seqlen/mean:27007.25 - actor/entropy:0.2508779466152191 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4620983600616455 - training/rollout_probs_diff_mean:0.003281368175521493 - training/rollout_probs_diff_std:0.009971853345632553 - training/rollout_actor_probs_pearson_corr:0.9991183876991272 - rollout_corr/rollout_is_mean:0.9999003410339355 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.37541350745596e-05 - rollout_corr/rollout_is_max:1.727583885192871 - rollout_corr/rollout_is_min:0.3678779602050781 - rollout_corr/rollout_is_std:0.03009769693017006 - rollout_corr/rollout_is_eff_sample_size:0.9990948294754013 - rollout_corr/rollout_is_seq_mean:0.999991238117218 - rollout_corr/rollout_is_seq_std:0.0015605590306222439 - rollout_corr/rollout_is_seq_max:1.005854606628418 - rollout_corr/rollout_is_seq_min:0.9957244396209717 - rollout_corr/rollout_is_seq_max_deviation:0.005854606628417969 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2427316303364933) - rollout_corr/training_log_ppl:np.float64(0.2052579416122171) - rollout_corr/kl:np.float64(0.0006245356830429927) - rollout_corr/k3_kl:np.float64(0.0004809370777678623) - rollout_corr/rollout_ppl:np.float64(1.2419258821755648) - rollout_corr/rollout_log_ppl:np.float64(0.20463340327114565) - rollout_corr/log_ppl_diff:np.float64(0.0006245383410714567) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013257119280751795) - rollout_corr/log_ppl_diff_max:np.float64(0.005434125661849976) - rollout_corr/log_ppl_diff_min:np.float64(-0.0038042664527893066) - rollout_corr/ppl_ratio:np.float64(1.0006259840447456) - rollout_corr/chi2_token:np.float64(0.0006594203878194094) - rollout_corr/chi2_seq:np.float64(0.44312575249932706) - actor/pg_loss:np.float64(-5.7110912166535854e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002576871158908034) - actor/ppo_kl:np.float64(0.000146509498037517) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1650708168745041) - perf/mfu/actor:np.float64(0.07864794728282518) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.82149505615234) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:3 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00015104975318536162 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00015104975318536162 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:578.6328125 - response_length/max:1942.0 - response_length/min:89.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:578.6328125 - response_length_non_aborted/max:1942.0 - response_length_non_aborted/min:89.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.34375 - prompt_length/max:345.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.873881936073303e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0787797346711159) - timing_s/agent_loop/generate_sequences/max:np.float64(13.899392530322075) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.687431053593173) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.899392530322075) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:341 - timing_s/agent_loop/slowest/response_length:1942 - timing_s/gen:19.621694235131145 - timing_s/reward:0.06307606771588326 - timing_s/old_log_prob:2.414205303415656 - timing_s/adv:0.1221406627446413 - timing_s/update_actor:10.004864446818829 - timing_s/step:32.31770410388708 - timing_s/stop_profile:4.9777328968048096e-05 - timing_per_token_ms/update_actor:0.04630638276212327 - timing_per_token_ms/gen:0.13246266276332375 - timing_per_token_ms/adv:0.0005653142338846111 - perf/total_num_tokens:216058 - perf/time_per_step:32.31770410388708 - perf/throughput:835.6797225812722 (TaskRunner pid=2049544) Training Progress: 68%|██████▊ | 68/100 [51:41<20:57, 39.29s/it] (TaskRunner pid=2049544) step:69 - global_seqlen/min:23146 - global_seqlen/max:36753 - global_seqlen/minmax_diff:13607 - global_seqlen/balanced_min:29343 - global_seqlen/balanced_max:29374 - global_seqlen/mean:29363.125 - actor/entropy:0.2744224965572357 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24491748213768005 - training/rollout_probs_diff_mean:0.0033163181506097317 - training/rollout_probs_diff_std:0.00957618560642004 - training/rollout_actor_probs_pearson_corr:0.9992418885231018 - rollout_corr/rollout_is_mean:0.9999679327011108 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9769123792648315 - rollout_corr/rollout_is_min:0.547244668006897 - rollout_corr/rollout_is_std:0.030115514993667603 - rollout_corr/rollout_is_eff_sample_size:0.9990938775276863 - rollout_corr/rollout_is_seq_mean:1.000019907951355 - rollout_corr/rollout_is_seq_std:0.001533227856270969 - rollout_corr/rollout_is_seq_max:1.0057690143585205 - rollout_corr/rollout_is_seq_min:0.990231990814209 - rollout_corr/rollout_is_seq_max_deviation:0.009768009185791016 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2841168828308582) - rollout_corr/training_log_ppl:np.float64(0.23235505366756115) - rollout_corr/kl:np.float64(0.0005125461427646361) - rollout_corr/k3_kl:np.float64(0.0005065118215270559) - rollout_corr/rollout_ppl:np.float64(1.2834058650769293) - rollout_corr/rollout_log_ppl:np.float64(0.2318425054399995) - rollout_corr/log_ppl_diff:np.float64(0.0005125482275616378) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012335704232100397) - rollout_corr/log_ppl_diff_max:np.float64(0.011537343263626099) - rollout_corr/log_ppl_diff_min:np.float64(-0.005053341388702393) - rollout_corr/ppl_ratio:np.float64(1.0005140437278897) - rollout_corr/chi2_token:np.float64(0.00102224200963974) - rollout_corr/chi2_seq:np.float64(0.8634938539471477) - actor/pg_loss:np.float64(-5.3855590522289276e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000181908647164164) - actor/ppo_kl:np.float64(5.1798504737021744e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18398911505937576) - perf/mfu/actor:np.float64(0.08486034559374442) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.82257843017578) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:3 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015627924352884293 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015627924352884293 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:646.87890625 - response_length/max:2353.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:646.87890625 - response_length_non_aborted/max:2353.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:363.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.917755722999573e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.203237522393465) - timing_s/agent_loop/generate_sequences/max:np.float64(16.094254648312926) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.346816844648856) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.094254648312926) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:260 - timing_s/agent_loop/slowest/response_length:2353 - timing_s/gen:22.094875816255808 - timing_s/reward:0.05950486660003662 - timing_s/old_log_prob:2.4449626710265875 - timing_s/adv:0.12635387293994427 - timing_s/update_actor:10.193446822464466 - timing_s/step:34.9527602866292 - timing_s/stop_profile:2.9163435101509094e-05 - timing_per_token_ms/update_actor:0.04339391167690967 - timing_per_token_ms/gen:0.13342235745107703 - timing_per_token_ms/adv:0.0005378935013726581 - perf/total_num_tokens:234905 - perf/time_per_step:34.9527602866292 - perf/throughput:840.0802900603117 (TaskRunner pid=2049544) Training Progress: 69%|██████▉ | 69/100 [52:16<19:37, 37.99s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) (TaskRunner pid=2049544) - Electric field $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2049544) - Distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2049544) - Coulomb's constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We are to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) = \frac{0.5}{8.99 \times 10^9} (TaskRunner pid=2049544) \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60/actor (TaskRunner pid=2049544) step:70 - global_seqlen/min:18463 - global_seqlen/max:39933 - global_seqlen/minmax_diff:21470 - global_seqlen/balanced_min:27119 - global_seqlen/balanced_max:27177 - global_seqlen/mean:27165.625 - actor/entropy:0.34878426790237427 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35282185673713684 - training/rollout_probs_diff_mean:0.003718312829732895 - training/rollout_probs_diff_std:0.009631498716771603 - training/rollout_actor_probs_pearson_corr:0.999345600605011 - rollout_corr/rollout_is_mean:1.000012993812561 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.0160612621111795e-05 - rollout_corr/rollout_is_max:1.975199818611145 - rollout_corr/rollout_is_min:0.19730398058891296 - rollout_corr/rollout_is_std:0.03194605931639671 - rollout_corr/rollout_is_eff_sample_size:0.9989806087004571 - rollout_corr/rollout_is_seq_mean:1.0000489950180054 - rollout_corr/rollout_is_seq_std:0.0017589139752089977 - rollout_corr/rollout_is_seq_max:1.0087381601333618 - rollout_corr/rollout_is_seq_min:0.992647647857666 - rollout_corr/rollout_is_seq_max_deviation:0.008738160133361816 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3483932986855507) - rollout_corr/training_log_ppl:np.float64(0.2762820063435356) - rollout_corr/kl:np.float64(0.0006076907076355553) - rollout_corr/k3_kl:np.float64(0.0005228528713416836) - rollout_corr/rollout_ppl:np.float64(1.347519775852561) - rollout_corr/rollout_log_ppl:np.float64(0.27567431312490953) - rollout_corr/log_ppl_diff:np.float64(0.0006076932186260819) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012910107034258544) - rollout_corr/log_ppl_diff_max:np.float64(0.009554415941238403) - rollout_corr/log_ppl_diff_min:np.float64(-0.007596790790557861) - rollout_corr/ppl_ratio:np.float64(1.0006093357224017) - rollout_corr/chi2_token:np.float64(0.0008673274423927069) - rollout_corr/chi2_seq:np.float64(0.4545943159610033) - actor/pg_loss:np.float64(4.701910074800253e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021670004468887782) - actor/ppo_kl:np.float64(0.0001372517156355002) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2546451985836029) - perf/mfu/actor:np.float64(0.07849234372480743) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.89488983154297) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.63828125) - val-aux/sciknoweval/reward/std@16:np.float64(0.18977686952476835) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7111625) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15430243477041897) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.56455) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1662157425344542) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6377124999999999) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.19070491787466565) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7680125) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11699107745399087) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5010874999999999) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1336089734386206) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6575625) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1445571378929248) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8131625) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08665177217150197) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.44706250000000003) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10195007783293546) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6680875000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09773915943586196) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8506) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.061461459372971074) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.40423749999999997) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07051862491369472) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6758124999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06817587534083876) - val-aux/sciknoweval/score/mean@16:np.float64(0.63828125) - val-aux/sciknoweval/score/std@16:np.float64(0.18977686952476835) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7111625) - val-aux/sciknoweval/score/best@2/std:np.float64(0.15430243477041897) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.56455) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1662157425344542) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6377124999999999) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.19070491787466565) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7680125) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11699107745399087) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5010874999999999) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1336089734386206) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6575625) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1445571378929248) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8131625) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08665177217150197) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.44706250000000003) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10195007783293546) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6680875000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09773915943586196) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8506) - val-aux/sciknoweval/score/best@16/std:np.float64(0.061461459372971074) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.40423749999999997) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07051862491369472) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6758124999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06817587534083876) - val-core/sciknoweval/acc/mean@16:np.float64(0.63828125) - val-aux/sciknoweval/acc/std@16:np.float64(0.18977686952476835) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7111625) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.15430243477041897) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.56455) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1662157425344542) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6377124999999999) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.19070491787466565) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7680125) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11699107745399087) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5010874999999999) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1336089734386206) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6575625) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1445571378929248) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8131625) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08665177217150197) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.44706250000000003) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10195007783293546) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6680875000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09773915943586196) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8506) - val-core/sciknoweval/acc/best@16/std:np.float64(0.061461459372971074) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.40423749999999997) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07051862491369472) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6758124999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06817587534083876) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.041294709008662854) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9961125) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.016882665811088163) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.966925) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.04991881605672479) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9813124999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.04115652748695172) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9996875) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.004110435910939457) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9449500000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.05482807368910221) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9914250000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.027274560942092924) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.92125) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.05037233272292474) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9971) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.01160302730045884) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8982125) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.037001893862708676) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9991249999999999) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.005012406355781736) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:3 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0005754174781031907 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0005754174781031907 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:581.26953125 - response_length/max:2622.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:581.26953125 - response_length_non_aborted/max:2622.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.65625 - prompt_length/max:375.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7524307370185852e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.118723189458251) - timing_s/agent_loop/generate_sequences/max:np.float64(16.3854166790843) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.583953251349158) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.3854166790843) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:2622 - timing_s/gen:22.14735895767808 - timing_s/reward:0.0681693684309721 - timing_s/old_log_prob:2.4311879333108664 - timing_s/adv:0.11663812026381493 - timing_s/update_actor:9.943649336695671 - timing_s/step:34.796333791688085 - timing_s/testing:115.27732796221972 - timing_s/save_checkpoint:6.6401550360023975 - timing_s/stop_profile:0.000131901353597641 - timing_per_token_ms/update_actor:0.04575474214515436 - timing_per_token_ms/gen:0.1488347767728106 - timing_per_token_ms/adv:0.0005366990464227075 - perf/total_num_tokens:217325 - perf/time_per_step:34.796333791688085 - perf/throughput:780.7036558112666 (TaskRunner pid=2049544) Training Progress: 70%|███████ | 70/100 [54:53<36:48, 73.62s/it] (TaskRunner pid=2049544) step:71 - global_seqlen/min:20204 - global_seqlen/max:38376 - global_seqlen/minmax_diff:18172 - global_seqlen/balanced_min:28759 - global_seqlen/balanced_max:29243 - global_seqlen/mean:28856.5 - actor/entropy:0.32371971011161804 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35350361466407776 - training/rollout_probs_diff_mean:0.003485669381916523 - training/rollout_probs_diff_std:0.009408628568053246 - training/rollout_actor_probs_pearson_corr:0.9993824362754822 - rollout_corr/rollout_is_mean:1.0000401735305786 - rollout_corr/rollout_is_ratio_fraction_high:6.257665518205613e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.257665518205613e-06 - rollout_corr/rollout_is_max:3.121652126312256 - rollout_corr/rollout_is_min:0.4736731946468353 - rollout_corr/rollout_is_std:0.03072100132703781 - rollout_corr/rollout_is_eff_sample_size:0.9990571099353601 - rollout_corr/rollout_is_seq_mean:1.0000032186508179 - rollout_corr/rollout_is_seq_std:0.0015029723290354013 - rollout_corr/rollout_is_seq_max:1.0061728954315186 - rollout_corr/rollout_is_seq_min:0.9937267303466797 - rollout_corr/rollout_is_seq_max_deviation:0.0062732696533203125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3052765680477023) - rollout_corr/training_log_ppl:np.float64(0.24597317739971913) - rollout_corr/kl:np.float64(0.00043000688371730433) - rollout_corr/k3_kl:np.float64(0.000540082649621354) - rollout_corr/rollout_ppl:np.float64(1.3047043732367456) - rollout_corr/rollout_log_ppl:np.float64(0.2455431708949618) - rollout_corr/log_ppl_diff:np.float64(0.0004300065047573298) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012972943659406155) - rollout_corr/log_ppl_diff_max:np.float64(0.00797504186630249) - rollout_corr/log_ppl_diff_min:np.float64(-0.0062088742852211) - rollout_corr/ppl_ratio:np.float64(1.0004315802361816) - rollout_corr/chi2_token:np.float64(0.0013474705629050732) - rollout_corr/chi2_seq:np.float64(1.4930782679002732) - actor/pg_loss:np.float64(0.0001349709928035736) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00038389957057916035) - actor/ppo_kl:np.float64(-3.772802561208266e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17576251924037933) - perf/mfu/actor:np.float64(0.08244675822096596) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.2421646118164) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:3 - critic/score/mean:0.75 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009262127801775932 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009262127801775932 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:624.234375 - response_length/max:3404.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:624.234375 - response_length_non_aborted/max:3404.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:437.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012946873903274536 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2765609622001648) - timing_s/agent_loop/generate_sequences/max:np.float64(20.10214041545987) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.958212114004709) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.10214041545987) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:3404 - timing_s/gen:25.802896425127983 - timing_s/reward:0.07092655450105667 - timing_s/old_log_prob:2.582106838002801 - timing_s/adv:0.1316581778228283 - timing_s/update_actor:10.102314312011003 - timing_s/step:38.781780099496245 - timing_s/stop_profile:0.00012562982738018036 - timing_per_token_ms/update_actor:0.04376099974014089 - timing_per_token_ms/gen:0.16146589838256853 - timing_per_token_ms/adv:0.0005703142178661146 - perf/total_num_tokens:230852 - perf/time_per_step:38.781780099496245 - perf/throughput:744.0736326689355 (TaskRunner pid=2049544) Training Progress: 71%|███████ | 71/100 [55:32<30:32, 63.19s/it] (TaskRunner pid=2049544) step:72 - global_seqlen/min:24019 - global_seqlen/max:35372 - global_seqlen/minmax_diff:11353 - global_seqlen/balanced_min:30681 - global_seqlen/balanced_max:31067 - global_seqlen/mean:30756.75 - actor/entropy:0.2652339041233063 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5334677696228027 - training/rollout_probs_diff_mean:0.003048401791602373 - training/rollout_probs_diff_std:0.009212449193000793 - training/rollout_actor_probs_pearson_corr:0.9992727637290955 - rollout_corr/rollout_is_mean:1.0000463724136353 - rollout_corr/rollout_is_ratio_fraction_high:1.1159842870256398e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.6739762941142544e-05 - rollout_corr/rollout_is_max:2.703545570373535 - rollout_corr/rollout_is_min:0.2786954641342163 - rollout_corr/rollout_is_std:0.02862177975475788 - rollout_corr/rollout_is_eff_sample_size:0.9991814643117487 - rollout_corr/rollout_is_seq_mean:1.0000512599945068 - rollout_corr/rollout_is_seq_std:0.0012803305871784687 - rollout_corr/rollout_is_seq_max:1.005553960800171 - rollout_corr/rollout_is_seq_min:0.9965124726295471 - rollout_corr/rollout_is_seq_max_deviation:0.0055539608001708984 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.272761202417314) - rollout_corr/training_log_ppl:np.float64(0.2287760855979286) - rollout_corr/kl:np.float64(0.0003352667156821809) - rollout_corr/k3_kl:np.float64(0.0004365081685619998) - rollout_corr/rollout_ppl:np.float64(1.2723114336840808) - rollout_corr/rollout_log_ppl:np.float64(0.22844081881339662) - rollout_corr/log_ppl_diff:np.float64(0.00033526678453199565) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010565653501544148) - rollout_corr/log_ppl_diff_max:np.float64(0.00969955325126648) - rollout_corr/log_ppl_diff_min:np.float64(-0.005003869533538818) - rollout_corr/ppl_ratio:np.float64(1.0003363261930645) - rollout_corr/chi2_token:np.float64(0.0010922125075012445) - rollout_corr/chi2_seq:np.float64(1.312244726344943) - actor/pg_loss:np.float64(8.467817679047585e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000129777238043971) - actor/ppo_kl:np.float64(-3.5065210759910315e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1640554666519165) - perf/mfu/actor:np.float64(0.08815191325739583) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.06222915649414) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:3 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006141400896012783 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006141400896012783 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:700.0546875 - response_length/max:3313.0 - response_length/min:189.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:700.0546875 - response_length_non_aborted/max:3313.0 - response_length_non_aborted/min:189.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.09375 - prompt_length/max:355.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013216771185398102 - timing_s/agent_loop/generate_sequences/min:np.float64(2.07568240724504) - timing_s/agent_loop/generate_sequences/max:np.float64(20.67868852801621) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.946097736770753) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.67868852801621) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:246 - timing_s/agent_loop/slowest/response_length:3313 - timing_s/gen:26.445967487990856 - timing_s/reward:0.0720598753541708 - timing_s/old_log_prob:2.519180826842785 - timing_s/adv:0.12123813852667809 - timing_s/update_actor:10.210091643035412 - timing_s/step:39.45813396573067 - timing_s/stop_profile:0.00012371689081192017 - timing_per_token_ms/update_actor:0.04149532884259314 - timing_per_token_ms/gen:0.14756641494520994 - timing_per_token_ms/adv:0.0004927298012902781 - perf/total_num_tokens:246054 - perf/time_per_step:39.45813396573067 - perf/throughput:779.4780672272083 (TaskRunner pid=2049544) Training Progress: 72%|███████▏ | 72/100 [56:11<26:10, 56.08s/it] (TaskRunner pid=2049544) step:73 - global_seqlen/min:21925 - global_seqlen/max:46347 - global_seqlen/minmax_diff:24422 - global_seqlen/balanced_min:36563 - global_seqlen/balanced_max:36628 - global_seqlen/mean:36610.375 - actor/entropy:0.3390262722969055 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43396425247192383 - training/rollout_probs_diff_mean:0.0032661668956279755 - training/rollout_probs_diff_std:0.008885001763701439 - training/rollout_actor_probs_pearson_corr:0.9994308948516846 - rollout_corr/rollout_is_mean:0.9999864101409912 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.7025687813875265e-05 - rollout_corr/rollout_is_max:1.9699597358703613 - rollout_corr/rollout_is_min:0.2820616662502289 - rollout_corr/rollout_is_std:0.029777150601148605 - rollout_corr/rollout_is_eff_sample_size:0.9991141068069422 - rollout_corr/rollout_is_seq_mean:0.9999426007270813 - rollout_corr/rollout_is_seq_std:0.0015595207223668694 - rollout_corr/rollout_is_seq_max:1.0064419507980347 - rollout_corr/rollout_is_seq_min:0.992620587348938 - rollout_corr/rollout_is_seq_max_deviation:0.007379412651062012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4027120932005346) - rollout_corr/training_log_ppl:np.float64(0.31371801071509253) - rollout_corr/kl:np.float64(0.000663529713321509) - rollout_corr/k3_kl:np.float64(0.0005334890627182176) - rollout_corr/rollout_ppl:np.float64(1.4016525098122656) - rollout_corr/rollout_log_ppl:np.float64(0.3130544779851334) - rollout_corr/log_ppl_diff:np.float64(0.0006635327299591154) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013385674392338842) - rollout_corr/log_ppl_diff_max:np.float64(0.008049964904785156) - rollout_corr/log_ppl_diff_min:np.float64(-0.005723848938941956) - rollout_corr/ppl_ratio:np.float64(1.0006653110031039) - rollout_corr/chi2_token:np.float64(0.0007862187922000885) - rollout_corr/chi2_seq:np.float64(0.7408852751832455) - actor/pg_loss:np.float64(-9.118067100644112e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00025507958866910485) - actor/ppo_kl:np.float64(9.864521248204028e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.22031916305422783) - perf/mfu/actor:np.float64(0.10239718785831436) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.18194198608398) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:3 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006957425735890865 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006957425735890865 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:867.23046875 - response_length/max:3153.0 - response_length/min:79.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:867.23046875 - response_length_non_aborted/max:3153.0 - response_length_non_aborted/min:79.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.84375 - prompt_length/max:375.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010944902896881104 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9543074388056993) - timing_s/agent_loop/generate_sequences/max:np.float64(21.861840976402164) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.284751043880533) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.861840976402164) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:309 - timing_s/agent_loop/slowest/response_length:3153 - timing_s/gen:27.61041198298335 - timing_s/reward:0.07980250380933285 - timing_s/old_log_prob:2.6316978093236685 - timing_s/adv:0.14758859016001225 - timing_s/update_actor:10.437388250604272 - timing_s/step:40.99476144276559 - timing_s/stop_profile:0.00011257268488407135 - timing_per_token_ms/update_actor:0.03563671585788274 - timing_per_token_ms/gen:0.12436506291572648 - timing_per_token_ms/adv:0.00050391654742683 - perf/total_num_tokens:292883 - perf/time_per_step:40.99476144276559 - perf/throughput:893.0500803404648 (TaskRunner pid=2049544) Training Progress: 73%|███████▎ | 73/100 [56:52<23:12, 51.57s/it] (TaskRunner pid=2049544) step:74 - global_seqlen/min:18401 - global_seqlen/max:45796 - global_seqlen/minmax_diff:27395 - global_seqlen/balanced_min:30702 - global_seqlen/balanced_max:30981 - global_seqlen/mean:30746.0 - actor/entropy:0.31715160608291626 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4399150013923645 - training/rollout_probs_diff_mean:0.0034521878696978092 - training/rollout_probs_diff_std:0.009627916850149632 - training/rollout_actor_probs_pearson_corr:0.9993271231651306 - rollout_corr/rollout_is_mean:1.0001097917556763 - rollout_corr/rollout_is_ratio_fraction_high:1.1016612916137092e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.652492028370034e-05 - rollout_corr/rollout_is_max:2.2787067890167236 - rollout_corr/rollout_is_min:0.41993895173072815 - rollout_corr/rollout_is_std:0.030713239684700966 - rollout_corr/rollout_is_eff_sample_size:0.9990577048586795 - rollout_corr/rollout_is_seq_mean:1.000206708908081 - rollout_corr/rollout_is_seq_std:0.002247885102406144 - rollout_corr/rollout_is_seq_max:1.0164759159088135 - rollout_corr/rollout_is_seq_min:0.9938144683837891 - rollout_corr/rollout_is_seq_max_deviation:0.016475915908813477 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3208666159771383) - rollout_corr/training_log_ppl:np.float64(0.2606007744325325) - rollout_corr/kl:np.float64(0.0003702222969650393) - rollout_corr/k3_kl:np.float64(0.000720645587279023) - rollout_corr/rollout_ppl:np.float64(1.3202885040082037) - rollout_corr/rollout_log_ppl:np.float64(0.26023055279802065) - rollout_corr/log_ppl_diff:np.float64(0.0003702216345118359) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016211537149501964) - rollout_corr/log_ppl_diff_max:np.float64(0.01012013852596283) - rollout_corr/log_ppl_diff_min:np.float64(-0.012066826224327087) - rollout_corr/ppl_ratio:np.float64(1.0003729804884642) - rollout_corr/chi2_token:np.float64(0.0021484598983079195) - rollout_corr/chi2_seq:np.float64(2.6817988054826856) - actor/pg_loss:np.float64(-0.00016311672516167164) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024571236758674786) - actor/ppo_kl:np.float64(-5.575577448624358e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21021737158298492) - perf/mfu/actor:np.float64(0.08564326289574913) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.05369186401367) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:3 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008819486945867538 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008819486945867538 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:709.15625 - response_length/max:4610.0 - response_length/min:62.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:709.15625 - response_length_non_aborted/max:4610.0 - response_length_non_aborted/min:62.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.65625 - prompt_length/max:353.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.643472731113434e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.46839356422424316) - timing_s/agent_loop/generate_sequences/max:np.float64(26.743163952603936) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.783247412800847) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(26.743163952603936) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:4610 - timing_s/gen:33.11821131967008 - timing_s/reward:0.07101857103407383 - timing_s/old_log_prob:2.5942629985511303 - timing_s/adv:0.12228076718747616 - timing_s/update_actor:10.489771209657192 - timing_s/step:46.487874053418636 - timing_s/stop_profile:0.00010906346142292023 - timing_per_token_ms/update_actor:0.042646893944160186 - timing_per_token_ms/gen:0.18242525954958622 - timing_per_token_ms/adv:0.0004971409581225044 - perf/total_num_tokens:245968 - perf/time_per_step:46.487874053418636 - perf/throughput:661.3767703093963 (TaskRunner pid=2049544) Training Progress: 74%|███████▍ | 74/100 [57:39<21:41, 50.06s/it] (TaskRunner pid=2049544) step:75 - global_seqlen/min:19402 - global_seqlen/max:28818 - global_seqlen/minmax_diff:9416 - global_seqlen/balanced_min:23785 - global_seqlen/balanced_max:23996 - global_seqlen/mean:23814.375 - actor/entropy:0.25116294622421265 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135130643844604 - training/rollout_probs_diff_mean:0.0031402192544192076 - training/rollout_probs_diff_std:0.009609016589820385 - training/rollout_actor_probs_pearson_corr:0.9991532564163208 - rollout_corr/rollout_is_mean:1.0000470876693726 - rollout_corr/rollout_is_ratio_fraction_high:8.058602361415979e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.4175806174753234e-05 - rollout_corr/rollout_is_max:2.1670117378234863 - rollout_corr/rollout_is_min:0.4087337255477905 - rollout_corr/rollout_is_std:0.03037172183394432 - rollout_corr/rollout_is_eff_sample_size:0.9990784086315643 - rollout_corr/rollout_is_seq_mean:1.0000663995742798 - rollout_corr/rollout_is_seq_std:0.0019097818294540048 - rollout_corr/rollout_is_seq_max:1.0100950002670288 - rollout_corr/rollout_is_seq_min:0.9921054244041443 - rollout_corr/rollout_is_seq_max_deviation:0.010095000267028809 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2536835842765868) - rollout_corr/training_log_ppl:np.float64(0.20439182232803432) - rollout_corr/kl:np.float64(0.00044191136634275097) - rollout_corr/k3_kl:np.float64(0.0004956063722261206) - rollout_corr/rollout_ppl:np.float64(1.2531235865317285) - rollout_corr/rollout_log_ppl:np.float64(0.20394990891509224) - rollout_corr/log_ppl_diff:np.float64(0.0004419134129420854) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013838859085808508) - rollout_corr/log_ppl_diff_max:np.float64(0.009681947529315948) - rollout_corr/log_ppl_diff_min:np.float64(-0.005931958556175232) - rollout_corr/ppl_ratio:np.float64(1.0004437156021595) - rollout_corr/chi2_token:np.float64(0.0011102964635938406) - rollout_corr/chi2_seq:np.float64(0.39504921343177557) - actor/pg_loss:np.float64(1.7428305000066757e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018432266847412393) - actor/ppo_kl:np.float64(3.7105337884923983e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16733924113214016) - perf/mfu/actor:np.float64(0.06918613903772251) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.97526931762695) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:3 - critic/score/mean:0.859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0008511898340657353 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0008511898340657353 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:484.73046875 - response_length/max:2220.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:484.73046875 - response_length_non_aborted/max:2220.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.46875 - prompt_length/max:383.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016090832650661469 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2748903073370457) - timing_s/agent_loop/generate_sequences/max:np.float64(13.663885425776243) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.775116956734564) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.663885425776243) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:315 - timing_s/agent_loop/slowest/response_length:2220 - timing_s/gen:19.10645161010325 - timing_s/reward:0.06184021010994911 - timing_s/old_log_prob:2.374053070321679 - timing_s/adv:0.11829666048288345 - timing_s/update_actor:9.632479302585125 - timing_s/step:31.360355062410235 - timing_s/stop_profile:3.0426308512687683e-05 - timing_per_token_ms/update_actor:0.05056021469482783 - timing_per_token_ms/gen:0.15397129211710156 - timing_per_token_ms/adv:0.0006209309528534942 - perf/total_num_tokens:190515 - perf/time_per_step:31.360355062410235 - perf/throughput:759.3783601176395 (TaskRunner pid=2049544) Training Progress: 75%|███████▌ | 75/100 [58:10<18:31, 44.46s/it] (TaskRunner pid=2049544) step:76 - global_seqlen/min:25932 - global_seqlen/max:42011 - global_seqlen/minmax_diff:16079 - global_seqlen/balanced_min:31987 - global_seqlen/balanced_max:32015 - global_seqlen/mean:32005.0 - actor/entropy:0.35323843359947205 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8143891096115112 - training/rollout_probs_diff_mean:0.003405920695513487 - training/rollout_probs_diff_std:0.009273294359445572 - training/rollout_actor_probs_pearson_corr:0.9994474053382874 - rollout_corr/rollout_is_mean:1.0001150369644165 - rollout_corr/rollout_is_ratio_fraction_high:1.081595564755844e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.081595564755844e-05 - rollout_corr/rollout_is_max:3.5232980251312256 - rollout_corr/rollout_is_min:0.16788354516029358 - rollout_corr/rollout_is_std:0.030408985912799835 - rollout_corr/rollout_is_eff_sample_size:0.9990763858113328 - rollout_corr/rollout_is_seq_mean:1.0001401901245117 - rollout_corr/rollout_is_seq_std:0.0016239474061876535 - rollout_corr/rollout_is_seq_max:1.010636329650879 - rollout_corr/rollout_is_seq_min:0.9931896328926086 - rollout_corr/rollout_is_seq_max_deviation:0.010636329650878906 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3853302095085382) - rollout_corr/training_log_ppl:np.float64(0.3024687277356861) - rollout_corr/kl:np.float64(0.0005689695157187202) - rollout_corr/k3_kl:np.float64(0.0005752426737899441) - rollout_corr/rollout_ppl:np.float64(1.38448647223413) - rollout_corr/rollout_log_ppl:np.float64(0.30189975767279975) - rollout_corr/log_ppl_diff:np.float64(0.0005689700628863648) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012543359625851735) - rollout_corr/log_ppl_diff_max:np.float64(0.007834061980247498) - rollout_corr/log_ppl_diff_min:np.float64(-0.006428062915802002) - rollout_corr/ppl_ratio:np.float64(1.0005704772192985) - rollout_corr/chi2_token:np.float64(0.0012044101022183895) - rollout_corr/chi2_seq:np.float64(1.3872064049355686) - actor/pg_loss:np.float64(5.75545709580183e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002793574865336268) - actor/ppo_kl:np.float64(0.00019198946114684645) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3208663538098335) - perf/mfu/actor:np.float64(0.09134199443488283) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.03634643554688) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:3 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008177538402378559 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008177538402378559 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:722.3125 - response_length/max:2396.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:722.3125 - response_length_non_aborted/max:2396.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.84375 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.119371831417084e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.99646220728755) - timing_s/agent_loop/generate_sequences/max:np.float64(16.732603766024113) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.14587117084011) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.732603766024113) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:306 - timing_s/agent_loop/slowest/response_length:2371 - timing_s/gen:22.66927962191403 - timing_s/reward:0.07421145401895046 - timing_s/old_log_prob:2.5292600374668837 - timing_s/adv:0.1157302763313055 - timing_s/update_actor:10.305034121498466 - timing_s/step:35.780901934951544 - timing_s/stop_profile:0.00012325309216976166 - timing_per_token_ms/update_actor:0.040247750826036814 - timing_per_token_ms/gen:0.12259496204634653 - timing_per_token_ms/adv:0.00045200076679934975 - perf/total_num_tokens:256040 - perf/time_per_step:35.780901934951544 - perf/throughput:894.471583141867 (TaskRunner pid=2049544) Training Progress: 76%|███████▌ | 76/100 [58:46<16:44, 41.87s/it] (TaskRunner pid=2049544) step:77 - global_seqlen/min:16244 - global_seqlen/max:41465 - global_seqlen/minmax_diff:25221 - global_seqlen/balanced_min:29420 - global_seqlen/balanced_max:29517 - global_seqlen/mean:29492.5 - actor/entropy:0.3045775294303894 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2915552258491516 - training/rollout_probs_diff_mean:0.0032992891501635313 - training/rollout_probs_diff_std:0.009267176501452923 - training/rollout_actor_probs_pearson_corr:0.9993472695350647 - rollout_corr/rollout_is_mean:1.0000176429748535 - rollout_corr/rollout_is_ratio_fraction_high:5.8924742916133255e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.140592098236084 - rollout_corr/rollout_is_min:0.5226767659187317 - rollout_corr/rollout_is_std:0.029582349583506584 - rollout_corr/rollout_is_eff_sample_size:0.9991255307618554 - rollout_corr/rollout_is_seq_mean:1.0000417232513428 - rollout_corr/rollout_is_seq_std:0.0017432396998628974 - rollout_corr/rollout_is_seq_max:1.008344054222107 - rollout_corr/rollout_is_seq_min:0.9936603307723999 - rollout_corr/rollout_is_seq_max_deviation:0.008344054222106934 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2967352382838726) - rollout_corr/training_log_ppl:np.float64(0.24739519167633262) - rollout_corr/kl:np.float64(0.0005334063558146696) - rollout_corr/k3_kl:np.float64(0.0005630067024355867) - rollout_corr/rollout_ppl:np.float64(1.2960236202925444) - rollout_corr/rollout_log_ppl:np.float64(0.24686178589763585) - rollout_corr/log_ppl_diff:np.float64(0.0005334057786967605) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014061912952456623) - rollout_corr/log_ppl_diff_max:np.float64(0.01065683364868164) - rollout_corr/log_ppl_diff_min:np.float64(-0.0076304227113723755) - rollout_corr/ppl_ratio:np.float64(1.0005355728790164) - rollout_corr/chi2_token:np.float64(0.0011709912214428186) - rollout_corr/chi2_seq:np.float64(0.44916587066836655) - actor/pg_loss:np.float64(2.6725465431809425e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018346831632243266) - actor/ppo_kl:np.float64(9.181251817746983e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16028263419866562) - perf/mfu/actor:np.float64(0.08482457572663438) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.09542083740234) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:3 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006819802336394787 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006819802336394787 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:662.921875 - response_length/max:2972.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:662.921875 - response_length_non_aborted/max:2972.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.71875 - prompt_length/max:355.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013144128024578094 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8779960200190544) - timing_s/agent_loop/generate_sequences/max:np.float64(18.480581056326628) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.09187524789013) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.480581056326628) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:315 - timing_s/agent_loop/slowest/response_length:2972 - timing_s/gen:24.529541220515966 - timing_s/reward:0.07079425640404224 - timing_s/old_log_prob:2.487271063029766 - timing_s/adv:0.11753782816231251 - timing_s/update_actor:10.21386044844985 - timing_s/step:37.50457299686968 - timing_s/stop_profile:2.83215194940567e-05 - timing_per_token_ms/update_actor:0.043290075648257396 - timing_per_token_ms/gen:0.1445396871126639 - timing_per_token_ms/adv:0.0004981682977126071 - perf/total_num_tokens:235940 - perf/time_per_step:37.50457299686968 - perf/throughput:786.3707714379681 (TaskRunner pid=2049544) Training Progress: 77%|███████▋ | 77/100 [59:24<15:33, 40.57s/it] (TaskRunner pid=2049544) step:78 - global_seqlen/min:16539 - global_seqlen/max:34942 - global_seqlen/minmax_diff:18403 - global_seqlen/balanced_min:26643 - global_seqlen/balanced_max:26991 - global_seqlen/mean:26772.625 - actor/entropy:0.324239581823349 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30065640807151794 - training/rollout_probs_diff_mean:0.003476528450846672 - training/rollout_probs_diff_std:0.009609517641365528 - training/rollout_actor_probs_pearson_corr:0.9993136525154114 - rollout_corr/rollout_is_mean:0.9999270439147949 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.365252956020413e-05 - rollout_corr/rollout_is_max:1.927759051322937 - rollout_corr/rollout_is_min:0.17178574204444885 - rollout_corr/rollout_is_std:0.031049082055687904 - rollout_corr/rollout_is_eff_sample_size:0.9990367639843101 - rollout_corr/rollout_is_seq_mean:0.9997465014457703 - rollout_corr/rollout_is_seq_std:0.0017120024422183633 - rollout_corr/rollout_is_seq_max:1.0067331790924072 - rollout_corr/rollout_is_seq_min:0.9950670599937439 - rollout_corr/rollout_is_seq_max_deviation:0.0067331790924072266 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3026048908941448) - rollout_corr/training_log_ppl:np.float64(0.24134253649390303) - rollout_corr/kl:np.float64(0.0007427987794557112) - rollout_corr/k3_kl:np.float64(0.0005388006466660045) - rollout_corr/rollout_ppl:np.float64(1.301613979972899) - rollout_corr/rollout_log_ppl:np.float64(0.24059973818657454) - rollout_corr/log_ppl_diff:np.float64(0.000742798307328485) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014510368491755798) - rollout_corr/log_ppl_diff_max:np.float64(0.0068367719650268555) - rollout_corr/log_ppl_diff_min:np.float64(-0.005617767572402954) - rollout_corr/ppl_ratio:np.float64(1.0007447020616382) - rollout_corr/chi2_token:np.float64(0.0006613847799599171) - rollout_corr/chi2_seq:np.float64(0.6144026103429496) - actor/pg_loss:np.float64(5.164649337530136e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011791538452143868) - actor/ppo_kl:np.float64(-3.060101226282086e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16195594891905785) - perf/mfu/actor:np.float64(0.07693655141257658) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.07821655273438) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:3 - critic/score/mean:0.78125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0012876042164862156 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0012876042164862156 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:572.23828125 - response_length/max:3226.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:572.23828125 - response_length_non_aborted/max:3226.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.40625 - prompt_length/max:355.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.397596836090088e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2444690354168415) - timing_s/agent_loop/generate_sequences/max:np.float64(19.064789429306984) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.27938631172583) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.064789429306984) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:251 - timing_s/agent_loop/slowest/response_length:3226 - timing_s/gen:24.83233668655157 - timing_s/reward:0.06715011782944202 - timing_s/old_log_prob:2.454452695325017 - timing_s/adv:0.12591312639415264 - timing_s/update_actor:10.206827508285642 - timing_s/step:37.77338271215558 - timing_s/stop_profile:0.00012371130287647247 - timing_per_token_ms/update_actor:0.04765514918823631 - timing_per_token_ms/gen:0.1695121042408277 - timing_per_token_ms/adv:0.0005878818681122632 - perf/total_num_tokens:214181 - perf/time_per_step:37.77338271215558 - perf/throughput:708.7695905875142 (TaskRunner pid=2049544) Training Progress: 78%|███████▊ | 78/100 [1:00:02<14:34, 39.74s/it] (TaskRunner pid=2049544) step:79 - global_seqlen/min:20249 - global_seqlen/max:46165 - global_seqlen/minmax_diff:25916 - global_seqlen/balanced_min:30262 - global_seqlen/balanced_max:30358 - global_seqlen/mean:30336.375 - actor/entropy:0.27736997604370117 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4035380482673645 - training/rollout_probs_diff_mean:0.0030377230141311884 - training/rollout_probs_diff_std:0.009076771326363087 - training/rollout_actor_probs_pearson_corr:0.9993468523025513 - rollout_corr/rollout_is_mean:0.9998963475227356 - rollout_corr/rollout_is_ratio_fraction_high:5.790153409179766e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.211137977312319e-05 - rollout_corr/rollout_is_max:4.029329299926758 - rollout_corr/rollout_is_min:0.07581690698862076 - rollout_corr/rollout_is_std:0.02898186817765236 - rollout_corr/rollout_is_eff_sample_size:0.9991605182489611 - rollout_corr/rollout_is_seq_mean:0.9999415874481201 - rollout_corr/rollout_is_seq_std:0.0017486511496827006 - rollout_corr/rollout_is_seq_max:1.0085891485214233 - rollout_corr/rollout_is_seq_min:0.9941185712814331 - rollout_corr/rollout_is_seq_max_deviation:0.00858914852142334 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2839201726019382) - rollout_corr/training_log_ppl:np.float64(0.2279481466684956) - rollout_corr/kl:np.float64(0.0005260294094213691) - rollout_corr/k3_kl:np.float64(0.0005059725925065095) - rollout_corr/rollout_ppl:np.float64(1.2831925540231168) - rollout_corr/rollout_log_ppl:np.float64(0.2274221143743489) - rollout_corr/log_ppl_diff:np.float64(0.0005260322941467166) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012799371324945241) - rollout_corr/log_ppl_diff_max:np.float64(0.008957505226135254) - rollout_corr/log_ppl_diff_min:np.float64(-0.0052579790353775024) - rollout_corr/ppl_ratio:np.float64(1.0005276040174067) - rollout_corr/chi2_token:np.float64(0.0009765762370079756) - rollout_corr/chi2_seq:np.float64(0.7638932466506958) - actor/pg_loss:np.float64(0.00014414661563932896) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017305412563928257) - actor/ppo_kl:np.float64(-2.993323226707645e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20632774755358696) - perf/mfu/actor:np.float64(0.08763345813710535) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.01940155029297) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:3 - critic/score/mean:0.8046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008221293799579144 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008221293799579144 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:674.63671875 - response_length/max:2725.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:674.63671875 - response_length_non_aborted/max:2725.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.375 - prompt_length/max:365.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010620616376399994 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1851273197680712) - timing_s/agent_loop/generate_sequences/max:np.float64(17.865634333342314) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.507378077250905) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.865634333342314) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:308 - timing_s/agent_loop/slowest/response_length:2725 - timing_s/gen:23.788950365036726 - timing_s/reward:0.07290799915790558 - timing_s/old_log_prob:2.4474152121692896 - timing_s/adv:0.1101104523986578 - timing_s/update_actor:10.210453910753131 - timing_s/step:36.71885127387941 - timing_s/stop_profile:0.00012296251952648163 - timing_per_token_ms/update_actor:0.04207182759456729 - timing_per_token_ms/gen:0.13774166863553142 - timing_per_token_ms/adv:0.0004537063689986765 - perf/total_num_tokens:242691 - perf/time_per_step:36.71885127387941 - perf/throughput:826.1798489752948 (TaskRunner pid=2049544) Training Progress: 79%|███████▉ | 79/100 [1:00:38<13:35, 38.85s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) The electric field $ E $ due to a point charge $ q $ at a distance $ r $ is given by: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) - $ E = 2.0 \, \text{N/C} $ (TaskRunner pid=2049544) - $ r = 50 \, \text{cm} = 0.50 \, \text{m} $ (TaskRunner pid=2049544) - $ k = 8.99 \times 10^9 \, \text{N m}^2/\text{C}^2 $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We need to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the known values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.5}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \text{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \text{C} = 55.6 \, \text{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70/actor (TaskRunner pid=2049544) step:80 - global_seqlen/min:18167 - global_seqlen/max:36780 - global_seqlen/minmax_diff:18613 - global_seqlen/balanced_min:28127 - global_seqlen/balanced_max:28499 - global_seqlen/mean:28202.25 - actor/entropy:0.23095937073230743 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44294559955596924 - training/rollout_probs_diff_mean:0.002918842015787959 - training/rollout_probs_diff_std:0.0094361687079072 - training/rollout_actor_probs_pearson_corr:0.9991447329521179 - rollout_corr/rollout_is_mean:0.9999209046363831 - rollout_corr/rollout_is_ratio_fraction_high:1.9175455236108974e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.9175455236108974e-05 - rollout_corr/rollout_is_max:2.320603370666504 - rollout_corr/rollout_is_min:0.33252665400505066 - rollout_corr/rollout_is_std:0.028494464233517647 - rollout_corr/rollout_is_eff_sample_size:0.999188605215701 - rollout_corr/rollout_is_seq_mean:0.9999192953109741 - rollout_corr/rollout_is_seq_std:0.001614381093531847 - rollout_corr/rollout_is_seq_max:1.0071684122085571 - rollout_corr/rollout_is_seq_min:0.9931057691574097 - rollout_corr/rollout_is_seq_max_deviation:0.007168412208557129 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2288768324069679) - rollout_corr/training_log_ppl:np.float64(0.19351409275259357) - rollout_corr/kl:np.float64(0.0005485934744235976) - rollout_corr/k3_kl:np.float64(0.0005014402207166313) - rollout_corr/rollout_ppl:np.float64(1.2281699823215604) - rollout_corr/rollout_log_ppl:np.float64(0.1929654997366015) - rollout_corr/log_ppl_diff:np.float64(0.0005485930159920827) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012800928816432133) - rollout_corr/log_ppl_diff_max:np.float64(0.00843721628189087) - rollout_corr/log_ppl_diff_min:np.float64(-0.006566546857357025) - rollout_corr/ppl_ratio:np.float64(1.0005502616986632) - rollout_corr/chi2_token:np.float64(0.0008651481475681067) - rollout_corr/chi2_seq:np.float64(0.5386446032207459) - actor/pg_loss:np.float64(-2.616771962493658e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001737377756398928) - actor/ppo_kl:np.float64(2.498516215898583e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1959109902381897) - perf/mfu/actor:np.float64(0.08114905418503308) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.12442779541016) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6390625) - val-aux/sciknoweval/reward/std@16:np.float64(0.18886344890574386) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7177) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15087202226244528) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.56135) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.17040711495924588) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6400125) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1892037078209971) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7728999999999999) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.10375600250036085) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.48866250000000006) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1332606477643524) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6633375) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.15024050549177956) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.812) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06993051354730821) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4333875) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08524525960422448) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6773) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10302193213109104) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8416625) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04398837601915231) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.40309999999999996) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04059727090755087) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.683725) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.05843863672565752) - val-aux/sciknoweval/score/mean@16:np.float64(0.6390625) - val-aux/sciknoweval/score/std@16:np.float64(0.18886344890574386) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7177) - val-aux/sciknoweval/score/best@2/std:np.float64(0.15087202226244528) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.56135) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.17040711495924588) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6400125) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1892037078209971) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7728999999999999) - val-aux/sciknoweval/score/best@4/std:np.float64(0.10375600250036085) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.48866250000000006) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1332606477643524) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6633375) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.15024050549177956) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.812) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06993051354730821) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4333875) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08524525960422448) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6773) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10302193213109104) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8416625) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04398837601915231) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.40309999999999996) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04059727090755087) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.683725) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.05843863672565752) - val-core/sciknoweval/acc/mean@16:np.float64(0.6390625) - val-aux/sciknoweval/acc/std@16:np.float64(0.18886344890574386) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7177) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.15087202226244528) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.56135) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.17040711495924588) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6400125) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1892037078209971) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7728999999999999) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.10375600250036085) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.48866250000000006) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1332606477643524) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6633375) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.15024050549177956) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.812) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06993051354730821) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4333875) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08524525960422448) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6773) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10302193213109104) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8416625) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04398837601915231) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.40309999999999996) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04059727090755087) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.683725) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.05843863672565752) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.975) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.06354165489638078) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.996175) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.023966893234937735) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9539) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.08044933488834552) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9746749999999998) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.06396939198866787) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9998125000000002) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.003145810047315943) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9212250000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.09233866134852486) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9907999999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.03956667018273212) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8777874999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.08983250151225204) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9978250000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.018387073504573324) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8370125000000002) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0672232524014538) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998249999999999) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.004527947017243354) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:3 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006327900337055326 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006327900337055326 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:611.1328125 - response_length/max:2845.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:611.1328125 - response_length_non_aborted/max:2845.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.1875 - prompt_length/max:363.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000109061598777771 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1131607126444578) - timing_s/agent_loop/generate_sequences/max:np.float64(17.91330724582076) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.142933668168553) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.91330724582076) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:2845 - timing_s/gen:23.700068602338433 - timing_s/reward:0.06908616237342358 - timing_s/old_log_prob:2.398565139621496 - timing_s/adv:0.14442053809762 - timing_s/update_actor:10.117256434634328 - timing_s/step:36.51560699753463 - timing_s/testing:115.6058545038104 - timing_s/save_checkpoint:6.825648196041584 - timing_s/stop_profile:0.00023227371275424957 - timing_per_token_ms/update_actor:0.04484241698195325 - timing_per_token_ms/gen:0.15148653628851666 - timing_per_token_ms/adv:0.0006401108869754187 - perf/total_num_tokens:225618 - perf/time_per_step:36.51560699753463 - perf/throughput:772.3341419986278 (TaskRunner pid=2049544) Training Progress: 80%|████████ | 80/100 [1:03:17<24:57, 74.90s/it] (TaskRunner pid=2049544) step:81 - global_seqlen/min:22002 - global_seqlen/max:45026 - global_seqlen/minmax_diff:23024 - global_seqlen/balanced_min:31986 - global_seqlen/balanced_max:32078 - global_seqlen/mean:32047.5 - actor/entropy:0.30386513471603394 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135500192642212 - training/rollout_probs_diff_mean:0.003210825379937887 - training/rollout_probs_diff_std:0.00900829304009676 - training/rollout_actor_probs_pearson_corr:0.9993411898612976 - rollout_corr/rollout_is_mean:1.0000028610229492 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.263490493234713e-06 - rollout_corr/rollout_is_max:1.9500842094421387 - rollout_corr/rollout_is_min:0.46151214838027954 - rollout_corr/rollout_is_std:0.029131613671779633 - rollout_corr/rollout_is_eff_sample_size:0.9991520686654056 - rollout_corr/rollout_is_seq_mean:0.9999109506607056 - rollout_corr/rollout_is_seq_std:0.0019725700840353966 - rollout_corr/rollout_is_seq_max:1.0081653594970703 - rollout_corr/rollout_is_seq_min:0.9903681874275208 - rollout_corr/rollout_is_seq_max_deviation:0.009631812572479248 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3244363577105105) - rollout_corr/training_log_ppl:np.float64(0.2665938182544778) - rollout_corr/kl:np.float64(0.0010818528244627146) - rollout_corr/k3_kl:np.float64(0.0008180101519030814) - rollout_corr/rollout_ppl:np.float64(1.3230207040905952) - rollout_corr/rollout_log_ppl:np.float64(0.26551196482614614) - rollout_corr/log_ppl_diff:np.float64(0.0010818534283316694) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018223601437057368) - rollout_corr/log_ppl_diff_max:np.float64(0.01393115520477295) - rollout_corr/log_ppl_diff_min:np.float64(-0.010075211524963379) - rollout_corr/ppl_ratio:np.float64(1.0010864462237805) - rollout_corr/chi2_token:np.float64(0.0015508977230638266) - rollout_corr/chi2_seq:np.float64(1.0036336749326438) - actor/pg_loss:np.float64(7.214734796434641e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002760089199682625) - actor/ppo_kl:np.float64(0.0004848889264348344) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21540894359350204) - perf/mfu/actor:np.float64(0.0899665554810477) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.27264404296875) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:3 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004168026614934206 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004168026614934206 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:742.140625 - response_length/max:3366.0 - response_length/min:63.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:742.140625 - response_length_non_aborted/max:3366.0 - response_length_non_aborted/min:63.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.34375 - prompt_length/max:460.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.953603148460388e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6883454918861389) - timing_s/agent_loop/generate_sequences/max:np.float64(21.4869614392519) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.608749296545284) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.4869614392519) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:211 - timing_s/agent_loop/slowest/response_length:3366 - timing_s/gen:27.376880433410406 - timing_s/reward:0.07466047257184982 - timing_s/old_log_prob:2.649170648306608 - timing_s/adv:0.12651739083230495 - timing_s/update_actor:10.34231029637158 - timing_s/step:40.673489751294255 - timing_s/stop_profile:0.00011551380157470703 - timing_per_token_ms/update_actor:0.04033977024873851 - timing_per_token_ms/gen:0.1440979453092322 - timing_per_token_ms/adv:0.000493476054420411 - perf/total_num_tokens:256380 - perf/time_per_step:40.673489751294255 - perf/throughput:787.921080683278 (TaskRunner pid=2049544) Training Progress: 81%|████████ | 81/100 [1:03:58<20:28, 64.65s/it] (TaskRunner pid=2049544) step:82 - global_seqlen/min:25371 - global_seqlen/max:44114 - global_seqlen/minmax_diff:18743 - global_seqlen/balanced_min:33520 - global_seqlen/balanced_max:33584 - global_seqlen/mean:33563.875 - actor/entropy:0.2920665442943573 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.913862407207489 - training/rollout_probs_diff_mean:0.0031529199331998825 - training/rollout_probs_diff_std:0.009444354102015495 - training/rollout_actor_probs_pearson_corr:0.9993178248405457 - rollout_corr/rollout_is_mean:0.9999819993972778 - rollout_corr/rollout_is_ratio_fraction_high:2.0209470676491037e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.547131175058894e-05 - rollout_corr/rollout_is_max:2.713327169418335 - rollout_corr/rollout_is_min:0.049278318881988525 - rollout_corr/rollout_is_std:0.02968292124569416 - rollout_corr/rollout_is_eff_sample_size:0.9991195807527394 - rollout_corr/rollout_is_seq_mean:0.9999279379844666 - rollout_corr/rollout_is_seq_std:0.0012865856988355517 - rollout_corr/rollout_is_seq_max:1.004623532295227 - rollout_corr/rollout_is_seq_min:0.9942681193351746 - rollout_corr/rollout_is_seq_max_deviation:0.0057318806648254395 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3072700020857155) - rollout_corr/training_log_ppl:np.float64(0.24861305610102136) - rollout_corr/kl:np.float64(0.0007281105056762271) - rollout_corr/k3_kl:np.float64(0.0005667303261134293) - rollout_corr/rollout_ppl:np.float64(1.306269062217325) - rollout_corr/rollout_log_ppl:np.float64(0.2478849448962137) - rollout_corr/log_ppl_diff:np.float64(0.0007281112048076466) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011902832047780976) - rollout_corr/log_ppl_diff_max:np.float64(0.012343257665634155) - rollout_corr/log_ppl_diff_min:np.float64(-0.00298440083861351) - rollout_corr/ppl_ratio:np.float64(1.0007296884432435) - rollout_corr/chi2_token:np.float64(0.0007541878148913383) - rollout_corr/chi2_seq:np.float64(0.7926566984970123) - actor/pg_loss:np.float64(0.0001337927533313632) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021164456336464355) - actor/ppo_kl:np.float64(0.00017838788005519746) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1409779703244567) - perf/mfu/actor:np.float64(0.09575702121449964) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.184974193573) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:3 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00978454202413559 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00978454202413559 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:773.15234375 - response_length/max:2703.0 - response_length/min:191.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:773.15234375 - response_length_non_aborted/max:2703.0 - response_length_non_aborted/min:191.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:275.71875 - prompt_length/max:382.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.613321602344513e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2882147133350372) - timing_s/agent_loop/generate_sequences/max:np.float64(18.499180192127824) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.74611435400584) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.499180192127824) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:240 - timing_s/agent_loop/slowest/response_length:2703 - timing_s/gen:24.018707640469074 - timing_s/reward:0.07196270488202572 - timing_s/old_log_prob:2.514760909602046 - timing_s/adv:0.12197888456285 - timing_s/update_actor:10.373511025682092 - timing_s/step:37.19780015014112 - timing_s/stop_profile:3.86945903301239e-05 - timing_per_token_ms/update_actor:0.03863346762584063 - timing_per_token_ms/gen:0.12135134489215253 - timing_per_token_ms/adv:0.0004542789105952829 - perf/total_num_tokens:268511 - perf/time_per_step:37.19780015014112 - perf/throughput:902.3080629641123 (TaskRunner pid=2049544) Training Progress: 82%|████████▏ | 82/100 [1:04:35<16:55, 56.42s/it] (TaskRunner pid=2049544) step:83 - global_seqlen/min:19276 - global_seqlen/max:41937 - global_seqlen/minmax_diff:22661 - global_seqlen/balanced_min:29583 - global_seqlen/balanced_max:29645 - global_seqlen/mean:29615.125 - actor/entropy:0.34415289759635925 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5622131824493408 - training/rollout_probs_diff_mean:0.003517036559060216 - training/rollout_probs_diff_std:0.009729686193168163 - training/rollout_actor_probs_pearson_corr:0.9993311762809753 - rollout_corr/rollout_is_mean:0.9999920129776001 - rollout_corr/rollout_is_ratio_fraction_high:1.2048701137246098e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.0121751478873193e-05 - rollout_corr/rollout_is_max:8.23111629486084 - rollout_corr/rollout_is_min:0.2471064180135727 - rollout_corr/rollout_is_std:0.03159145638346672 - rollout_corr/rollout_is_eff_sample_size:0.9990024989924954 - rollout_corr/rollout_is_seq_mean:0.9999005794525146 - rollout_corr/rollout_is_seq_std:0.0016829889500513673 - rollout_corr/rollout_is_seq_max:1.004620909690857 - rollout_corr/rollout_is_seq_min:0.9928184747695923 - rollout_corr/rollout_is_seq_max_deviation:0.007181525230407715 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.372836111113429) - rollout_corr/training_log_ppl:np.float64(0.2916473155346466) - rollout_corr/kl:np.float64(0.0007178986236571205) - rollout_corr/k3_kl:np.float64(0.0006351841151968074) - rollout_corr/rollout_ppl:np.float64(1.3718020445667207) - rollout_corr/rollout_log_ppl:np.float64(0.2909294152777875) - rollout_corr/log_ppl_diff:np.float64(0.0007179002568591386) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014387322589755058) - rollout_corr/log_ppl_diff_max:np.float64(0.009183712303638458) - rollout_corr/log_ppl_diff_min:np.float64(-0.0071634650230407715) - rollout_corr/ppl_ratio:np.float64(1.0007200299296528) - rollout_corr/chi2_token:np.float64(0.0011453672777861357) - rollout_corr/chi2_seq:np.float64(0.5700722692999989) - actor/pg_loss:np.float64(0.00010708440095186234) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003794109481987107) - actor/ppo_kl:np.float64(5.79457945377726e-05) - actor/pg_clipfrac_lower:np.float64(4.787071247847052e-06) - actor/grad_norm:np.float64(0.27837222814559937) - perf/mfu/actor:np.float64(0.08514566987298042) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.15073013305664) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:3 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003308121347799897 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003308121347799897 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:648.41015625 - response_length/max:2853.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:648.41015625 - response_length_non_aborted/max:2853.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.0625 - prompt_length/max:375.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.170570850372314e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8073275871574879) - timing_s/agent_loop/generate_sequences/max:np.float64(18.502459589391947) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.3145277952207834) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.502459589391947) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:2853 - timing_s/gen:24.35667880065739 - timing_s/reward:0.10609085857868195 - timing_s/old_log_prob:2.4029372967779636 - timing_s/adv:0.11332383565604687 - timing_s/update_actor:10.112380102276802 - timing_s/step:37.18459020368755 - timing_s/stop_profile:3.2028183341026306e-05 - timing_per_token_ms/update_actor:0.042682497973066134 - timing_per_token_ms/gen:0.14673316827009206 - timing_per_token_ms/adv:0.0004783190838129455 - perf/total_num_tokens:236921 - perf/time_per_step:37.18459020368755 - perf/throughput:796.4354276267675 (TaskRunner pid=2049544) Training Progress: 83%|████████▎ | 83/100 [1:05:12<14:21, 50.66s/it] (TaskRunner pid=2049544) step:84 - global_seqlen/min:25752 - global_seqlen/max:42945 - global_seqlen/minmax_diff:17193 - global_seqlen/balanced_min:33932 - global_seqlen/balanced_max:34032 - global_seqlen/mean:33969.0 - actor/entropy:0.3499566912651062 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6138913631439209 - training/rollout_probs_diff_mean:0.0033732259180396795 - training/rollout_probs_diff_std:0.009146650321781635 - training/rollout_actor_probs_pearson_corr:0.9994776844978333 - rollout_corr/rollout_is_mean:0.9999618530273438 - rollout_corr/rollout_is_ratio_fraction_high:4.901384272670839e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.4506920453859493e-05 - rollout_corr/rollout_is_max:2.404789447784424 - rollout_corr/rollout_is_min:0.2902381718158722 - rollout_corr/rollout_is_std:0.03017878346145153 - rollout_corr/rollout_is_eff_sample_size:0.9990900697549667 - rollout_corr/rollout_is_seq_mean:1.0001087188720703 - rollout_corr/rollout_is_seq_std:0.001655240310356021 - rollout_corr/rollout_is_seq_max:1.0082566738128662 - rollout_corr/rollout_is_seq_min:0.9946045279502869 - rollout_corr/rollout_is_seq_max_deviation:0.008256673812866211 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3845429411157966) - rollout_corr/training_log_ppl:np.float64(0.29742183088092133) - rollout_corr/kl:np.float64(0.00039445633141355074) - rollout_corr/k3_kl:np.float64(0.0005463047522766828) - rollout_corr/rollout_ppl:np.float64(1.3839370189234614) - rollout_corr/rollout_log_ppl:np.float64(0.29702737282786984) - rollout_corr/log_ppl_diff:np.float64(0.00039445805305149406) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012936670245835558) - rollout_corr/log_ppl_diff_max:np.float64(0.00665166974067688) - rollout_corr/log_ppl_diff_min:np.float64(-0.005310043692588806) - rollout_corr/ppl_ratio:np.float64(1.000396022805944) - rollout_corr/chi2_token:np.float64(0.0013724297750741243) - rollout_corr/chi2_seq:np.float64(1.7039004366379231) - actor/pg_loss:np.float64(0.00012169475667178631) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021371153513882746) - actor/ppo_kl:np.float64(1.5052716766383867e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2066413126885891) - perf/mfu/actor:np.float64(0.09535977782956703) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.16190719604492) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:3 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017204470932483673 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017204470932483673 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:796.96875 - response_length/max:3594.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:796.96875 - response_length_non_aborted/max:3594.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.5625 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7237460017204285e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9937344156205654) - timing_s/agent_loop/generate_sequences/max:np.float64(23.47761689312756) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.72609610089421) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.47761689312756) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:356 - timing_s/agent_loop/slowest/response_length:3594 - timing_s/gen:29.4396906606853 - timing_s/reward:0.07473043724894524 - timing_s/old_log_prob:2.594569843262434 - timing_s/adv:0.12372583709657192 - timing_s/update_actor:10.489614827558398 - timing_s/step:42.81442074663937 - timing_s/stop_profile:2.9224902391433716e-05 - timing_per_token_ms/update_actor:0.038599954471571135 - timing_per_token_ms/gen:0.14429523321121682 - timing_per_token_ms/adv:0.000455289518003812 - perf/total_num_tokens:271752 - perf/time_per_step:42.81442074663937 - perf/throughput:793.4009010892977 (TaskRunner pid=2049544) Training Progress: 84%|████████▍ | 84/100 [1:05:55<12:52, 48.31s/it] (TaskRunner pid=2049544) step:85 - global_seqlen/min:20755 - global_seqlen/max:54433 - global_seqlen/minmax_diff:33678 - global_seqlen/balanced_min:33282 - global_seqlen/balanced_max:33493 - global_seqlen/mean:33418.125 - actor/entropy:0.29106423258781433 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967511296272278 - training/rollout_probs_diff_mean:0.003082117298617959 - training/rollout_probs_diff_std:0.009240406565368176 - training/rollout_actor_probs_pearson_corr:0.9993290901184082 - rollout_corr/rollout_is_mean:1.0000280141830444 - rollout_corr/rollout_is_ratio_fraction_high:4.992785306967562e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.9971141227870248e-05 - rollout_corr/rollout_is_max:2.3519351482391357 - rollout_corr/rollout_is_min:0.26859748363494873 - rollout_corr/rollout_is_std:0.029412612318992615 - rollout_corr/rollout_is_eff_sample_size:0.9991358839466532 - rollout_corr/rollout_is_seq_mean:0.9999825954437256 - rollout_corr/rollout_is_seq_std:0.0015941840829327703 - rollout_corr/rollout_is_seq_max:1.0068665742874146 - rollout_corr/rollout_is_seq_min:0.9917056560516357 - rollout_corr/rollout_is_seq_max_deviation:0.008294343948364258 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2718443693593144) - rollout_corr/training_log_ppl:np.float64(0.22443578000093112) - rollout_corr/kl:np.float64(0.0005144908149414817) - rollout_corr/k3_kl:np.float64(0.0004981835849378058) - rollout_corr/rollout_ppl:np.float64(1.2711914060637355) - rollout_corr/rollout_log_ppl:np.float64(0.22392128901265096) - rollout_corr/log_ppl_diff:np.float64(0.0005144909882801585) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011518053579493426) - rollout_corr/log_ppl_diff_max:np.float64(0.009792208671569824) - rollout_corr/log_ppl_diff_min:np.float64(-0.005565732717514038) - rollout_corr/ppl_ratio:np.float64(1.000515874940902) - rollout_corr/chi2_token:np.float64(0.000967925414443016) - rollout_corr/chi2_seq:np.float64(1.0396802069153637) - actor/pg_loss:np.float64(-0.0001277067931368947) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000293373741897085) - actor/ppo_kl:np.float64(1.70920517623685e-05) - actor/pg_clipfrac_lower:np.float64(5.778476406703703e-06) - actor/grad_norm:np.float64(0.20695016533136368) - perf/mfu/actor:np.float64(0.09336689611238912) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.0907974243164) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:3 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.014600777067244053 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.014600777067244053 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:782.37890625 - response_length/max:4261.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:782.37890625 - response_length_non_aborted/max:4261.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.9375 - prompt_length/max:365.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.2804364562034607e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7515706215053797) - timing_s/agent_loop/generate_sequences/max:np.float64(26.178023107349873) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.071110863718786) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(26.178023107349873) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:4261 - timing_s/gen:31.903686491772532 - timing_s/reward:0.07636177726089954 - timing_s/old_log_prob:2.5928792282938957 - timing_s/adv:0.12288173846900463 - timing_s/update_actor:10.525410840287805 - timing_s/step:45.31749305874109 - timing_s/stop_profile:0.00011776015162467957 - timing_per_token_ms/update_actor:0.039370142850204064 - timing_per_token_ms/gen:0.15928826092183063 - timing_per_token_ms/adv:0.0004596373168340707 - perf/total_num_tokens:267345 - perf/time_per_step:45.31749305874109 - perf/throughput:737.4221905144448 (TaskRunner pid=2049544) Training Progress: 85%|████████▌ | 85/100 [1:06:41<11:51, 47.43s/it] (TaskRunner pid=2049544) step:86 - global_seqlen/min:22104 - global_seqlen/max:49662 - global_seqlen/minmax_diff:27558 - global_seqlen/balanced_min:36725 - global_seqlen/balanced_max:36775 - global_seqlen/mean:36762.375 - actor/entropy:0.3210241496562958 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24132895469665527 - training/rollout_probs_diff_mean:0.003310859901830554 - training/rollout_probs_diff_std:0.009120251052081585 - training/rollout_actor_probs_pearson_corr:0.9993811845779419 - rollout_corr/rollout_is_mean:0.9998584389686584 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.2271813577390276e-05 - rollout_corr/rollout_is_max:1.9132159948349 - rollout_corr/rollout_is_min:0.41626274585723877 - rollout_corr/rollout_is_std:0.029449068009853363 - rollout_corr/rollout_is_eff_sample_size:0.9991331468769348 - rollout_corr/rollout_is_seq_mean:0.9998147487640381 - rollout_corr/rollout_is_seq_std:0.0013719926355406642 - rollout_corr/rollout_is_seq_max:1.0041395425796509 - rollout_corr/rollout_is_seq_min:0.9918246865272522 - rollout_corr/rollout_is_seq_max_deviation:0.008175313472747803 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.318041059654206) - rollout_corr/training_log_ppl:np.float64(0.26210865678149275) - rollout_corr/kl:np.float64(0.0005536363921763154) - rollout_corr/k3_kl:np.float64(0.0004948169941201286) - rollout_corr/rollout_ppl:np.float64(1.317284729797393) - rollout_corr/rollout_log_ppl:np.float64(0.2615550198825076) - rollout_corr/log_ppl_diff:np.float64(0.0005536368989851326) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010979093494825065) - rollout_corr/log_ppl_diff_max:np.float64(0.0066015273332595825) - rollout_corr/log_ppl_diff_min:np.float64(-0.002927526831626892) - rollout_corr/ppl_ratio:np.float64(1.0005547057371587) - rollout_corr/chi2_token:np.float64(0.0008581595029681921) - rollout_corr/chi2_seq:np.float64(0.48298407695256174) - actor/pg_loss:np.float64(0.00010274478700011969) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012036708881169034) - actor/ppo_kl:np.float64(-9.579434273687681e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1470967587083578) - perf/mfu/actor:np.float64(0.1030748564778111) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.2135009765625) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:3 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007343573961406946 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007343573961406946 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:876.94921875 - response_length/max:3278.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:876.94921875 - response_length_non_aborted/max:3278.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.875 - prompt_length/max:365.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010039471089839935 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4529647156596184) - timing_s/agent_loop/generate_sequences/max:np.float64(22.549825344234705) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.41551528852142) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.549825344234705) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:244 - timing_s/agent_loop/slowest/response_length:3278 - timing_s/gen:28.685686687007546 - timing_s/reward:0.07788657583296299 - timing_s/old_log_prob:2.5779322646558285 - timing_s/adv:0.1303798407316208 - timing_s/update_actor:10.655327502638102 - timing_s/step:42.21963391080499 - timing_s/stop_profile:0.00011177733540534973 - timing_per_token_ms/update_actor:0.036230410516996325 - timing_per_token_ms/gen:0.1277764564074118 - timing_per_token_ms/adv:0.0004433195649479284 - perf/total_num_tokens:294099 - perf/time_per_step:42.21963391080499 - perf/throughput:870.7412072228237 (TaskRunner pid=2049544) Training Progress: 86%|████████▌ | 86/100 [1:07:23<10:42, 45.88s/it] (TaskRunner pid=2049544) step:87 - global_seqlen/min:29440 - global_seqlen/max:47768 - global_seqlen/minmax_diff:18328 - global_seqlen/balanced_min:35517 - global_seqlen/balanced_max:36384 - global_seqlen/mean:35692.75 - actor/entropy:0.3466551899909973 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6292303800582886 - training/rollout_probs_diff_mean:0.003489995375275612 - training/rollout_probs_diff_std:0.00928904302418232 - training/rollout_actor_probs_pearson_corr:0.9994551539421082 - rollout_corr/rollout_is_mean:1.0000280141830444 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.662482751882635e-06 - rollout_corr/rollout_is_max:1.702067494392395 - rollout_corr/rollout_is_min:0.26140108704566956 - rollout_corr/rollout_is_std:0.03009967692196369 - rollout_corr/rollout_is_eff_sample_size:0.9990948294754013 - rollout_corr/rollout_is_seq_mean:1.0000789165496826 - rollout_corr/rollout_is_seq_std:0.001175415120087564 - rollout_corr/rollout_is_seq_max:1.0036451816558838 - rollout_corr/rollout_is_seq_min:0.9961031079292297 - rollout_corr/rollout_is_seq_max_deviation:0.0038968920707702637 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.384202916175127) - rollout_corr/training_log_ppl:np.float64(0.2980515792296501) - rollout_corr/kl:np.float64(0.00042411336519165843) - rollout_corr/k3_kl:np.float64(0.00048466515880818406) - rollout_corr/rollout_ppl:np.float64(1.3836216540075839) - rollout_corr/rollout_log_ppl:np.float64(0.2976274649699917) - rollout_corr/log_ppl_diff:np.float64(0.00042411425965838134) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010522096417844296) - rollout_corr/log_ppl_diff_max:np.float64(0.004691094160079956) - rollout_corr/log_ppl_diff_min:np.float64(-0.005741387605667114) - rollout_corr/ppl_ratio:np.float64(1.0004250849597156) - rollout_corr/chi2_token:np.float64(0.001095710089430213) - rollout_corr/chi2_seq:np.float64(1.8486225868109614) - actor/pg_loss:np.float64(7.82626448199153e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016853076499501185) - actor/ppo_kl:np.float64(3.361720549453773e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18693841621279716) - perf/mfu/actor:np.float64(0.0996124497047341) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.18145751953125) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:3 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006155060138553381 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006155060138553381 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:837.8046875 - response_length/max:4550.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:837.8046875 - response_length_non_aborted/max:4550.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.59375 - prompt_length/max:364.0 - prompt_length/min:219.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001092962920665741 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4342339634895325) - timing_s/agent_loop/generate_sequences/max:np.float64(28.061727035790682) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.031970594347513) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(28.061727035790682) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:239 - timing_s/agent_loop/slowest/response_length:4550 - timing_s/gen:33.89907926507294 - timing_s/reward:0.07478075660765171 - timing_s/old_log_prob:2.563651405274868 - timing_s/adv:0.12060143612325191 - timing_s/update_actor:10.524738831445575 - timing_s/step:47.28383187763393 - timing_s/stop_profile:0.00016075000166893005 - timing_per_token_ms/update_actor:0.036858811773558965 - timing_per_token_ms/gen:0.15805387622540745 - timing_per_token_ms/adv:0.00042235970933611136 - perf/total_num_tokens:285542 - perf/time_per_step:47.28383187763393 - perf/throughput:754.8616214601526 (TaskRunner pid=2049544) Training Progress: 87%|████████▋ | 87/100 [1:08:10<10:02, 46.32s/it] (TaskRunner pid=2049544) step:88 - global_seqlen/min:22456 - global_seqlen/max:46812 - global_seqlen/minmax_diff:24356 - global_seqlen/balanced_min:35428 - global_seqlen/balanced_max:35596 - global_seqlen/mean:35553.625 - actor/entropy:0.3236216604709625 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21511626243591309 - training/rollout_probs_diff_mean:0.0032852201256901026 - training/rollout_probs_diff_std:0.009024620987474918 - training/rollout_actor_probs_pearson_corr:0.999454140663147 - rollout_corr/rollout_is_mean:1.0000885725021362 - rollout_corr/rollout_is_ratio_fraction_high:4.604963123711059e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.604963123711059e-06 - rollout_corr/rollout_is_max:2.0104920864105225 - rollout_corr/rollout_is_min:0.23549965023994446 - rollout_corr/rollout_is_std:0.029910963028669357 - rollout_corr/rollout_is_eff_sample_size:0.999106252989479 - rollout_corr/rollout_is_seq_mean:1.0000383853912354 - rollout_corr/rollout_is_seq_std:0.0014727648813277483 - rollout_corr/rollout_is_seq_max:1.0041821002960205 - rollout_corr/rollout_is_seq_min:0.9893348813056946 - rollout_corr/rollout_is_seq_max_deviation:0.01066511869430542 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.338182927109301) - rollout_corr/training_log_ppl:np.float64(0.2705517132417299) - rollout_corr/kl:np.float64(0.0005242268059881816) - rollout_corr/k3_kl:np.float64(0.000508854052498009) - rollout_corr/rollout_ppl:np.float64(1.3374784495681524) - rollout_corr/rollout_log_ppl:np.float64(0.27002748603990767) - rollout_corr/log_ppl_diff:np.float64(0.0005242272018222138) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011741442867787555) - rollout_corr/log_ppl_diff_max:np.float64(0.013874843716621399) - rollout_corr/log_ppl_diff_min:np.float64(-0.0045333802700042725) - rollout_corr/ppl_ratio:np.float64(1.0005257281009108) - rollout_corr/chi2_token:np.float64(0.0009958110749721527) - rollout_corr/chi2_seq:np.float64(1.3633345067501068) - actor/pg_loss:np.float64(1.946359407156706e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016658261313295952) - actor/ppo_kl:np.float64(7.023494554836418e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.13375208340585232) - perf/mfu/actor:np.float64(0.09886611268499279) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.15729522705078) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:3 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02385486103594303 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02385486103594303 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:848.26953125 - response_length/max:4442.0 - response_length/min:80.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:848.26953125 - response_length_non_aborted/max:4442.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.78125 - prompt_length/max:375.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.980984032154083e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.990953391417861) - timing_s/agent_loop/generate_sequences/max:np.float64(25.68886986002326) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.6535395380997215) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(25.68886986002326) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:4442 - timing_s/gen:31.64418187364936 - timing_s/reward:0.07790577039122581 - timing_s/old_log_prob:2.6047119572758675 - timing_s/adv:0.11337987333536148 - timing_s/update_actor:10.512858772650361 - timing_s/step:45.041199665516615 - timing_s/stop_profile:0.00016839616000652313 - timing_per_token_ms/update_actor:0.036961276004381975 - timing_per_token_ms/gen:0.1457202939516081 - timing_per_token_ms/adv:0.00039862276116486536 - perf/total_num_tokens:284429 - perf/time_per_step:45.041199665516615 - perf/throughput:789.3578604483693 (TaskRunner pid=2049544) Training Progress: 88%|████████▊ | 88/100 [1:08:55<09:11, 45.95s/it] (TaskRunner pid=2049544) step:89 - global_seqlen/min:23526 - global_seqlen/max:43993 - global_seqlen/minmax_diff:20467 - global_seqlen/balanced_min:32615 - global_seqlen/balanced_max:32696 - global_seqlen/mean:32672.25 - actor/entropy:0.3472384810447693 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7165358662605286 - training/rollout_probs_diff_mean:0.003400100627914071 - training/rollout_probs_diff_std:0.009368252009153366 - training/rollout_actor_probs_pearson_corr:0.9993821382522583 - rollout_corr/rollout_is_mean:0.9999688267707825 - rollout_corr/rollout_is_ratio_fraction_high:5.2063269322388805e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.123795977444388e-05 - rollout_corr/rollout_is_max:2.0695598125457764 - rollout_corr/rollout_is_min:0.2369716465473175 - rollout_corr/rollout_is_std:0.03049706295132637 - rollout_corr/rollout_is_eff_sample_size:0.9990706743631297 - rollout_corr/rollout_is_seq_mean:1.0000097751617432 - rollout_corr/rollout_is_seq_std:0.0016654012724757195 - rollout_corr/rollout_is_seq_max:1.007972240447998 - rollout_corr/rollout_is_seq_min:0.9951695799827576 - rollout_corr/rollout_is_seq_max_deviation:0.007972240447998047 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3569685937836766) - rollout_corr/training_log_ppl:np.float64(0.27856632304610685) - rollout_corr/kl:np.float64(0.0005558686453759343) - rollout_corr/k3_kl:np.float64(0.0005486493795388014) - rollout_corr/rollout_ppl:np.float64(1.3561613205820322) - rollout_corr/rollout_log_ppl:np.float64(0.27801045119122136) - rollout_corr/log_ppl_diff:np.float64(0.0005558718548854813) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011855626216856763) - rollout_corr/log_ppl_diff_max:np.float64(0.005568467080593109) - rollout_corr/log_ppl_diff_min:np.float64(-0.006579384207725525) - rollout_corr/ppl_ratio:np.float64(1.0005573213566095) - rollout_corr/chi2_token:np.float64(0.0010650090407580137) - rollout_corr/chi2_seq:np.float64(0.8179540059063584) - actor/pg_loss:np.float64(3.63390427082777e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022707523396547913) - actor/ppo_kl:np.float64(6.390776004217003e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3162117004394531) - perf/mfu/actor:np.float64(0.09205693492850642) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.16839599609375) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:4 - critic/score/mean:0.79296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.79296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01907663233578205 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01907663233578205 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:750.2890625 - response_length/max:3252.0 - response_length/min:118.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:750.2890625 - response_length_non_aborted/max:3252.0 - response_length_non_aborted/min:118.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003261677920818329 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3473277054727077) - timing_s/agent_loop/generate_sequences/max:np.float64(20.700606567785144) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.879736434209917) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.700606567785144) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:3252 - timing_s/gen:26.722555236890912 - timing_s/reward:0.0684677753597498 - timing_s/old_log_prob:2.669492883607745 - timing_s/adv:0.1653036493808031 - timing_s/update_actor:10.382515193894506 - timing_s/step:40.15039100125432 - timing_s/stop_profile:0.00012563727796077728 - timing_per_token_ms/update_actor:0.03972222296403869 - timing_per_token_ms/gen:0.13912635357669914 - timing_per_token_ms/adv:0.0006324313805324209 - perf/total_num_tokens:261378 - perf/time_per_step:40.15039100125432 - perf/throughput:813.7467453051529 (TaskRunner pid=2049544) Training Progress: 89%|████████▉ | 89/100 [1:09:36<08:07, 44.35s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2049544) (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) (TaskRunner pid=2049544) - Electric field $ E = 2.0 \, \text{N/C} $ (TaskRunner pid=2049544) - Distance $ r = 50 \, \text{cm} = 0.50 \, \text{m} $ (TaskRunner pid=2049544) - Coulomb's constant $ k = 8.99 \times 10^9 \, \text{N m}^2/\text{C}^2 $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We are to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the known values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \text{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \text{C} = 55.6 \, \text{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80/actor (TaskRunner pid=2049544) step:90 - global_seqlen/min:22189 - global_seqlen/max:43070 - global_seqlen/minmax_diff:20881 - global_seqlen/balanced_min:29230 - global_seqlen/balanced_max:29298 - global_seqlen/mean:29287.75 - actor/entropy:0.3101608455181122 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3454935550689697 - training/rollout_probs_diff_mean:0.0033272632863372564 - training/rollout_probs_diff_std:0.009299682453274727 - training/rollout_actor_probs_pearson_corr:0.999302327632904 - rollout_corr/rollout_is_mean:0.9999826550483704 - rollout_corr/rollout_is_ratio_fraction_high:1.2050661098328419e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.2050661098328419e-05 - rollout_corr/rollout_is_max:2.8188319206237793 - rollout_corr/rollout_is_min:0.3406546115875244 - rollout_corr/rollout_is_std:0.03000447526574135 - rollout_corr/rollout_is_eff_sample_size:0.9991005411997865 - rollout_corr/rollout_is_seq_mean:1.0000338554382324 - rollout_corr/rollout_is_seq_std:0.0013818239094689488 - rollout_corr/rollout_is_seq_max:1.0056010484695435 - rollout_corr/rollout_is_seq_min:0.9940690398216248 - rollout_corr/rollout_is_seq_max_deviation:0.005930960178375244 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3156746202148497) - rollout_corr/training_log_ppl:np.float64(0.2531014566193335) - rollout_corr/kl:np.float64(0.0005285690323812986) - rollout_corr/k3_kl:np.float64(0.0004731722787880699) - rollout_corr/rollout_ppl:np.float64(1.3148961896076798) - rollout_corr/rollout_log_ppl:np.float64(0.25257288754801266) - rollout_corr/log_ppl_diff:np.float64(0.0005285690713208169) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010851562838070095) - rollout_corr/log_ppl_diff_max:np.float64(0.006567090749740601) - rollout_corr/log_ppl_diff_min:np.float64(-0.00448918342590332) - rollout_corr/ppl_ratio:np.float64(1.0005297421012074) - rollout_corr/chi2_token:np.float64(0.0008521552663296461) - rollout_corr/chi2_seq:np.float64(0.22603251412510872) - actor/pg_loss:np.float64(0.00011028861626982689) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012377322673273738) - actor/ppo_kl:np.float64(0.00010799788028137414) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.14029540959745646) - perf/mfu/actor:np.float64(0.08421706394502831) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.20226669311523) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6625) - val-aux/sciknoweval/reward/std@16:np.float64(0.18997502251966206) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.739025) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1632991687244558) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.587875) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15720634779503662) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6636) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1893282012924375) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8018125000000002) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.12815819440702003) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5259125000000001) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1162500052562232) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6786875) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1497440626684113) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8526875) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09374063114057554) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4800625) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07881481778772832) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.688175) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10800278634919543) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8915625) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.06088379895875816) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.449325) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04701965592013402) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.693225) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.0730352339040406) - val-aux/sciknoweval/score/mean@16:np.float64(0.6625) - val-aux/sciknoweval/score/std@16:np.float64(0.18997502251966206) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.739025) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1632991687244558) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.587875) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15720634779503662) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6636) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1893282012924375) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8018125000000002) - val-aux/sciknoweval/score/best@4/std:np.float64(0.12815819440702003) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5259125000000001) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1162500052562232) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6786875) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1497440626684113) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8526875) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09374063114057554) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4800625) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07881481778772832) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.688175) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10800278634919543) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8915625) - val-aux/sciknoweval/score/best@16/std:np.float64(0.06088379895875816) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.449325) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04701965592013402) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.693225) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.0730352339040406) - val-core/sciknoweval/acc/mean@16:np.float64(0.6625) - val-aux/sciknoweval/acc/std@16:np.float64(0.18997502251966206) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.739025) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1632991687244558) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.587875) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15720634779503662) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6636) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1893282012924375) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8018125000000002) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.12815819440702003) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5259125000000001) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1162500052562232) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6786875) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1497440626684113) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8526875) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09374063114057554) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4800625) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07881481778772832) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.688175) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10800278634919543) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8915625) - val-core/sciknoweval/acc/best@16/std:np.float64(0.06088379895875816) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.449325) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04701965592013402) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.693225) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.0730352339040406) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9890625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.03660810518441162) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9990500000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.010677570009542227) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9789875) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.048719255966368245) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.989175) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.036500484252753186) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9618874999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.06036258960895739) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9962249999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.021651315045931742) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9349000000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.06689717910953397) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9992000000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.009789340111118103) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9021374999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.05987592930423645) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999375) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0017437187447881502) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:4 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01157616637647152 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01157616637647152 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:648.3046875 - response_length/max:2634.0 - response_length/min:157.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:648.3046875 - response_length_non_aborted/max:2634.0 - response_length_non_aborted/min:157.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.9375 - prompt_length/max:437.0 - prompt_length/min:210.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011180341243743896 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8763771429657936) - timing_s/agent_loop/generate_sequences/max:np.float64(17.981070213019848) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.232629431753594) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.981070213019848) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:2634 - timing_s/gen:23.51352482289076 - timing_s/reward:0.0758508276194334 - timing_s/old_log_prob:2.507504915818572 - timing_s/adv:0.1161189116537571 - timing_s/update_actor:10.261307056993246 - timing_s/step:36.5611352995038 - timing_s/testing:112.82421230524778 - timing_s/save_checkpoint:6.677826223894954 - timing_s/stop_profile:0.0001322384923696518 - timing_per_token_ms/update_actor:0.04379521752692357 - timing_per_token_ms/gen:0.14167675802809465 - timing_per_token_ms/adv:0.0004955950510612675 - perf/total_num_tokens:234302 - perf/time_per_step:36.5611352995038 - perf/throughput:801.0623783993241 (TaskRunner pid=2049544) Training Progress: 90%|█████████ | 90/100 [1:12:12<12:58, 77.88s/it] (TaskRunner pid=2049544) step:91 - global_seqlen/min:21497 - global_seqlen/max:45454 - global_seqlen/minmax_diff:23957 - global_seqlen/balanced_min:32904 - global_seqlen/balanced_max:33139 - global_seqlen/mean:33002.875 - actor/entropy:0.26041534543037415 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34585317969322205 - training/rollout_probs_diff_mean:0.002946822438389063 - training/rollout_probs_diff_std:0.008950787596404552 - training/rollout_actor_probs_pearson_corr:0.9992419481277466 - rollout_corr/rollout_is_mean:0.9999063611030579 - rollout_corr/rollout_is_ratio_fraction_high:5.014869202568661e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.5044607607705984e-05 - rollout_corr/rollout_is_max:5.096095561981201 - rollout_corr/rollout_is_min:0.37255313992500305 - rollout_corr/rollout_is_std:0.028246553614735603 - rollout_corr/rollout_is_eff_sample_size:0.9992028873298154 - rollout_corr/rollout_is_seq_mean:1.0000227689743042 - rollout_corr/rollout_is_seq_std:0.001477584009990096 - rollout_corr/rollout_is_seq_max:1.0067633390426636 - rollout_corr/rollout_is_seq_min:0.995822548866272 - rollout_corr/rollout_is_seq_max_deviation:0.006763339042663574 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2352091469801962) - rollout_corr/training_log_ppl:np.float64(0.20221140759531409) - rollout_corr/kl:np.float64(0.00045496249386900445) - rollout_corr/k3_kl:np.float64(0.00047493476955651204) - rollout_corr/rollout_ppl:np.float64(1.2346553695388138) - rollout_corr/rollout_log_ppl:np.float64(0.2017564445413882) - rollout_corr/log_ppl_diff:np.float64(0.0004549630539258942) - rollout_corr/log_ppl_abs_diff:np.float64(0.001167977330624126) - rollout_corr/log_ppl_diff_max:np.float64(0.009916573762893677) - rollout_corr/log_ppl_diff_min:np.float64(-0.006213173270225525) - rollout_corr/ppl_ratio:np.float64(1.0004563292022794) - rollout_corr/chi2_token:np.float64(0.0009643365629017353) - rollout_corr/chi2_seq:np.float64(0.6764775053597987) - actor/pg_loss:np.float64(5.630298983305693e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001349058953792337) - actor/ppo_kl:np.float64(3.4980036296872186e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12929855473339558) - perf/mfu/actor:np.float64(0.09015558565254489) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.73992156982422) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:4 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012762214988470078 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012762214988470078 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:778.93359375 - response_length/max:5011.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:778.93359375 - response_length_non_aborted/max:5011.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:252.40625 - prompt_length/max:340.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.5048614740371704e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9828272927552462) - timing_s/agent_loop/generate_sequences/max:np.float64(30.033628037199378) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.231723692668311) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(30.033628037199378) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:5011 - timing_s/gen:35.68446674197912 - timing_s/reward:0.0753241665661335 - timing_s/old_log_prob:2.7556536067277193 - timing_s/adv:0.12121403589844704 - timing_s/update_actor:10.860587490722537 - timing_s/step:49.55868074670434 - timing_s/stop_profile:0.00011924095451831818 - timing_per_token_ms/update_actor:0.04113500524849175 - timing_per_token_ms/gen:0.17895292914480998 - timing_per_token_ms/adv:0.0004591040776691691 - perf/total_num_tokens:264023 - perf/time_per_step:49.55868074670434 - perf/throughput:665.9353013991337 (TaskRunner pid=2049544) Training Progress: 91%|█████████ | 91/100 [1:13:02<10:24, 69.39s/it] (TaskRunner pid=2049544) step:92 - global_seqlen/min:27995 - global_seqlen/max:54733 - global_seqlen/minmax_diff:26738 - global_seqlen/balanced_min:38799 - global_seqlen/balanced_max:39948 - global_seqlen/mean:39032.875 - actor/entropy:0.365276038646698 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4553599953651428 - training/rollout_probs_diff_mean:0.0034760674461722374 - training/rollout_probs_diff_std:0.009343450888991356 - training/rollout_actor_probs_pearson_corr:0.999370813369751 - rollout_corr/rollout_is_mean:0.9999127984046936 - rollout_corr/rollout_is_ratio_fraction_high:8.297963177028578e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.0744908397318795e-05 - rollout_corr/rollout_is_max:3.3402388095855713 - rollout_corr/rollout_is_min:0.3921363651752472 - rollout_corr/rollout_is_std:0.03065885230898857 - rollout_corr/rollout_is_eff_sample_size:0.999060798471362 - rollout_corr/rollout_is_seq_mean:0.9999949932098389 - rollout_corr/rollout_is_seq_std:0.001438560662791133 - rollout_corr/rollout_is_seq_max:1.005057454109192 - rollout_corr/rollout_is_seq_min:0.9950283765792847 - rollout_corr/rollout_is_seq_max_deviation:0.0050574541091918945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4253517426550388) - rollout_corr/training_log_ppl:np.float64(0.3268485037915525) - rollout_corr/kl:np.float64(0.0007143817989492263) - rollout_corr/k3_kl:np.float64(0.0007667658379375553) - rollout_corr/rollout_ppl:np.float64(1.4242633627727628) - rollout_corr/rollout_log_ppl:np.float64(0.3261341213074047) - rollout_corr/log_ppl_diff:np.float64(0.0007143824841477908) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013714263404835947) - rollout_corr/log_ppl_diff_max:np.float64(0.013571739196777344) - rollout_corr/log_ppl_diff_min:np.float64(-0.0037727057933807373) - rollout_corr/ppl_ratio:np.float64(1.000716715119779) - rollout_corr/chi2_token:np.float64(0.001422470435500145) - rollout_corr/chi2_seq:np.float64(0.832664096262306) - actor/pg_loss:np.float64(2.3467233404517174e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003434544945548623) - actor/ppo_kl:np.float64(0.00016380211024369373) - actor/pg_clipfrac_lower:np.float64(2.0755844616360264e-06) - actor/grad_norm:np.float64(0.215729471296072) - perf/mfu/actor:np.float64(0.10647986423451218) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.4896354675293) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:4 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011413848027586937 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011413848027586937 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:941.49609375 - response_length/max:5473.0 - response_length/min:148.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:941.49609375 - response_length_non_aborted/max:5473.0 - response_length_non_aborted/min:148.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.28125 - prompt_length/max:460.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.439482629299164e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7557445913553238) - timing_s/agent_loop/generate_sequences/max:np.float64(32.87127552181482) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.810325469457894) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(32.87127552181482) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:5473 - timing_s/gen:38.770385617390275 - timing_s/reward:0.08092910423874855 - timing_s/old_log_prob:2.7198645491153 - timing_s/adv:0.12597152031958103 - timing_s/update_actor:10.861796030774713 - timing_s/step:52.657433899119496 - timing_s/stop_profile:0.00011220574378967285 - timing_per_token_ms/update_actor:0.034784127580836384 - timing_per_token_ms/gen:0.1608576178098782 - timing_per_token_ms/adv:0.00040341481481821745 - perf/total_num_tokens:312263 - perf/time_per_step:52.657433899119496 - perf/throughput:741.2604851724968 (TaskRunner pid=2049544) Training Progress: 92%|█████████▏| 92/100 [1:13:54<08:35, 64.39s/it] (TaskRunner pid=2049544) step:93 - global_seqlen/min:17843 - global_seqlen/max:51915 - global_seqlen/minmax_diff:34072 - global_seqlen/balanced_min:30929 - global_seqlen/balanced_max:31113 - global_seqlen/mean:31065.0 - actor/entropy:0.4133561849594116 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311214327812195 - training/rollout_probs_diff_mean:0.0036772009916603565 - training/rollout_probs_diff_std:0.009407786652445793 - training/rollout_actor_probs_pearson_corr:0.9994404911994934 - rollout_corr/rollout_is_mean:0.9999762177467346 - rollout_corr/rollout_is_ratio_fraction_high:5.474292720464291e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.6422878616140224e-05 - rollout_corr/rollout_is_max:2.111367702484131 - rollout_corr/rollout_is_min:0.21797895431518555 - rollout_corr/rollout_is_std:0.031457215547561646 - rollout_corr/rollout_is_eff_sample_size:0.9990114219410976 - rollout_corr/rollout_is_seq_mean:1.0000097751617432 - rollout_corr/rollout_is_seq_std:0.0018004179000854492 - rollout_corr/rollout_is_seq_max:1.007155179977417 - rollout_corr/rollout_is_seq_min:0.9935778975486755 - rollout_corr/rollout_is_seq_max_deviation:0.007155179977416992 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4254543129354715) - rollout_corr/training_log_ppl:np.float64(0.3189181691413978) - rollout_corr/kl:np.float64(0.0006560000657124254) - rollout_corr/k3_kl:np.float64(0.0006174342987037562) - rollout_corr/rollout_ppl:np.float64(1.4244641666300595) - rollout_corr/rollout_log_ppl:np.float64(0.31826216776971705) - rollout_corr/log_ppl_diff:np.float64(0.0006560013716807589) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014309252292150632) - rollout_corr/log_ppl_diff_max:np.float64(0.00973115861415863) - rollout_corr/log_ppl_diff_min:np.float64(-0.004590287804603577) - rollout_corr/ppl_ratio:np.float64(1.0006579444743693) - rollout_corr/chi2_token:np.float64(0.0011531137861311436) - rollout_corr/chi2_seq:np.float64(0.8776039613876492) - actor/pg_loss:np.float64(0.00012877536937594414) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000180150422920633) - actor/ppo_kl:np.float64(9.64217530006195e-05) - actor/pg_clipfrac_lower:np.float64(3.570612534531392e-06) - actor/grad_norm:np.float64(0.18000993877649307) - perf/mfu/actor:np.float64(0.08642174822282235) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.50779724121094) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:4 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.016780760139226913 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.016780760139226913 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:713.5625 - response_length/max:4647.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:713.5625 - response_length_non_aborted/max:4647.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:257.21875 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010390952229499817 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7623797114938498) - timing_s/agent_loop/generate_sequences/max:np.float64(27.237358892336488) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.646888445196964) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.237358892336488) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:4647 - timing_s/gen:33.228857627138495 - timing_s/reward:0.07251898013055325 - timing_s/old_log_prob:2.6381760984659195 - timing_s/adv:0.12340777926146984 - timing_s/update_actor:10.632954278960824 - timing_s/step:46.78136018849909 - timing_s/stop_profile:0.0001156739890575409 - timing_per_token_ms/update_actor:0.04278510493707075 - timing_per_token_ms/gen:0.18190449344802978 - timing_per_token_ms/adv:0.0004965708162782466 - perf/total_num_tokens:248520 - perf/time_per_step:46.78136018849909 - perf/throughput:664.0465320979944 (TaskRunner pid=2049544) Training Progress: 93%|█████████▎| 93/100 [1:14:41<06:53, 59.12s/it] (TaskRunner pid=2049544) step:94 - global_seqlen/min:22853 - global_seqlen/max:49175 - global_seqlen/minmax_diff:26322 - global_seqlen/balanced_min:33215 - global_seqlen/balanced_max:33359 - global_seqlen/mean:33330.0 - actor/entropy:0.36058807373046875 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8867406845092773 - training/rollout_probs_diff_mean:0.003440327476710081 - training/rollout_probs_diff_std:0.009632454253733158 - training/rollout_actor_probs_pearson_corr:0.9993698000907898 - rollout_corr/rollout_is_mean:0.9999211430549622 - rollout_corr/rollout_is_ratio_fraction_high:1.0051463505078573e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.523158713709563e-05 - rollout_corr/rollout_is_max:2.117000102996826 - rollout_corr/rollout_is_min:0.0013813151745125651 - rollout_corr/rollout_is_std:0.030682172626256943 - rollout_corr/rollout_is_eff_sample_size:0.9990594896328889 - rollout_corr/rollout_is_seq_mean:0.9999591112136841 - rollout_corr/rollout_is_seq_std:0.0020710721146315336 - rollout_corr/rollout_is_seq_max:1.011606216430664 - rollout_corr/rollout_is_seq_min:0.9853842854499817 - rollout_corr/rollout_is_seq_max_deviation:0.01461571455001831 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3442378360778093) - rollout_corr/training_log_ppl:np.float64(0.2689030548208393) - rollout_corr/kl:np.float64(0.0007270200237998736) - rollout_corr/k3_kl:np.float64(0.0005578451756775848) - rollout_corr/rollout_ppl:np.float64(1.3432344142347574) - rollout_corr/rollout_log_ppl:np.float64(0.26817603285599034) - rollout_corr/log_ppl_diff:np.float64(0.0007270219648489729) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013523521920433268) - rollout_corr/log_ppl_diff_max:np.float64(0.018366605043411255) - rollout_corr/log_ppl_diff_min:np.float64(-0.00493505597114563) - rollout_corr/ppl_ratio:np.float64(1.0007292504888028) - rollout_corr/chi2_token:np.float64(0.0007120152004063129) - rollout_corr/chi2_seq:np.float64(0.3402676989790052) - actor/pg_loss:np.float64(-4.822376649826765e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.676928118551587e-05) - actor/ppo_kl:np.float64(0.00014517965372551345) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16034816205501556) - perf/mfu/actor:np.float64(0.09308032359557787) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.54820251464844) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:4 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0465502105653286 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0465502105653286 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:777.25 - response_length/max:3614.0 - response_length/min:72.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:777.25 - response_length_non_aborted/max:3614.0 - response_length_non_aborted/min:72.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.3125 - prompt_length/max:341.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001496635377407074 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8505959454923868) - timing_s/agent_loop/generate_sequences/max:np.float64(23.250541422516108) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.11456365948834) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.250541422516108) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:293 - timing_s/agent_loop/slowest/response_length:3614 - timing_s/gen:29.206484995782375 - timing_s/reward:0.07558401487767696 - timing_s/old_log_prob:2.6541784573346376 - timing_s/adv:0.12619186379015446 - timing_s/update_actor:10.563141915947199 - timing_s/step:42.71316540054977 - timing_s/stop_profile:0.00011851266026496887 - timing_per_token_ms/update_actor:0.03961574375917792 - timing_per_token_ms/gen:0.14678395884821474 - timing_per_token_ms/adv:0.0004732668158946687 - perf/total_num_tokens:266640 - perf/time_per_step:42.71316540054977 - perf/throughput:780.3214696790185 (TaskRunner pid=2049544) Training Progress: 94%|█████████▍| 94/100 [1:15:24<05:25, 54.21s/it] (TaskRunner pid=2049544) step:95 - global_seqlen/min:27066 - global_seqlen/max:58256 - global_seqlen/minmax_diff:31190 - global_seqlen/balanced_min:37178 - global_seqlen/balanced_max:37249 - global_seqlen/mean:37221.375 - actor/entropy:0.3698161244392395 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553989887237549 - training/rollout_probs_diff_mean:0.0034687381703406572 - training/rollout_probs_diff_std:0.009108307771384716 - training/rollout_actor_probs_pearson_corr:0.999431848526001 - rollout_corr/rollout_is_mean:0.9999311566352844 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.3214753380452748e-05 - rollout_corr/rollout_is_max:1.8554537296295166 - rollout_corr/rollout_is_min:0.4261438846588135 - rollout_corr/rollout_is_std:0.030212340876460075 - rollout_corr/rollout_is_eff_sample_size:0.9990878089036473 - rollout_corr/rollout_is_seq_mean:0.9999582767486572 - rollout_corr/rollout_is_seq_std:0.0013638396048918366 - rollout_corr/rollout_is_seq_max:1.0052945613861084 - rollout_corr/rollout_is_seq_min:0.9958161115646362 - rollout_corr/rollout_is_seq_max_deviation:0.0052945613861083984 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3786341231316328) - rollout_corr/training_log_ppl:np.float64(0.30179976880026516) - rollout_corr/kl:np.float64(0.0004993855567150263) - rollout_corr/k3_kl:np.float64(0.0005308137937447555) - rollout_corr/rollout_ppl:np.float64(1.377938257995993) - rollout_corr/rollout_log_ppl:np.float64(0.30130038285278715) - rollout_corr/log_ppl_diff:np.float64(0.0004993859474780038) - rollout_corr/log_ppl_abs_diff:np.float64(0.001156414728029631) - rollout_corr/log_ppl_diff_max:np.float64(0.0048898980021476746) - rollout_corr/log_ppl_diff_min:np.float64(-0.00559312105178833) - rollout_corr/ppl_ratio:np.float64(1.0005006100982428) - rollout_corr/chi2_token:np.float64(0.001144631765782833) - rollout_corr/chi2_seq:np.float64(0.9150484425481409) - actor/pg_loss:np.float64(-2.8662499971687794e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000206823247367538) - actor/ppo_kl:np.float64(-2.0183409755958337e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1875382363796234) - perf/mfu/actor:np.float64(0.10496417728509784) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.40451049804688) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:4 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.021443139761686325 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.021443139761686325 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:886.79296875 - response_length/max:3209.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:886.79296875 - response_length_non_aborted/max:3209.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.375 - prompt_length/max:375.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.542517364025116e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.471508091315627) - timing_s/agent_loop/generate_sequences/max:np.float64(22.023407490924) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.383664465822221) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.023407490924) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:277 - timing_s/agent_loop/slowest/response_length:3175 - timing_s/gen:28.187608854845166 - timing_s/reward:0.08735943771898746 - timing_s/old_log_prob:2.6067604515701532 - timing_s/adv:0.13234028778970242 - timing_s/update_actor:10.463114799931645 - timing_s/step:41.565891321748495 - timing_s/stop_profile:0.00011608004570007324 - timing_per_token_ms/update_actor:0.035138125606360746 - timing_per_token_ms/gen:0.12416409575782276 - timing_per_token_ms/adv:0.00044443645549668175 - perf/total_num_tokens:297771 - perf/time_per_step:41.565891321748495 - perf/throughput:895.4788124686426 (TaskRunner pid=2049544) Training Progress: 95%|█████████▌| 95/100 [1:16:06<04:12, 50.43s/it] (TaskRunner pid=2049544) step:96 - global_seqlen/min:20271 - global_seqlen/max:71208 - global_seqlen/minmax_diff:50937 - global_seqlen/balanced_min:37576 - global_seqlen/balanced_max:37647 - global_seqlen/mean:37613.5 - actor/entropy:0.334806889295578 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31711164116859436 - training/rollout_probs_diff_mean:0.003466662485152483 - training/rollout_probs_diff_std:0.009574675001204014 - training/rollout_actor_probs_pearson_corr:0.9992694854736328 - rollout_corr/rollout_is_mean:1.0000592470169067 - rollout_corr/rollout_is_ratio_fraction_high:8.720068763068411e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.720068763068411e-06 - rollout_corr/rollout_is_max:2.086963415145874 - rollout_corr/rollout_is_min:0.4454006552696228 - rollout_corr/rollout_is_std:0.030977971851825714 - rollout_corr/rollout_is_eff_sample_size:0.9990414042159874 - rollout_corr/rollout_is_seq_mean:1.0001949071884155 - rollout_corr/rollout_is_seq_std:0.0014157574623823166 - rollout_corr/rollout_is_seq_max:1.0063108205795288 - rollout_corr/rollout_is_seq_min:0.9960592985153198 - rollout_corr/rollout_is_seq_max_deviation:0.006310820579528809 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3516691238619387) - rollout_corr/training_log_ppl:np.float64(0.27739123623177875) - rollout_corr/kl:np.float64(0.00038778805755357304) - rollout_corr/k3_kl:np.float64(0.0005165411663199393) - rollout_corr/rollout_ppl:np.float64(1.3510792162269354) - rollout_corr/rollout_log_ppl:np.float64(0.277003448762116) - rollout_corr/log_ppl_diff:np.float64(0.0003877874696627259) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011570610222406685) - rollout_corr/log_ppl_diff_max:np.float64(0.00403740257024765) - rollout_corr/log_ppl_diff_min:np.float64(-0.005538493394851685) - rollout_corr/ppl_ratio:np.float64(1.000388921238482) - rollout_corr/chi2_token:np.float64(0.0012935972772538662) - rollout_corr/chi2_seq:np.float64(2.2381823365576565) - actor/pg_loss:np.float64(5.385885015130043e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015849828230329877) - actor/ppo_kl:np.float64(8.225317838395085e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11362059134989977) - perf/mfu/actor:np.float64(0.1031450502424441) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.59148025512695) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:4 - critic/score/mean:0.67578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.67578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007041455246508121 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0007041455246508121 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:895.921875 - response_length/max:4074.0 - response_length/min:184.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:895.921875 - response_length_non_aborted/max:4074.0 - response_length_non_aborted/min:184.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.5 - prompt_length/max:355.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.657893002033234e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2360860873013735) - timing_s/agent_loop/generate_sequences/max:np.float64(26.039900217205286) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.459900658301194) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(26.039900217205286) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:314 - timing_s/agent_loop/slowest/response_length:3982 - timing_s/gen:32.00374800339341 - timing_s/reward:0.0823327787220478 - timing_s/old_log_prob:2.7716220505535603 - timing_s/adv:0.1664755642414093 - timing_s/update_actor:10.801547262817621 - timing_s/step:45.91564348526299 - timing_s/stop_profile:0.00011668726801872253 - timing_per_token_ms/update_actor:0.035896510770127815 - timing_per_token_ms/gen:0.13953743526828777 - timing_per_token_ms/adv:0.0005532440621100446 - perf/total_num_tokens:300908 - perf/time_per_step:45.91564348526299 - perf/throughput:819.1870383363432 (TaskRunner pid=2049544) Training Progress: 96%|█████████▌| 96/100 [1:16:52<03:16, 49.09s/it] (TaskRunner pid=2049544) step:97 - global_seqlen/min:20006 - global_seqlen/max:63623 - global_seqlen/minmax_diff:43617 - global_seqlen/balanced_min:40703 - global_seqlen/balanced_max:40790 - global_seqlen/mean:40767.5 - actor/entropy:0.4114430248737335 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3968365788459778 - training/rollout_probs_diff_mean:0.0036807965952903032 - training/rollout_probs_diff_std:0.009112636558711529 - training/rollout_actor_probs_pearson_corr:0.9994710087776184 - rollout_corr/rollout_is_mean:0.9999697208404541 - rollout_corr/rollout_is_ratio_fraction_high:3.885426394845126e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.5541705579380505e-05 - rollout_corr/rollout_is_max:6.432518005371094 - rollout_corr/rollout_is_min:0.14259245991706848 - rollout_corr/rollout_is_std:0.030860379338264465 - rollout_corr/rollout_is_eff_sample_size:0.9990483051530745 - rollout_corr/rollout_is_seq_mean:1.000016450881958 - rollout_corr/rollout_is_seq_std:0.0015233448939397931 - rollout_corr/rollout_is_seq_max:1.0063254833221436 - rollout_corr/rollout_is_seq_min:0.994326114654541 - rollout_corr/rollout_is_seq_max_deviation:0.006325483322143555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3867830368690193) - rollout_corr/training_log_ppl:np.float64(0.30247966502793133) - rollout_corr/kl:np.float64(0.0005262127845906406) - rollout_corr/k3_kl:np.float64(0.0005436497510515892) - rollout_corr/rollout_ppl:np.float64(1.386001723818481) - rollout_corr/rollout_log_ppl:np.float64(0.30195345119864214) - rollout_corr/log_ppl_diff:np.float64(0.0005262138292891905) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011905098654096946) - rollout_corr/log_ppl_diff_max:np.float64(0.007206156849861145) - rollout_corr/log_ppl_diff_min:np.float64(-0.003447234630584717) - rollout_corr/ppl_ratio:np.float64(1.0005276133306324) - rollout_corr/chi2_token:np.float64(0.001151977339759469) - rollout_corr/chi2_seq:np.float64(0.950474871089682) - actor/pg_loss:np.float64(6.92273024469614e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019685469499108876) - actor/ppo_kl:np.float64(3.492400511362348e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16588949039578438) - perf/mfu/actor:np.float64(0.11018383492360453) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.53912734985352) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:4 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.043718818575143814 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.043718818575143814 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1005.359375 - response_length/max:4322.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1005.359375 - response_length_non_aborted/max:4322.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.625 - prompt_length/max:353.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012983568012714386 - timing_s/agent_loop/generate_sequences/min:np.float64(1.021046968176961) - timing_s/agent_loop/generate_sequences/max:np.float64(28.757525622844696) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.163475782763271) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(28.757525622844696) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:353 - timing_s/agent_loop/slowest/response_length:4322 - timing_s/gen:34.78881582431495 - timing_s/reward:0.08717712201178074 - timing_s/old_log_prob:2.7238584086298943 - timing_s/adv:0.1590470802038908 - timing_s/update_actor:10.954232659190893 - timing_s/step:48.80308444797993 - timing_s/stop_profile:0.00013154186308383942 - timing_per_token_ms/update_actor:0.0335875165854875 - timing_per_token_ms/gen:0.13516938837291917 - timing_per_token_ms/adv:0.000487665052443401 - perf/total_num_tokens:326140 - perf/time_per_step:48.80308444797993 - perf/throughput:835.3467913171513 (TaskRunner pid=2049544) Training Progress: 97%|█████████▋| 97/100 [1:17:40<02:27, 49.03s/it] (TaskRunner pid=2049544) step:98 - global_seqlen/min:23082 - global_seqlen/max:43864 - global_seqlen/minmax_diff:20782 - global_seqlen/balanced_min:32541 - global_seqlen/balanced_max:33196 - global_seqlen/mean:32691.375 - actor/entropy:0.3132493495941162 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6835976243019104 - training/rollout_probs_diff_mean:0.0034196285996586084 - training/rollout_probs_diff_std:0.009468399919569492 - training/rollout_actor_probs_pearson_corr:0.9993211627006531 - rollout_corr/rollout_is_mean:1.0001163482666016 - rollout_corr/rollout_is_ratio_fraction_high:1.5585144865326583e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.5585144865326583e-05 - rollout_corr/rollout_is_max:5.03674840927124 - rollout_corr/rollout_is_min:0.0648747980594635 - rollout_corr/rollout_is_std:0.03038937970995903 - rollout_corr/rollout_is_eff_sample_size:0.9990775757045947 - rollout_corr/rollout_is_seq_mean:1.0001397132873535 - rollout_corr/rollout_is_seq_std:0.001354765729047358 - rollout_corr/rollout_is_seq_max:1.0069159269332886 - rollout_corr/rollout_is_seq_min:0.9959137439727783 - rollout_corr/rollout_is_seq_max_deviation:0.006915926933288574 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2990983063355088) - rollout_corr/training_log_ppl:np.float64(0.24720221558527555) - rollout_corr/kl:np.float64(0.0005138034307572426) - rollout_corr/k3_kl:np.float64(0.0005089092589116717) - rollout_corr/rollout_ppl:np.float64(1.298400120344013) - rollout_corr/rollout_log_ppl:np.float64(0.24668841203674674) - rollout_corr/log_ppl_diff:np.float64(0.0005138035485288128) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011324260121909901) - rollout_corr/log_ppl_diff_max:np.float64(0.00813785195350647) - rollout_corr/log_ppl_diff_min:np.float64(-0.005736410617828369) - rollout_corr/ppl_ratio:np.float64(1.0005151100922376) - rollout_corr/chi2_token:np.float64(0.0009681761730462313) - rollout_corr/chi2_seq:np.float64(4.5980356687214226) - actor/pg_loss:np.float64(-9.167857933789492e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002514609163881687) - actor/ppo_kl:np.float64(0.00017433107315412144) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1871425360441208) - perf/mfu/actor:np.float64(0.09212177950831188) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.56390380859375) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:4 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0003707965661305934 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0003707965661305934 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:751.91796875 - response_length/max:4148.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:751.91796875 - response_length_non_aborted/max:4148.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.6875 - prompt_length/max:350.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010955892503261566 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1268691644072533) - timing_s/agent_loop/generate_sequences/max:np.float64(24.762351796031) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.324899325562001) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(24.762351796031) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:304 - timing_s/agent_loop/slowest/response_length:4148 - timing_s/gen:30.538867769762874 - timing_s/reward:0.07450433447957039 - timing_s/old_log_prob:2.632541459053755 - timing_s/adv:0.1884304340928793 - timing_s/update_actor:10.359910435974598 - timing_s/step:43.87974337860942 - timing_s/stop_profile:0.000130457803606987 - timing_per_token_ms/update_actor:0.03961255237801484 - timing_per_token_ms/gen:0.15865088637787156 - timing_per_token_ms/adv:0.0007204898619776597 - perf/total_num_tokens:261531 - perf/time_per_step:43.87974337860942 - perf/throughput:745.0220188830104 (TaskRunner pid=2049544) Training Progress: 98%|█████████▊| 98/100 [1:18:24<01:34, 47.50s/it] (TaskRunner pid=2049544) step:99 - global_seqlen/min:21119 - global_seqlen/max:73608 - global_seqlen/minmax_diff:52489 - global_seqlen/balanced_min:37066 - global_seqlen/balanced_max:37818 - global_seqlen/mean:37233.375 - actor/entropy:0.36280471086502075 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29852432012557983 - training/rollout_probs_diff_mean:0.0034003094770014286 - training/rollout_probs_diff_std:0.009052536450326443 - training/rollout_actor_probs_pearson_corr:0.9994275569915771 - rollout_corr/rollout_is_mean:0.9999655485153198 - rollout_corr/rollout_is_ratio_fraction_high:4.338112375990022e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.7352449503960088e-05 - rollout_corr/rollout_is_max:2.0474398136138916 - rollout_corr/rollout_is_min:0.40364018082618713 - rollout_corr/rollout_is_std:0.029825152829289436 - rollout_corr/rollout_is_eff_sample_size:0.9991112508590329 - rollout_corr/rollout_is_seq_mean:0.9999865293502808 - rollout_corr/rollout_is_seq_std:0.0013986814301460981 - rollout_corr/rollout_is_seq_max:1.0060322284698486 - rollout_corr/rollout_is_seq_min:0.9942169785499573 - rollout_corr/rollout_is_seq_max_deviation:0.006032228469848633 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2896207650192082) - rollout_corr/training_log_ppl:np.float64(0.2305254971361137) - rollout_corr/kl:np.float64(0.0005066159342224097) - rollout_corr/k3_kl:np.float64(0.0004654821964606981) - rollout_corr/rollout_ppl:np.float64(1.2889360883273184) - rollout_corr/rollout_log_ppl:np.float64(0.23001887879217975) - rollout_corr/log_ppl_diff:np.float64(0.0005066183439339511) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010047785108326934) - rollout_corr/log_ppl_diff_max:np.float64(0.0074303895235061646) - rollout_corr/log_ppl_diff_min:np.float64(-0.004489123821258545) - rollout_corr/ppl_ratio:np.float64(1.0005076131783426) - rollout_corr/chi2_token:np.float64(0.000825623283162713) - rollout_corr/chi2_seq:np.float64(0.5253964632283896) - actor/pg_loss:np.float64(7.684435695409775e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010054797257907921) - actor/ppo_kl:np.float64(4.189955548028479e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08311776397749782) - perf/mfu/actor:np.float64(0.10069730946959708) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.47393417358398) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:4 - critic/score/mean:0.86328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.86328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017708174884319305 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017708174884319305 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:900.44921875 - response_length/max:6057.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:900.44921875 - response_length_non_aborted/max:6057.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.09375 - prompt_length/max:365.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011433474719524384 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8707099817693233) - timing_s/agent_loop/generate_sequences/max:np.float64(35.631088424474) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.754026362330478) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(35.631088424474) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:6057 - timing_s/gen:41.34765339829028 - timing_s/reward:0.08056613802909851 - timing_s/old_log_prob:2.6803389824926853 - timing_s/adv:0.13033771142363548 - timing_s/update_actor:10.983067074790597 - timing_s/step:55.307873111218214 - timing_s/stop_profile:0.00011896714568138123 - timing_per_token_ms/update_actor:0.03687238624886475 - timing_per_token_ms/gen:0.17937077152588887 - timing_per_token_ms/adv:0.000437570161930108 - perf/total_num_tokens:297867 - perf/time_per_step:55.307873111218214 - perf/throughput:673.2020760430919 (TaskRunner pid=2049544) Training Progress: 99%|█████████▉| 99/100 [1:19:20<00:49, 49.85s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) (TaskRunner pid=2049544) - Electric field $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2049544) - Distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2049544) - Coulomb's constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We use the formula for the electric field due to a point charge: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Solving for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90/actor (TaskRunner pid=2049544) step:100 - global_seqlen/min:17010 - global_seqlen/max:43433 - global_seqlen/minmax_diff:26423 - global_seqlen/balanced_min:31299 - global_seqlen/balanced_max:31491 - global_seqlen/mean:31419.0 - actor/entropy:0.37953582406044006 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.630595862865448 - training/rollout_probs_diff_mean:0.0036211644764989614 - training/rollout_probs_diff_std:0.00978316180408001 - training/rollout_actor_probs_pearson_corr:0.999374508857727 - rollout_corr/rollout_is_mean:1.0000176429748535 - rollout_corr/rollout_is_ratio_fraction_high:1.0963950444420334e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.644592521188315e-05 - rollout_corr/rollout_is_max:2.3022942543029785 - rollout_corr/rollout_is_min:0.3131349980831146 - rollout_corr/rollout_is_std:0.031683772802352905 - rollout_corr/rollout_is_eff_sample_size:0.998997264270168 - rollout_corr/rollout_is_seq_mean:1.0000271797180176 - rollout_corr/rollout_is_seq_std:0.0016033316496759653 - rollout_corr/rollout_is_seq_max:1.0072500705718994 - rollout_corr/rollout_is_seq_min:0.9959959983825684 - rollout_corr/rollout_is_seq_max_deviation:0.007250070571899414 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3458213401027024) - rollout_corr/training_log_ppl:np.float64(0.26966726542741526) - rollout_corr/kl:np.float64(0.00037315004849824973) - rollout_corr/k3_kl:np.float64(0.0005444729254122649) - rollout_corr/rollout_ppl:np.float64(1.3452776027843356) - rollout_corr/rollout_log_ppl:np.float64(0.2692941162531497) - rollout_corr/log_ppl_diff:np.float64(0.0003731491742655635) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012809229083359241) - rollout_corr/log_ppl_diff_max:np.float64(0.0049625784158706665) - rollout_corr/log_ppl_diff_min:np.float64(-0.0106421560049057) - rollout_corr/ppl_ratio:np.float64(1.0003747479058802) - rollout_corr/chi2_token:np.float64(0.0014326744712889194) - rollout_corr/chi2_seq:np.float64(1.9170411999803036) - actor/pg_loss:np.float64(-7.347844075411558e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001629320678375734) - actor/ppo_kl:np.float64(-0.00012778303023264215) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12031856318935752) - perf/mfu/actor:np.float64(0.0881824181802986) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.57986450195312) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6828125) - val-aux/sciknoweval/reward/std@16:np.float64(0.17979486326514935) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7532) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1476854115376502) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6133875) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15413287282151064) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6832499999999999) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.17928347494256552) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.807675) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11309093981217563) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5515125000000001) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12241577016075818) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7018125) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13581544160753714) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8526375000000002) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0888708326383256) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4995375) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08946281937139845) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.710175) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.08949529688241312) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8910875) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.061136881006381635) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.46395) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.054003469751280585) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7157250000000001) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.05584489583302406) - val-aux/sciknoweval/score/mean@16:np.float64(0.6828125) - val-aux/sciknoweval/score/std@16:np.float64(0.17979486326514935) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7532) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1476854115376502) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6133875) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15413287282151064) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6832499999999999) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.17928347494256552) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.807675) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11309093981217563) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5515125000000001) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12241577016075818) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7018125) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13581544160753714) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8526375000000002) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0888708326383256) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4995375) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08946281937139845) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.710175) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.08949529688241312) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8910875) - val-aux/sciknoweval/score/best@16/std:np.float64(0.061136881006381635) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.46395) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.054003469751280585) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7157250000000001) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.05584489583302406) - val-core/sciknoweval/acc/mean@16:np.float64(0.6828125) - val-aux/sciknoweval/acc/std@16:np.float64(0.17979486326514935) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7532) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1476854115376502) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6133875) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15413287282151064) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6832499999999999) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.17928347494256552) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.807675) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11309093981217563) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5515125000000001) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12241577016075818) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7018125) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13581544160753714) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8526375000000002) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0888708326383256) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4995375) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08946281937139845) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.710175) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.08949529688241312) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8910875) - val-core/sciknoweval/acc/best@16/std:np.float64(0.061136881006381635) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.46395) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.054003469751280585) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7157250000000001) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.05584489583302406) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9859375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.03383512807052093) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99745) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.014287439995541685) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9741250000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.042452979144002284) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9857875) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.03387729307844336) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9998374999999999) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.002749370152057837) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9560875) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.04764508939236297) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9946999999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.021483511676811038) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.933025) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0436722700672832) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9988624999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.009237722502874987) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9139250000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.029568343249240757) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998750000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0017543234207183663) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:4 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018172748386859894 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0018172748386859894 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:712.5625 - response_length/max:3652.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:712.5625 - response_length_non_aborted/max:3652.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.28125 - prompt_length/max:383.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.623674511909485e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2800081353634596) - timing_s/agent_loop/generate_sequences/max:np.float64(22.433935675770044) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.539074072250514) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.433935675770044) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:3652 - timing_s/gen:28.36895477026701 - timing_s/reward:0.07503514550626278 - timing_s/old_log_prob:2.573247427120805 - timing_s/adv:0.12070851400494576 - timing_s/update_actor:10.477075356990099 - timing_s/step:41.70409296452999 - timing_s/testing:119.00530271604657 - timing_s/save_checkpoint:6.902973407879472 - timing_s/stop_profile:0.00019830837845802307 - timing_per_token_ms/update_actor:0.041682880410699334 - timing_per_token_ms/gen:0.15551790835380125 - timing_per_token_ms/adv:0.00048023693467704956 - perf/total_num_tokens:251352 - perf/time_per_step:41.70409296452999 - perf/throughput:753.3792912537954 (TaskRunner pid=2049544) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2049544) "np.float64(0.6828125), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2049544) "np.float64(0.17979486326514935), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2049544) "np.float64(0.7532), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2049544) "np.float64(0.1476854115376502), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2049544) "np.float64(0.6133875), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2049544) "np.float64(0.15413287282151064), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2049544) "np.float64(0.6832499999999999), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2049544) "np.float64(0.17928347494256552), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2049544) "np.float64(0.807675), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2049544) "np.float64(0.11309093981217563), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2049544) "np.float64(0.5515125000000001), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2049544) "np.float64(0.12241577016075818), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2049544) (TaskRunner pid=2049544) Training Progress: 100%|██████████| 100/100 [1:22:07<00:00, 85.21s/it] (TaskRunner pid=2049544) "np.float64(0.7018125), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2049544) "np.float64(0.13581544160753714), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2049544) "np.float64(0.8526375000000002), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2049544) "np.float64(0.0888708326383256), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2049544) "np.float64(0.4995375), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2049544) "np.float64(0.08946281937139845), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2049544) "np.float64(0.710175), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2049544) "np.float64(0.08949529688241312), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "np.float64(0.8910875), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2049544) 'np.float64(0.061136881006381635), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/reward/worst@16/mean': np.float64(0.46395), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/reward/worst@16/std': np.float64(0.054003469751280585), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/reward/maj@16/mean': np.float64(0.7157250000000001), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/reward/maj@16/std': np.float64(0.05584489583302406), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/mean@16': np.float64(0.6828125), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/std@16': np.float64(0.17979486326514935), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@2/mean': np.float64(0.7532), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@2/std': np.float64(0.1476854115376502), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@2/mean': np.float64(0.6133875), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@2/std': np.float64(0.15413287282151064), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@2/mean': np.float64(0.6832499999999999), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@2/std': np.float64(0.17928347494256552), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@4/mean': np.float64(0.807675), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@4/std': np.float64(0.11309093981217563), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@4/mean': np.float64(0.5515125000000001), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@4/std': np.float64(0.12241577016075818), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@4/mean': np.float64(0.7018125), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@4/std': np.float64(0.13581544160753714), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@8/mean': np.float64(0.8526375000000002), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@8/std': np.float64(0.0888708326383256), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@8/mean': np.float64(0.4995375), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@8/std': np.float64(0.08946281937139845), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@8/mean': np.float64(0.710175), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@8/std': np.float64(0.08949529688241312), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@16/mean': np.float64(0.8910875), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@16/std': np.float64(0.061136881006381635), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@16/mean': np.float64(0.46395), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@16/std': np.float64(0.054003469751280585), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@16/mean': np.float64(0.7157250000000001), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@16/std': np.float64(0.05584489583302406), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-core/sciknoweval/acc/mean@16': np.float64(0.6828125), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/std@16': np.float64(0.17979486326514935), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@2/mean': np.float64(0.7532), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@2/std': np.float64(0.1476854115376502), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@2/mean': np.float64(0.6133875), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@2/std': np.float64(0.15413287282151064), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@2/mean': np.float64(0.6832499999999999), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@2/std': np.float64(0.17928347494256552), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@4/mean': np.float64(0.807675), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@4/std': np.float64(0.11309093981217563), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@4/mean': np.float64(0.5515125000000001), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@4/std': np.float64(0.12241577016075818), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@4/mean': np.float64(0.7018125), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@4/std': np.float64(0.13581544160753714), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@8/mean': np.float64(0.8526375000000002), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@8/std': np.float64(0.0888708326383256), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@8/mean': np.float64(0.4995375), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@8/std': np.float64(0.08946281937139845), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@8/mean': np.float64(0.710175), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@8/std': np.float64(0.08949529688241312), " (TaskRunner pid=2049544) "'val-core/sciknoweval/acc/best@16/mean': np.float64(0.8910875), " (TaskRunner pid=2049544) "'val-core/sciknoweval/acc/best@16/std': np.float64(0.061136881006381635), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@16/mean': np.float64(0.46395), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@16/std': np.float64(0.054003469751280585), " (TaskRunner pid=2049544) "'val-core/sciknoweval/acc/maj@16/mean': np.float64(0.7157250000000001), " (TaskRunner pid=2049544) "'val-core/sciknoweval/acc/maj@16/std': np.float64(0.05584489583302406), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/mean@16': np.float64(0.9859375), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2049544) 'np.float64(0.03383512807052093), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@2/mean': np.float64(0.99745), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2049544) 'np.float64(0.014287439995541685), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': " (TaskRunner pid=2049544) 'np.float64(0.9741250000000001), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2049544) 'np.float64(0.042452979144002284), ' (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(0.9857875), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2049544) 'np.float64(0.03387729307844336), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@4/mean': " (TaskRunner pid=2049544) 'np.float64(0.9998374999999999), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@4/std': " (TaskRunner pid=2049544) 'np.float64(0.002749370152057837), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': np.float64(0.9560875), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2049544) 'np.float64(0.04764508939236297), ' (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': " (TaskRunner pid=2049544) 'np.float64(0.9946999999999999), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2049544) 'np.float64(0.021483511676811038), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': np.float64(0.933025), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2049544) (TaskRunner pid=2049544) 'np.float64(0.0436722700672832), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': " (TaskRunner pid=2049544) 'np.float64(0.9988624999999999), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@8/std': " (TaskRunner pid=2049544) 'np.float64(0.009237722502874987), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': " (TaskRunner pid=2049544) (TaskRunner pid=2049544) 'np.float64(0.9139250000000001), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2049544) 'np.float64(0.029568343249240757), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': " (TaskRunner pid=2049544) 'np.float64(0.9998750000000001), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@16/std': " (TaskRunner pid=2049544) "np.float64(0.0017543234207183663), 'val-aux/num_turns/min': np.int32(2), " (TaskRunner pid=2049544) "'val-aux/num_turns/max': np.int32(2), 'val-aux/num_turns/mean': " (TaskRunner pid=2049544) 'np.float64(2.0)}') (TaskRunner pid=2049544) Training Progress: 100%|██████████| 100/100 [1:22:07<00:00, 49.28s/it] (TaskRunner pid=2049544) (TaskRunner pid=2049544) wandb: updating run metadata (TaskRunner pid=2049544) wandb: uploading output.log; uploading wandb-summary.json (TaskRunner pid=2049544) wandb: uploading output.log; uploading config.yaml (TaskRunner pid=2049544) wandb: uploading history steps 98-99, summary, console lines 919-1044 (TaskRunner pid=2049544) wandb: (TaskRunner pid=2049544) wandb: Run history: (TaskRunner pid=2049544) wandb: actor/entropy ▁▁▂▁▂▁▁▁▂▃▃▂▂▂▃▂▃▃▃▃▄▄▃▄▅▄▆▄▇▆▅▄▆▆▆▅▇█▇▇ (TaskRunner pid=2049544) wandb: actor/grad_norm █▄▇▃█▆▃▇▆▁▄▄▃▃▃▂▁▂▁▂▄▃▂▂▂▂▄▂▃▃▂▃▂▃▂▁▃▂▂▁ (TaskRunner pid=2049544) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2049544) wandb: actor/lr ▁▂▃▄▆███████████████████████████████████ (TaskRunner pid=2049544) wandb: actor/pg_clipfrac ▂▄▅▃▆▇▄▃█▃▃▄▂▂▃▃▂▂▂▂▃▃▃▂▂▁▁▁▁▂▁▁▁▂▂▂▁▁▂▁ (TaskRunner pid=2049544) wandb: actor/pg_clipfrac_lower ▁▁▁▁▁▁▆▁▁▄▁▃▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂▁▁▁ (TaskRunner pid=2049544) wandb: actor/pg_loss ▆▂▅▅█▆▅▃▃▅▅▄▃▄▅▄▄▃▂▅▁▂▄▄▁▄▆▃▅▄▅▅▂▅▅▄▅▃▅▃ (TaskRunner pid=2049544) wandb: actor/ppo_kl ▁▂▂▃▄▅▃▁▄▃▄▂▄▁▂▂▃▃▂▂▇▄▂▃▂▂▄▄▃▁▄▃▁▂█▃▃▂▃▂ (TaskRunner pid=2049544) wandb: critic/advantages/max █████████▅█████████████▅███▁████████████ (TaskRunner pid=2049544) wandb: critic/advantages/mean ▅▇▆▅▂▄▃▅▅▅▃▂▃▆▃▇▆▂█▂▄▅▃▃▅▁▅▄▄▄▄▅▄▆▆▅▄▃▂▅ (TaskRunner pid=2049544) wandb: +204 ... (TaskRunner pid=2049544) wandb: (TaskRunner pid=2049544) wandb: Run summary: (TaskRunner pid=2049544) wandb: actor/entropy 0.37954 (TaskRunner pid=2049544) wandb: actor/grad_norm 0.12032 (TaskRunner pid=2049544) wandb: actor/kl_loss 0 (TaskRunner pid=2049544) wandb: actor/lr 0.0 (TaskRunner pid=2049544) wandb: actor/pg_clipfrac 0.00016 (TaskRunner pid=2049544) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2049544) wandb: actor/pg_loss -7e-05 (TaskRunner pid=2049544) wandb: actor/ppo_kl -0.00013 (TaskRunner pid=2049544) wandb: critic/advantages/max 0.875 (TaskRunner pid=2049544) wandb: critic/advantages/mean -0.00182 (TaskRunner pid=2049544) wandb: +204 ... (TaskRunner pid=2049544) wandb: (TaskRunner pid=2049544) wandb: 🚀 View run qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/o8ivivjg (TaskRunner pid=2049544) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2049544) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2049544) wandb: Find logs at: ./wandb/run-20260702_073403-o8ivivjg/logs main_ppo exit code: 0 [2026-07-02 08:56:17] Finished physics grpo [2026-07-02 08:56:17] Starting physics rlsd_tr Experiment: qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Dataset: physics Method: rlsd_tr Config: rlsd Model: Qwen/Qwen3-4B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet Ray tmp: /tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B 2026-07-02 08:56:19,506 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Config: rlsd Task: datasets/sciknoweval/physics Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-RLSD-TR-GRPO-matched-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/physics +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.model.use_fused_kernels=False actor_rollout_ref.actor.policy_loss.loss_mode=rlsd actor_rollout_ref.actor.self_distillation.token_reweight_lambda=0.5 actor_rollout_ref.actor.self_distillation.token_reweight_eps_w=0.2 actor_rollout_ref.actor.self_distillation.token_reweight_decay_steps=0 actor_rollout_ref.actor.self_distillation.teacher_regularization=trust-region actor_rollout_ref.actor.self_distillation.teacher_update_rate=0.1 actor_rollout_ref.actor.self_distillation.max_reprompt_len=10240 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/physics', 'EXPERIMENT': 'qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 08:56:29,254 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2074004) TaskRunner hostname: mole-workspace-rw, PID: 2074004 (TaskRunner pid=2074004) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2074004) 'calculate_entropy': False, (TaskRunner pid=2074004) 'calculate_sum_pi_squared': False, (TaskRunner pid=2074004) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2074004) 'async_save': False, (TaskRunner pid=2074004) 'load_contents': ['model', (TaskRunner pid=2074004) 'optimizer', (TaskRunner pid=2074004) 'extra'], (TaskRunner pid=2074004) 'save_contents': ['model', (TaskRunner pid=2074004) 'optimizer', (TaskRunner pid=2074004) 'extra']}, (TaskRunner pid=2074004) 'clip_ratio': 0.2, (TaskRunner pid=2074004) 'clip_ratio_c': 3.0, (TaskRunner pid=2074004) 'clip_ratio_high': 0.28, (TaskRunner pid=2074004) 'clip_ratio_low': 0.2, (TaskRunner pid=2074004) 'data_loader_seed': 42, (TaskRunner pid=2074004) 'entropy_checkpointing': False, (TaskRunner pid=2074004) 'entropy_coeff': 0, (TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2074004) 'freeze_vision_tower': False, (TaskRunner pid=2074004) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2074004) 'dtype': 'bfloat16', (TaskRunner pid=2074004) 'entropy_checkpointing': False, (TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2074004) 'forward_only': False, (TaskRunner pid=2074004) 'forward_prefetch': False, (TaskRunner pid=2074004) 'fsdp_size': -1, (TaskRunner pid=2074004) 'full_determinism': False, (TaskRunner pid=2074004) 'model_dtype': 'fp32', (TaskRunner pid=2074004) 'offload_policy': False, (TaskRunner pid=2074004) 'optimizer_offload': False, (TaskRunner pid=2074004) 'param_offload': False, (TaskRunner pid=2074004) 'reshard_after_forward': True, (TaskRunner pid=2074004) 'seed': 42, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_orig_params': False, (TaskRunner pid=2074004) 'use_torch_compile': True, (TaskRunner pid=2074004) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2074004) 'grad_clip': 1.0, (TaskRunner pid=2074004) 'kl_loss_coef': 0.001, (TaskRunner pid=2074004) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2074004) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2074004) 'loss_scale_factor': None, (TaskRunner pid=2074004) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2074004) 'betas': [0.9, 0.999], (TaskRunner pid=2074004) 'clip_grad': 1.0, (TaskRunner pid=2074004) 'lr': 1e-06, (TaskRunner pid=2074004) 'lr_scheduler_type': 'constant', (TaskRunner pid=2074004) 'lr_warmup_steps': 10, (TaskRunner pid=2074004) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2074004) 'min_lr_ratio': 0.0, (TaskRunner pid=2074004) 'num_cycles': 0.5, (TaskRunner pid=2074004) 'optimizer': 'AdamW', (TaskRunner pid=2074004) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2074004) 'override_optimizer_config': None, (TaskRunner pid=2074004) 'total_training_steps': -1, (TaskRunner pid=2074004) 'warmup_style': None, (TaskRunner pid=2074004) 'weight_decay': 0.01}, (TaskRunner pid=2074004) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2074004) 'clip_cov_lb': 1.0, (TaskRunner pid=2074004) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2074004) 'clip_cov_ub': 5.0, (TaskRunner pid=2074004) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2074004) 'loss_mode': 'rlsd', (TaskRunner pid=2074004) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2074004) 'ppo_epochs': 1, (TaskRunner pid=2074004) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2074004) 'ppo_micro_batch_size': None, (TaskRunner pid=2074004) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2074004) 'ppo_mini_batch_size': 8, (TaskRunner pid=2074004) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'all_ranks': False, (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'ranks': [], (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'tool': None, (TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2074004) 'analysis': True, (TaskRunner pid=2074004) 'contents': [], (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'level': 'level0'}, (TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'discrete': False}, (TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2074004) 'step_end': None, (TaskRunner pid=2074004) 'step_start': 0}, (TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2074004) 'rollout_n': 8, (TaskRunner pid=2074004) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2074004) 'mode': 'disabled', (TaskRunner pid=2074004) 'record_file': None, (TaskRunner pid=2074004) 'replay_file': None}, (TaskRunner pid=2074004) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2074004) 'alpha': 0.5, (TaskRunner pid=2074004) 'distillation_add_tail': True, (TaskRunner pid=2074004) 'distillation_topk': 100, (TaskRunner pid=2074004) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2074004) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2074004) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'loss_weight': 0.0, (TaskRunner pid=2074004) 'mask': 'all'}, (TaskRunner pid=2074004) 'feedback_template': '\n' (TaskRunner pid=2074004) 'The ' (TaskRunner pid=2074004) 'following ' (TaskRunner pid=2074004) 'is ' (TaskRunner pid=2074004) 'feedback ' (TaskRunner pid=2074004) 'from ' (TaskRunner pid=2074004) 'your ' (TaskRunner pid=2074004) 'unsuccessful ' (TaskRunner pid=2074004) 'earlier ' (TaskRunner pid=2074004) 'attempt:\n' (TaskRunner pid=2074004) '\n' (TaskRunner pid=2074004) '{feedback_raw}', (TaskRunner pid=2074004) 'full_logit_distillation': True, (TaskRunner pid=2074004) 'include_environment_feedback': True, (TaskRunner pid=2074004) 'is_clip': 2, (TaskRunner pid=2074004) 'max_reprompt_len': 10240, (TaskRunner pid=2074004) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2074004) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2074004) '\n' (TaskRunner pid=2074004) 'Correctly ' (TaskRunner pid=2074004) 'solve ' (TaskRunner pid=2074004) 'the ' (TaskRunner pid=2074004) 'original ' (TaskRunner pid=2074004) 'question.', (TaskRunner pid=2074004) 'reprompt_truncation': 'right', (TaskRunner pid=2074004) 'solution_template': '\n' (TaskRunner pid=2074004) 'Correct ' (TaskRunner pid=2074004) 'solution:\n' (TaskRunner pid=2074004) '\n' (TaskRunner pid=2074004) '{successful_previous_attempt}', (TaskRunner pid=2074004) 'srpo_dynamic_weighting': False, (TaskRunner pid=2074004) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2074004) 'success_reward_threshold': 0.5, (TaskRunner pid=2074004) 'teacher_regularization': 'trust-region', (TaskRunner pid=2074004) 'teacher_update_rate': 0.1, (TaskRunner pid=2074004) 'token_reweight_decay_steps': 0, (TaskRunner pid=2074004) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2074004) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2074004) 'shuffle': False, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2074004) 'tau_neg': 1.05, (TaskRunner pid=2074004) 'tau_pos': 1.0, (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_dynamic_bsz': False, (TaskRunner pid=2074004) 'use_fused_kernels': False, (TaskRunner pid=2074004) 'use_kl_loss': False, (TaskRunner pid=2074004) 'use_prefix_grouper': False, (TaskRunner pid=2074004) 'use_remove_padding': True, (TaskRunner pid=2074004) 'use_torch_compile': True}, (TaskRunner pid=2074004) 'hybrid_engine': True, (TaskRunner pid=2074004) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2074004) 'custom_chat_template': None, (TaskRunner pid=2074004) 'enable_activation_offload': False, (TaskRunner pid=2074004) 'enable_gradient_checkpointing': True, (TaskRunner pid=2074004) 'exclude_modules': None, (TaskRunner pid=2074004) 'external_lib': None, (TaskRunner pid=2074004) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2074004) 'hf_config_path': None, (TaskRunner pid=2074004) 'lora_adapter_path': None, (TaskRunner pid=2074004) 'lora_alpha': 16, (TaskRunner pid=2074004) 'lora_rank': 0, (TaskRunner pid=2074004) 'override_config': {}, (TaskRunner pid=2074004) 'path': 'Qwen/Qwen3-4B', (TaskRunner pid=2074004) 'target_modules': 'all-linear', (TaskRunner pid=2074004) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2074004) 'num_shards': 4}, (TaskRunner pid=2074004) 'tokenizer_path': None, (TaskRunner pid=2074004) 'trust_remote_code': True, (TaskRunner pid=2074004) 'use_fused_kernels': False, (TaskRunner pid=2074004) 'use_liger': False, (TaskRunner pid=2074004) 'use_remove_padding': True, (TaskRunner pid=2074004) 'use_shm': False}, (TaskRunner pid=2074004) 'nccl_timeout': 600, (TaskRunner pid=2074004) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2074004) 'entropy_checkpointing': False, (TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2074004) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2074004) 'dtype': 'bfloat16', (TaskRunner pid=2074004) 'entropy_checkpointing': False, (TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2074004) 'forward_only': True, (TaskRunner pid=2074004) 'forward_prefetch': False, (TaskRunner pid=2074004) 'fsdp_size': -1, (TaskRunner pid=2074004) 'full_determinism': False, (TaskRunner pid=2074004) 'model_dtype': 'fp32', (TaskRunner pid=2074004) 'offload_policy': False, (TaskRunner pid=2074004) 'optimizer_offload': False, (TaskRunner pid=2074004) 'param_offload': False, (TaskRunner pid=2074004) 'reshard_after_forward': True, (TaskRunner pid=2074004) 'seed': 42, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_orig_params': False, (TaskRunner pid=2074004) 'use_torch_compile': True, (TaskRunner pid=2074004) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2074004) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2074004) 'log_prob_micro_batch_size': None, (TaskRunner pid=2074004) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2074004) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2074004) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'all_ranks': False, (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'ranks': [], (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'tool': None, (TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2074004) 'analysis': True, (TaskRunner pid=2074004) 'contents': [], (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'level': 'level0'}, (TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'discrete': False}, (TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2074004) 'step_end': None, (TaskRunner pid=2074004) 'step_start': 0}, (TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2074004) 'rollout_n': 8, (TaskRunner pid=2074004) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2074004) 'mode': 'disabled', (TaskRunner pid=2074004) 'record_file': None, (TaskRunner pid=2074004) 'replay_file': None}, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_torch_compile': True}, (TaskRunner pid=2074004) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2074004) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2074004) 'agent_loop_config_path': None, (TaskRunner pid=2074004) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2074004) 'name': None, (TaskRunner pid=2074004) 'path': None}, (TaskRunner pid=2074004) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2074004) 'num_workers': 8}, (TaskRunner pid=2074004) 'calculate_log_probs': True, (TaskRunner pid=2074004) 'cudagraph_capture_sizes': None, (TaskRunner pid=2074004) 'data_parallel_size': 1, (TaskRunner pid=2074004) 'disable_log_stats': True, (TaskRunner pid=2074004) 'do_sample': True, (TaskRunner pid=2074004) 'dtype': 'bfloat16', (TaskRunner pid=2074004) 'enable_chunked_prefill': True, (TaskRunner pid=2074004) 'enable_prefix_caching': True, (TaskRunner pid=2074004) 'enable_rollout_routing_replay': False, (TaskRunner pid=2074004) 'enforce_eager': False, (TaskRunner pid=2074004) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2074004) 'expert_parallel_size': 1, (TaskRunner pid=2074004) 'free_cache_engine': True, (TaskRunner pid=2074004) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2074004) 'ignore_eos': False, (TaskRunner pid=2074004) 'layered_summon': False, (TaskRunner pid=2074004) 'load_format': 'dummy', (TaskRunner pid=2074004) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2074004) 'log_prob_micro_batch_size': None, (TaskRunner pid=2074004) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2074004) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2074004) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2074004) 'max_model_len': 10240, (TaskRunner pid=2074004) 'max_num_batched_tokens': 10240, (TaskRunner pid=2074004) 'max_num_seqs': 1024, (TaskRunner pid=2074004) 'mode': 'async', (TaskRunner pid=2074004) 'multi_stage_wake_up': False, (TaskRunner pid=2074004) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'format': 'hermes', (TaskRunner pid=2074004) 'interaction_config_path': None, (TaskRunner pid=2074004) 'max_assistant_turns': None, (TaskRunner pid=2074004) 'max_parallel_calls': 1, (TaskRunner pid=2074004) 'max_tool_response_length': 256, (TaskRunner pid=2074004) 'max_user_turns': None, (TaskRunner pid=2074004) 'num_repeat_rollouts': None, (TaskRunner pid=2074004) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2074004) 'tool_config_path': None, (TaskRunner pid=2074004) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2074004) 'use_inference_chat_template': False}, (TaskRunner pid=2074004) 'n': 8, (TaskRunner pid=2074004) 'name': 'vllm', (TaskRunner pid=2074004) 'over_sample_rate': 0, (TaskRunner pid=2074004) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2074004) 'profiler': (TaskRunner pid=2074004) {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'all_ranks': False, (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'ranks': [], (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'tool': None, (TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2074004) 'analysis': True, (TaskRunner pid=2074004) 'contents': [], (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'level': 'level0'}, (TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'discrete': False}, (TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2074004) 'step_end': None, (TaskRunner pid=2074004) 'step_start': 0}, (TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2074004) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2074004) 'port': 9090, (TaskRunner pid=2074004) 'served_model_name': 'Qwen/Qwen3-4B'}, (TaskRunner pid=2074004) 'prompt_length': 2048, (TaskRunner pid=2074004) 'quantization': None, (TaskRunner pid=2074004) 'quantization_config_file': None, (TaskRunner pid=2074004) 'response_length': 8192, (TaskRunner pid=2074004) 'scheduling_policy': 'fcfs', (TaskRunner pid=2074004) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2074004) 'skip_rollout': False, (TaskRunner pid=2074004) 'skip_tokenizer_init': True, (TaskRunner pid=2074004) 'temperature': 1.0, (TaskRunner pid=2074004) 'tensor_model_parallel_size': 2, (TaskRunner pid=2074004) 'top_k': -1, (TaskRunner pid=2074004) 'top_p': 1.0, (TaskRunner pid=2074004) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2074004) 'backend': None, (TaskRunner pid=2074004) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2074004) 'token2text': False}, (TaskRunner pid=2074004) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2074004) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2074004) 'do_sample': True, (TaskRunner pid=2074004) 'n': 16, (TaskRunner pid=2074004) 'temperature': 0.6, (TaskRunner pid=2074004) 'top_k': -1, (TaskRunner pid=2074004) 'top_p': 0.95}}}, (TaskRunner pid=2074004) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2074004) 'adv_estimator': 'grpo', (TaskRunner pid=2074004) 'gamma': 1.0, (TaskRunner pid=2074004) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2074004) 'horizon': 10000, (TaskRunner pid=2074004) 'kl_coef': 0.001, (TaskRunner pid=2074004) 'target_kl': 0.1, (TaskRunner pid=2074004) 'type': 'fixed'}, (TaskRunner pid=2074004) 'kl_penalty': 'kl', (TaskRunner pid=2074004) 'lam': 1.0, (TaskRunner pid=2074004) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2074004) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2074004) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2074004) 'loss_type': 'ppo_clip', (TaskRunner pid=2074004) 'rollout_is': 'token', (TaskRunner pid=2074004) 'rollout_is_batch_normalize': False, (TaskRunner pid=2074004) 'rollout_is_threshold': 2.0, (TaskRunner pid=2074004) 'rollout_rs': None, (TaskRunner pid=2074004) 'rollout_rs_threshold': None}, (TaskRunner pid=2074004) 'use_kl_in_reward': False, (TaskRunner pid=2074004) 'use_pf_ppo': False}, (TaskRunner pid=2074004) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2074004) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2074004) 'async_save': False, (TaskRunner pid=2074004) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2074004) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2074004) 'cliprange_value': 0.5, (TaskRunner pid=2074004) 'data_loader_seed': 42, (TaskRunner pid=2074004) 'enable': None, (TaskRunner pid=2074004) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2074004) 'forward_micro_batch_size': None, (TaskRunner pid=2074004) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2074004) 'grad_clip': 1.0, (TaskRunner pid=2074004) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2074004) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2074004) 'enable_activation_offload': False, (TaskRunner pid=2074004) 'enable_gradient_checkpointing': True, (TaskRunner pid=2074004) 'external_lib': None, (TaskRunner pid=2074004) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2074004) 'dtype': 'bfloat16', (TaskRunner pid=2074004) 'entropy_checkpointing': False, (TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2074004) 'forward_only': False, (TaskRunner pid=2074004) 'forward_prefetch': False, (TaskRunner pid=2074004) 'fsdp_size': -1, (TaskRunner pid=2074004) 'full_determinism': False, (TaskRunner pid=2074004) 'model_dtype': 'fp32', (TaskRunner pid=2074004) 'offload_policy': False, (TaskRunner pid=2074004) 'optimizer_offload': False, (TaskRunner pid=2074004) 'param_offload': False, (TaskRunner pid=2074004) 'reshard_after_forward': True, (TaskRunner pid=2074004) 'seed': 42, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_orig_params': False, (TaskRunner pid=2074004) 'use_torch_compile': True, (TaskRunner pid=2074004) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2074004) 'lora_alpha': 16, (TaskRunner pid=2074004) 'lora_rank': 0, (TaskRunner pid=2074004) 'override_config': {}, (TaskRunner pid=2074004) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2074004) 'target_modules': 'all-linear', (TaskRunner pid=2074004) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2074004) 'tokenizer_path': 'Qwen/Qwen3-4B', (TaskRunner pid=2074004) 'trust_remote_code': True, (TaskRunner pid=2074004) 'use_remove_padding': False, (TaskRunner pid=2074004) 'use_shm': False}, (TaskRunner pid=2074004) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2074004) 'betas': [0.9, 0.999], (TaskRunner pid=2074004) 'clip_grad': 1.0, (TaskRunner pid=2074004) 'lr': 1e-05, (TaskRunner pid=2074004) 'lr_scheduler_type': 'constant', (TaskRunner pid=2074004) 'lr_warmup_steps': -1, (TaskRunner pid=2074004) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2074004) 'min_lr_ratio': 0.0, (TaskRunner pid=2074004) 'num_cycles': 0.5, (TaskRunner pid=2074004) 'optimizer': 'AdamW', (TaskRunner pid=2074004) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2074004) 'override_optimizer_config': None, (TaskRunner pid=2074004) 'total_training_steps': -1, (TaskRunner pid=2074004) 'warmup_style': None, (TaskRunner pid=2074004) 'weight_decay': 0.01}, (TaskRunner pid=2074004) 'ppo_epochs': 1, (TaskRunner pid=2074004) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2074004) 'ppo_micro_batch_size': None, (TaskRunner pid=2074004) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2074004) 'ppo_mini_batch_size': 8, (TaskRunner pid=2074004) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'all_ranks': False, (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'ranks': [], (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'tool': None, (TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2074004) 'analysis': True, (TaskRunner pid=2074004) 'contents': [], (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'level': 'level0'}, (TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'discrete': False}, (TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2074004) 'step_end': None, (TaskRunner pid=2074004) 'step_start': 0}, (TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2074004) 'rollout_n': 8, (TaskRunner pid=2074004) 'shuffle': False, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_dynamic_bsz': False}, (TaskRunner pid=2074004) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2074004) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2074004) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2074004) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2074004) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2074004) 'dataloader_num_workers': 8, (TaskRunner pid=2074004) 'filter_overlong_prompts': True, (TaskRunner pid=2074004) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2074004) 'image_key': 'images', (TaskRunner pid=2074004) 'image_patch_size': 14, (TaskRunner pid=2074004) 'max_prompt_length': 2048, (TaskRunner pid=2074004) 'max_response_length': 8192, (TaskRunner pid=2074004) 'prompt_key': 'prompt', (TaskRunner pid=2074004) 'return_full_prompt': False, (TaskRunner pid=2074004) 'return_multi_modal_inputs': True, (TaskRunner pid=2074004) 'return_raw_chat': True, (TaskRunner pid=2074004) 'return_raw_input_ids': False, (TaskRunner pid=2074004) 'reward_fn_key': 'data_source', (TaskRunner pid=2074004) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2074004) 'seed': None, (TaskRunner pid=2074004) 'shuffle': True, (TaskRunner pid=2074004) 'tokenizer': None, (TaskRunner pid=2074004) 'tool_config_path': None, (TaskRunner pid=2074004) 'train_batch_size': 32, (TaskRunner pid=2074004) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet'], (TaskRunner pid=2074004) 'train_max_samples': 3200, (TaskRunner pid=2074004) 'truncation': 'error', (TaskRunner pid=2074004) 'trust_remote_code': True, (TaskRunner pid=2074004) 'use_shm': False, (TaskRunner pid=2074004) 'val_batch_size': None, (TaskRunner pid=2074004) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet'], (TaskRunner pid=2074004) 'val_max_samples': -1, (TaskRunner pid=2074004) 'validation_shuffle': False, (TaskRunner pid=2074004) 'video_key': 'videos'}, (TaskRunner pid=2074004) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2074004) 'cuda-memory-usage': 'true', (TaskRunner pid=2074004) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2074004) 'capture-range-end': None, (TaskRunner pid=2074004) 'cuda-graph-trace': 'graph', (TaskRunner pid=2074004) 'cuda-memory-usage': 'true', (TaskRunner pid=2074004) 'kill': 'none', (TaskRunner pid=2074004) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2074004) 'torch_memory': {'context': 'all', (TaskRunner pid=2074004) 'kw_args': {}, (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'stacks': 'all', (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2074004) 'profile_continuous_steps': False, (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'steps': None, (TaskRunner pid=2074004) 'tool': None}, (TaskRunner pid=2074004) 'max_model_len': 10240, (TaskRunner pid=2074004) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B', (TaskRunner pid=2074004) 'include_dashboard': False, (TaskRunner pid=2074004) 'num_cpus': None, (TaskRunner pid=2074004) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2074004) 'TASK': 'datasets/sciknoweval/physics', (TaskRunner pid=2074004) 'USER': 'root'}}}, (TaskRunner pid=2074004) 'timeline_json_file': None}, (TaskRunner pid=2074004) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2074004) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2074004) 'name': 'custom_reward_manager', (TaskRunner pid=2074004) 'path': None}, (TaskRunner pid=2074004) 'name': 'naive', (TaskRunner pid=2074004) 'source': 'register'}, (TaskRunner pid=2074004) 'reward_model': {'enable': False, (TaskRunner pid=2074004) 'enable_resource_pool': False, (TaskRunner pid=2074004) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2074004) 'launch_reward_fn_async': False, (TaskRunner pid=2074004) 'max_length': None, (TaskRunner pid=2074004) 'micro_batch_size': None, (TaskRunner pid=2074004) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2074004) 'model': {'external_lib': None, (TaskRunner pid=2074004) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2074004) 'forward_prefetch': False, (TaskRunner pid=2074004) 'fsdp_size': -1, (TaskRunner pid=2074004) 'param_offload': False, (TaskRunner pid=2074004) 'reshard_after_forward': True, (TaskRunner pid=2074004) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2074004) 'input_tokenizer': 'Qwen/Qwen3-4B', (TaskRunner pid=2074004) 'override_config': {}, (TaskRunner pid=2074004) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2074004) 'trust_remote_code': False, (TaskRunner pid=2074004) 'use_fused_kernels': False, (TaskRunner pid=2074004) 'use_remove_padding': False, (TaskRunner pid=2074004) 'use_shm': False}, (TaskRunner pid=2074004) 'n_gpus_per_node': 8, (TaskRunner pid=2074004) 'nnodes': 0, (TaskRunner pid=2074004) 'num_workers': 1, (TaskRunner pid=2074004) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'all_ranks': False, (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'ranks': [], (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'tool': None, (TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2074004) 'analysis': True, (TaskRunner pid=2074004) 'contents': [], (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'level': 'level0'}, (TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'discrete': False}, (TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2074004) 'step_end': None, (TaskRunner pid=2074004) 'step_start': 0}, (TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2074004) 'reward_loop_class_name': None, (TaskRunner pid=2074004) 'reward_loop_module_path': None, (TaskRunner pid=2074004) 'reward_loop_source': 'register', (TaskRunner pid=2074004) 'reward_manager': 'naive', (TaskRunner pid=2074004) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2074004) 'cudagraph_capture_sizes': None, (TaskRunner pid=2074004) 'data_parallel_size': 1, (TaskRunner pid=2074004) 'disable_log_stats': True, (TaskRunner pid=2074004) 'dtype': 'bfloat16', (TaskRunner pid=2074004) 'enable_chunked_prefill': True, (TaskRunner pid=2074004) 'enable_prefix_caching': True, (TaskRunner pid=2074004) 'enforce_eager': True, (TaskRunner pid=2074004) 'engine_kwargs': {}, (TaskRunner pid=2074004) 'expert_parallel_size': 1, (TaskRunner pid=2074004) 'free_cache_engine': True, (TaskRunner pid=2074004) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2074004) 'limit_images': None, (TaskRunner pid=2074004) 'load_format': 'auto', (TaskRunner pid=2074004) 'max_model_len': None, (TaskRunner pid=2074004) 'max_num_batched_tokens': 8192, (TaskRunner pid=2074004) 'max_num_seqs': 1024, (TaskRunner pid=2074004) 'name': '???', (TaskRunner pid=2074004) 'prompt_length': 2048, (TaskRunner pid=2074004) 'response_length': 2048, (TaskRunner pid=2074004) 'skip_tokenizer_init': False, (TaskRunner pid=2074004) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2074004) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2074004) 'memory_limit_mb': 1024, (TaskRunner pid=2074004) 'url': None}, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_dynamic_bsz': False, (TaskRunner pid=2074004) 'use_reward_loop': False}, (TaskRunner pid=2074004) 'trainer': {'balance_batch': True, (TaskRunner pid=2074004) 'critic_warmup': 0, (TaskRunner pid=2074004) 'default_hdfs_dir': None, (TaskRunner pid=2074004) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2074004) 'del_local_ckpt_after_load': False, (TaskRunner pid=2074004) 'device': 'cuda', (TaskRunner pid=2074004) 'esi_redundant_time': 0, (TaskRunner pid=2074004) 'experiment_name': 'qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2074004) 'group_name': 'QWEN3-RLSD-TR-GRPO-matched-generalization', (TaskRunner pid=2074004) 'log_val_generations': 0, (TaskRunner pid=2074004) 'logger': ['console', 'wandb'], (TaskRunner pid=2074004) 'max_actor_ckpt_to_keep': 1, (TaskRunner pid=2074004) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2074004) 'n_gpus_per_node': 8, (TaskRunner pid=2074004) 'nnodes': 1, (TaskRunner pid=2074004) 'project_name': 'SDPO-root', (TaskRunner pid=2074004) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2074004) 'resume_from_path': None, (TaskRunner pid=2074004) 'resume_mode': 'auto', (TaskRunner pid=2074004) 'rollout_data_dir': None, (TaskRunner pid=2074004) 'save_freq': 10, (TaskRunner pid=2074004) 'test_freq': 10, (TaskRunner pid=2074004) 'total_epochs': 30, (TaskRunner pid=2074004) 'total_training_steps': 100, (TaskRunner pid=2074004) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2074004) 'val_before_train': False, (TaskRunner pid=2074004) 'val_only': False, (TaskRunner pid=2074004) 'validation_data_dir': None}, (TaskRunner pid=2074004) 'transfer_queue': {'enable': False}, (TaskRunner pid=2074004) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/physics', (TaskRunner pid=2074004) 'dir': '/users/root/SDPO', (TaskRunner pid=2074004) 'log_dir': '/users/root/output', (TaskRunner pid=2074004) 'task': 'datasets/sciknoweval/physics'}} (TaskRunner pid=2074004) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2074004) use_critic=need_critic(config), (TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2074004) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2074004) Using dataset class: RLHFDataset (TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2074004) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2074004) dataset len: 720 (TaskRunner pid=2074004) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2074004) WARNING:2026-07-02 08:56:38,173:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2074004) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/720 [00:00 (TaskRunner pid=2074004) bind role actor_rollout_ref method chat_completion to class .WorkerDict'> (TaskRunner pid=2074004) bind role actor_rollout_ref method generate to class .WorkerDict'> (TaskRunner pid=2074004) bind role actor_rollout_ref method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2074004) bind role actor_rollout_ref method sleep to class .WorkerDict'> (TaskRunner pid=2074004) bind role actor_rollout_ref method wake_up to class .WorkerDict'> (TaskRunner pid=2074004) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 80/80 [00:00<00:00, 128.23 examples/s] (TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2074004) self.use_critic = need_critic(self.config) (WorkerDict pid=2074388) [W702 08:56:47.781903761 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:56733 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2074385) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2074386) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2074386) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2074387) [W702 08:56:47.990315263 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:56733 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2074386) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2074386) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2074386) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2074391) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2074385) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2074385) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2074385) Actor use_remove_padding=True (WorkerDict pid=2074385) Actor use_fused_kernels=False (WorkerDict pid=2074385) Actor use_prefix_grouper=False (WorkerDict pid=2074385) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2074385) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2074385) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2074385) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2074387) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.99s/it] [repeated 12x across cluster] (WorkerDict pid=2074385) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.90s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.95s/it] [repeated 2x across cluster] (WorkerDict pid=2074387) Loading checkpoint shards: 100%|██████████| 3/3 [00:06<00:00, 2.01s/it] [repeated 4x across cluster] (WorkerDict pid=2074385) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2074385) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2074385) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2074386) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2074386) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2074391) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2074391) warnings.warn( (WorkerDict pid=2074385) reference model: Qwen/Qwen3-4B (WorkerDict pid=2074391) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 8x across cluster] (WorkerDict pid=2074391) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 8x across cluster] (WorkerDict pid=2074385) Model config after override: Qwen3Config { (WorkerDict pid=2074385) "architectures": [ (WorkerDict pid=2074385) "Qwen3ForCausalLM" (WorkerDict pid=2074385) ], (WorkerDict pid=2074385) "attention_bias": false, (WorkerDict pid=2074385) "attention_dropout": 0.0, (WorkerDict pid=2074385) "dtype": "bfloat16", (WorkerDict pid=2074385) "eos_token_id": 151645, (WorkerDict pid=2074385) "head_dim": 128, (WorkerDict pid=2074385) "hidden_act": "silu", (WorkerDict pid=2074385) "hidden_size": 2560, (WorkerDict pid=2074385) "initializer_range": 0.02, (WorkerDict pid=2074385) "intermediate_size": 9728, (WorkerDict pid=2074385) "layer_types": [ (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention" (WorkerDict pid=2074385) ], (WorkerDict pid=2074385) "max_position_embeddings": 40960, (WorkerDict pid=2074385) "max_window_layers": 36, (WorkerDict pid=2074385) "model_type": "qwen3", (WorkerDict pid=2074385) "num_attention_heads": 32, (WorkerDict pid=2074385) "num_hidden_layers": 36, (WorkerDict pid=2074385) "num_key_value_heads": 8, (WorkerDict pid=2074385) "pad_token_id": 151643, (WorkerDict pid=2074385) "rms_norm_eps": 1e-06, (WorkerDict pid=2074385) "rope_scaling": null, (WorkerDict pid=2074385) "rope_theta": 1000000, (WorkerDict pid=2074385) "sliding_window": null, (WorkerDict pid=2074385) "tie_word_embeddings": true, (WorkerDict pid=2074385) "transformers_version": "4.57.1", (WorkerDict pid=2074385) "use_cache": true, (WorkerDict pid=2074385) "use_sliding_window": false, (WorkerDict pid=2074385) "vocab_size": 151936 (WorkerDict pid=2074385) } (WorkerDict pid=2074385) (WorkerDict pid=2074387) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (WorkerDict pid=2074391) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2074385) Ref use_remove_padding=True (WorkerDict pid=2074385) Ref use_fused_kernels=False (WorkerDict pid=2074385) Ref use_prefix_grouper=False (WorkerDict pid=2074392) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2074392) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2074004) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2074392) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:06<00:03, 3.02s/it] [repeated 14x across cluster] (WorkerDict pid=2074392) Loading checkpoint shards: 100%|██████████| 3/3 [00:06<00:00, 2.04s/it] [repeated 6x across cluster] (vLLMHttpServer pid=2075282) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2075282) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2075282) ['serve', (vLLMHttpServer pid=2075282) 'Qwen/Qwen3-4B', (vLLMHttpServer pid=2075282) '--dtype', (vLLMHttpServer pid=2075282) 'bfloat16', (vLLMHttpServer pid=2075282) '--load_format', (vLLMHttpServer pid=2075282) 'dummy', (vLLMHttpServer pid=2075282) '--trust_remote_code', (vLLMHttpServer pid=2075282) '--max_model_len', (vLLMHttpServer pid=2075282) '40960', (vLLMHttpServer pid=2075282) '--max_num_seqs', (vLLMHttpServer pid=2075282) '1024', (vLLMHttpServer pid=2075282) '--enable_chunked_prefill', (vLLMHttpServer pid=2075282) '--max_num_batched_tokens', (vLLMHttpServer pid=2075282) '10240', (vLLMHttpServer pid=2075282) '--enable_prefix_caching', (vLLMHttpServer pid=2075282) '--enable_sleep_mode', (vLLMHttpServer pid=2075282) '--logprobs_mode', (vLLMHttpServer pid=2075282) 'processed_logprobs', (vLLMHttpServer pid=2075282) '--disable_custom_all_reduce', (vLLMHttpServer pid=2075282) '--gpu_memory_utilization', (vLLMHttpServer pid=2075282) '0.8', (vLLMHttpServer pid=2075282) '--disable_log_stats', (vLLMHttpServer pid=2075282) '--tensor_parallel_size', (vLLMHttpServer pid=2075282) '2', (vLLMHttpServer pid=2075282) '--seed', (vLLMHttpServer pid=2075282) '0', (vLLMHttpServer pid=2075282) '--override_generation_config', (vLLMHttpServer pid=2075282) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2075282) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2075282) '--hf_overrides', (vLLMHttpServer pid=2075282) '{}', (vLLMHttpServer pid=2075282) '--scheduling_policy', (vLLMHttpServer pid=2075282) 'fcfs'] (vLLMHttpServer pid=2075282) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2075282) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2075282) WARNING 07-02 08:57:45 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2074385) WARNING 07-02 08:57:52 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2075287) WARNING 07-02 08:57:46 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2074389) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last): (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f: (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/aotautograd/aqwmczjqsm4pvwfimkpe3rswyj2xfwbwedbijxvwd7tfo5n2eqnk/.2074388.133380852872896.tmp' (vLLMHttpServer pid=2075287) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 3x across cluster] (vLLMHttpServer pid=2075287) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 3x across cluster] (vLLMHttpServer pid=2075287) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2075287) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2074385) 2026-07-02 08:58:08,146 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2074386) 2026-07-02 08:58:08,165 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. We need to find the charge $ q $ using the formula: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Rearranging to solve for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substituting the known values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest option is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2074004) step:10 - global_seqlen/min:14569 - global_seqlen/max:25213 - global_seqlen/minmax_diff:10644 - global_seqlen/balanced_min:21068 - global_seqlen/balanced_max:21078 - global_seqlen/mean:21074.625 - actor/entropy:0.13031111657619476 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5310845375061035 - training/rollout_probs_diff_mean:0.0036879268009215593 - training/rollout_probs_diff_std:0.013145294040441513 - training/rollout_actor_probs_pearson_corr:0.9974448680877686 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9999222755432129 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00015043777239043266 - rollout_corr/rollout_is_max:1.8829189538955688 - rollout_corr/rollout_is_min:0.005527450703084469 - rollout_corr/rollout_is_std:0.03613611310720444 - rollout_corr/rollout_is_eff_sample_size:0.9986955276850646 - rollout_corr/rollout_is_seq_mean:0.9999170899391174 - rollout_corr/rollout_is_seq_std:0.002527824370190501 - rollout_corr/rollout_is_seq_max:1.0131255388259888 - rollout_corr/rollout_is_seq_min:0.9898325800895691 - rollout_corr/rollout_is_seq_max_deviation:0.01312553882598877 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1374189392663538) - rollout_corr/training_log_ppl:np.float64(0.12561230757273734) - rollout_corr/kl:np.float64(0.0010259306004876123) - rollout_corr/k3_kl:np.float64(0.001012646021507635) - rollout_corr/rollout_ppl:np.float64(1.1362624615430832) - rollout_corr/rollout_log_ppl:np.float64(0.1245863765507238) - rollout_corr/log_ppl_diff:np.float64(0.0010259310220135376) - rollout_corr/log_ppl_abs_diff:np.float64(0.00206260806589853) - rollout_corr/log_ppl_diff_max:np.float64(0.03187315911054611) - rollout_corr/log_ppl_diff_min:np.float64(-0.010174185037612915) - rollout_corr/ppl_ratio:np.float64(1.0010317743290216) - rollout_corr/chi2_token:np.float64(0.0018072936218231916) - rollout_corr/chi2_seq:np.float64(0.3196559448260814) - actor/pg_loss:np.float64(0.0017917423974722624) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004963221986145072) - actor/ppo_kl:np.float64(8.269873148769591e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(95332.59919399978) - self_distillation/token_reweight_w_std:np.float64(1506740.680497941) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996795877814293) - self_distillation/token_reweight_w_clip_frac:np.float64(0.033084604699979536) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.3074006289243698) - perf/mfu/actor:np.float64(0.03828182711170613) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.1017189025879) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.596875) - val-aux/sciknoweval/reward/std@16:np.float64(0.17025334585854696) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6700875) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.14365860035136047) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.525375) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14433762594467914) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5972999999999999) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.16952408520330536) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.72475) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1008625343689749) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4628500000000001) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10408901059613167) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6178625) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1414437668650514) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7605999999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0665978685707884) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4221375) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0550829574380763) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6353249999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10804651530040861) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7887749999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.044373467337921746) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.40466250000000004) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.019017006085694965) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6468625) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07892990828797705) - val-aux/sciknoweval/score/mean@16:np.float64(0.596875) - val-aux/sciknoweval/score/std@16:np.float64(0.17025334585854696) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6700875) - val-aux/sciknoweval/score/best@2/std:np.float64(0.14365860035136047) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.525375) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.14433762594467914) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.5972999999999999) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.16952408520330536) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.72475) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1008625343689749) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4628500000000001) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10408901059613167) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6178625) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1414437668650514) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7605999999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0665978685707884) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4221375) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0550829574380763) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6353249999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10804651530040861) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7887749999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.044373467337921746) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.40466250000000004) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.019017006085694965) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6468625) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07892990828797705) - val-core/sciknoweval/acc/mean@16:np.float64(0.596875) - val-aux/sciknoweval/acc/std@16:np.float64(0.17025334585854696) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6700875) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.14365860035136047) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.525375) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.14433762594467914) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.5972999999999999) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.16952408520330536) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.72475) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1008625343689749) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4628500000000001) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10408901059613167) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6178625) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1414437668650514) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7605999999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0665978685707884) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4221375) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0550829574380763) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6353249999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10804651530040861) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7887749999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.044373467337921746) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.40466250000000004) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.019017006085694965) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6468625) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07892990828797705) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.02228859585888569) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9995125) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0055870721336611015) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9884875000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.02914679194594295) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9939625) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.021812516180988194) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999750000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0005584576975922169) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9776375000000002) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.03766353637974691) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.997625) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.013472998070955988) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9620000000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.04262616078525806) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9992875) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.005053486156825662) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9403500000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03977205801345875) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998875) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0015086398882979818) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0032043245155364275 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0032043245155364275 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:389.48828125 - response_length/max:1190.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:389.48828125 - response_length_non_aborted/max:1190.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.09375 - prompt_length/max:382.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015878118574619293 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1143263150006533) - timing_s/agent_loop/generate_sequences/max:np.float64(8.482053555548191) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.887139186161221) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.482053555548191) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:337 - timing_s/agent_loop/slowest/response_length:1190 - timing_s/gen:14.678978031501174 - timing_s/reward:0.05808144994080067 - timing_s/old_log_prob:2.4217102844268084 - timing_s/adv:0.5421234201639891 - timing_s/update_actor:15.863357232883573 - timing_s/step:33.64816468022764 - timing_s/testing:93.59819382615387 - timing_s/save_checkpoint:6.729555023834109 - timing_s/stop_profile:0.00014853663742542267 - timing_per_token_ms/update_actor:0.09409038851749184 - timing_per_token_ms/adv:0.003215498616013269 - timing_per_token_ms/gen:0.14721818523404281 - perf/total_num_tokens:168597 - perf/time_per_step:33.64816468022764 - perf/throughput:626.3231650308669 (TaskRunner pid=2074004) Training Progress: 10%|█ | 10/100 [08:42<1:43:00, 68.67s/it] (TaskRunner pid=2074004) step:11 - global_seqlen/min:15931 - global_seqlen/max:27826 - global_seqlen/minmax_diff:11895 - global_seqlen/balanced_min:22821 - global_seqlen/balanced_max:22841 - global_seqlen/mean:22835.0 - actor/entropy:0.12203283607959747 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40049630403518677 - training/rollout_probs_diff_mean:0.003223797306418419 - training/rollout_probs_diff_std:0.012214891612529755 - training/rollout_actor_probs_pearson_corr:0.9976435303688049 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:0.9997804164886475 - rollout_corr/rollout_is_ratio_fraction_high:2.6688492653192952e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.3376985306385905e-05 - rollout_corr/rollout_is_max:2.5879318714141846 - rollout_corr/rollout_is_min:0.06322140991687775 - rollout_corr/rollout_is_std:0.033195044845342636 - rollout_corr/rollout_is_eff_sample_size:0.9988987665423175 - rollout_corr/rollout_is_seq_mean:0.9999120235443115 - rollout_corr/rollout_is_seq_std:0.002345130778849125 - rollout_corr/rollout_is_seq_max:1.0110429525375366 - rollout_corr/rollout_is_seq_min:0.9912058711051941 - rollout_corr/rollout_is_seq_max_deviation:0.011042952537536621 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1257328824140131) - rollout_corr/training_log_ppl:np.float64(0.1140785860188771) - rollout_corr/kl:np.float64(0.0006342849212974677) - rollout_corr/k3_kl:np.float64(0.0007503947783931153) - rollout_corr/rollout_ppl:np.float64(1.1249900460243225) - rollout_corr/rollout_log_ppl:np.float64(0.11344430113604176) - rollout_corr/log_ppl_diff:np.float64(0.0006342848828353453) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019007355404028203) - rollout_corr/log_ppl_diff_max:np.float64(0.010488316416740417) - rollout_corr/log_ppl_diff_min:np.float64(-0.008857212960720062) - rollout_corr/ppl_ratio:np.float64(1.0006378127727658) - rollout_corr/chi2_token:np.float64(0.001916841370984912) - rollout_corr/chi2_seq:np.float64(0.37116734171286225) - actor/pg_loss:np.float64(0.0027762840036302805) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005819873397285846) - actor/ppo_kl:np.float64(-7.081770549888233e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(90827.28597010626) - self_distillation/token_reweight_w_std:np.float64(1824642.9953269707) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001720192376524) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04453263468167279) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.29148364439606667) - perf/mfu/actor:np.float64(0.04050194858164669) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.13693618774414) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0004592644691001624 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0004592644691001624 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:439.09375 - response_length/max:1273.0 - response_length/min:68.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:439.09375 - response_length_non_aborted/max:1273.0 - response_length_non_aborted/min:68.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.5 - prompt_length/max:437.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.3513795137405396e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6711478792130947) - timing_s/agent_loop/generate_sequences/max:np.float64(9.547774223610759) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.419399234910088) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.547774223610759) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:1273 - timing_s/gen:15.524424999952316 - timing_s/reward:0.057088837027549744 - timing_s/old_log_prob:2.659148082137108 - timing_s/adv:0.5871537029743195 - timing_s/update_actor:16.28799468278885 - timing_s/step:35.222145142033696 - timing_s/stop_profile:0.0001278892159461975 - timing_per_token_ms/update_actor:0.0891613459754152 - timing_per_token_ms/adv:0.003214110482670897 - timing_per_token_ms/gen:0.13810783040310579 - perf/total_num_tokens:182680 - perf/time_per_step:35.222145142033696 - perf/throughput:648.3137216066087 (TaskRunner pid=2074004) Training Progress: 11%|█ | 11/100 [09:18<1:26:42, 58.45s/it] (TaskRunner pid=2074004) step:12 - global_seqlen/min:16305 - global_seqlen/max:30523 - global_seqlen/minmax_diff:14218 - global_seqlen/balanced_min:22309 - global_seqlen/balanced_max:28829 - global_seqlen/mean:23127.125 - actor/entropy:0.12043065577745438 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6972464323043823 - training/rollout_probs_diff_mean:0.0032291633542627096 - training/rollout_probs_diff_std:0.012314102612435818 - training/rollout_actor_probs_pearson_corr:0.9975891709327698 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:0.9999233484268188 - rollout_corr/rollout_is_ratio_fraction_high:6.790191400796175e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.5463217752985656e-05 - rollout_corr/rollout_is_max:3.958061456680298 - rollout_corr/rollout_is_min:0.3131382167339325 - rollout_corr/rollout_is_std:0.033765628933906555 - rollout_corr/rollout_is_eff_sample_size:0.9988609428668661 - rollout_corr/rollout_is_seq_mean:0.9999712705612183 - rollout_corr/rollout_is_seq_std:0.0021552243269979954 - rollout_corr/rollout_is_seq_max:1.0135027170181274 - rollout_corr/rollout_is_seq_min:0.9933554530143738 - rollout_corr/rollout_is_seq_max_deviation:0.013502717018127441 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1240018154494464) - rollout_corr/training_log_ppl:np.float64(0.11423524462952628) - rollout_corr/kl:np.float64(0.0006360744373044014) - rollout_corr/k3_kl:np.float64(0.0007582616624119964) - rollout_corr/rollout_ppl:np.float64(1.1232725959271193) - rollout_corr/rollout_log_ppl:np.float64(0.11359916946094017) - rollout_corr/log_ppl_diff:np.float64(0.0006360751685861032) - rollout_corr/log_ppl_abs_diff:np.float64(0.001725419224385405) - rollout_corr/log_ppl_diff_max:np.float64(0.009453028440475464) - rollout_corr/log_ppl_diff_min:np.float64(-0.006760995835065842) - rollout_corr/ppl_ratio:np.float64(1.0006386556196958) - rollout_corr/chi2_token:np.float64(0.0019189040176570415) - rollout_corr/chi2_seq:np.float64(1.096296020084992) - actor/pg_loss:np.float64(0.0028759860433638096) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007010218323557638) - actor/ppo_kl:np.float64(-5.775291747767142e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(54891.713903531665) - self_distillation/token_reweight_w_std:np.float64(995585.2390469867) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996340507641435) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05258868928649463) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.3390292413532734) - perf/mfu/actor:np.float64(0.03925409114627591) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.01633071899414) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.022134963423013687 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.022134963423013687 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:460.22265625 - response_length/max:8192.0 - response_length/min:78.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:460.22265625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:262.5 - prompt_length/max:364.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.100325405597687e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.941336028277874) - timing_s/agent_loop/generate_sequences/max:np.float64(50.94393623247743) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.578305393028131) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.94393623247743) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:211 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.6255786344409 - timing_s/reward:0.06343746557831764 - timing_s/old_log_prob:2.6843459140509367 - timing_s/adv:0.540890708565712 - timing_s/update_actor:17.096599811688066 - timing_s/step:77.09979971125722 - timing_s/stop_profile:0.00011315196752548218 - timing_per_token_ms/update_actor:0.09240556171426445 - timing_per_token_ms/adv:0.002923464917092548 - timing_per_token_ms/gen:0.480623158240669 - perf/total_num_tokens:185017 - perf/time_per_step:77.09979971125722 - perf/throughput:299.96349000402455 (TaskRunner pid=2074004) Training Progress: 12%|█▏ | 12/100 [10:35<1:34:04, 64.14s/it] (TaskRunner pid=2074004) step:13 - global_seqlen/min:18538 - global_seqlen/max:25997 - global_seqlen/minmax_diff:7459 - global_seqlen/balanced_min:22090 - global_seqlen/balanced_max:22116 - global_seqlen/mean:22109.375 - actor/entropy:0.14604653418064117 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7997847199440002 - training/rollout_probs_diff_mean:0.0036600856110453606 - training/rollout_probs_diff_std:0.012734896503388882 - training/rollout_actor_probs_pearson_corr:0.9978328943252563 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.703125 - self_distillation/correct_sample_fraction:0.49609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.703125 - rollout_corr/rollout_is_mean:0.9999634623527527 - rollout_corr/rollout_is_ratio_fraction_high:2.843844413291663e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.531533239874989e-05 - rollout_corr/rollout_is_max:2.8094775676727295 - rollout_corr/rollout_is_min:6.210129504324868e-05 - rollout_corr/rollout_is_std:0.03579304739832878 - rollout_corr/rollout_is_eff_sample_size:0.9987203780861295 - rollout_corr/rollout_is_seq_mean:0.9999745488166809 - rollout_corr/rollout_is_seq_std:0.002158660674467683 - rollout_corr/rollout_is_seq_max:1.0080207586288452 - rollout_corr/rollout_is_seq_min:0.9927275776863098 - rollout_corr/rollout_is_seq_max_deviation:0.008020758628845215 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1676474194973707) - rollout_corr/training_log_ppl:np.float64(0.14873290639661718) - rollout_corr/kl:np.float64(0.0009227445474575546) - rollout_corr/k3_kl:np.float64(0.0009155499404727152) - rollout_corr/rollout_ppl:np.float64(1.1665417086333036) - rollout_corr/rollout_log_ppl:np.float64(0.14781016217602883) - rollout_corr/log_ppl_diff:np.float64(0.0009227442205883563) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018867802718887106) - rollout_corr/log_ppl_diff_max:np.float64(0.060229599475860596) - rollout_corr/log_ppl_diff_min:np.float64(-0.004808485507965088) - rollout_corr/ppl_ratio:np.float64(1.0009324494749308) - rollout_corr/chi2_token:np.float64(0.0014858830254524946) - rollout_corr/chi2_seq:np.float64(0.4379577168729156) - actor/pg_loss:np.float64(0.002721810364164412) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005266812750051031) - actor/ppo_kl:np.float64(1.293608784180833e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.49609375) - self_distillation/token_reweight_w_mean:np.float64(196224.12132495665) - self_distillation/token_reweight_w_std:np.float64(2690730.0869071595) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001536855706945) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06831683132622857) - self_distillation/empty_target_batch:np.float64(0.296875) - actor/grad_norm:np.float64(0.4452543556690216) - perf/mfu/actor:np.float64(0.03950919290459659) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(142.9030246734619) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0021506573539227247 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0021506573539227247 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:412.07421875 - response_length/max:1343.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:412.07421875 - response_length_non_aborted/max:1343.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.84375 - prompt_length/max:363.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001416429877281189 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2486017309129238) - timing_s/agent_loop/generate_sequences/max:np.float64(9.4201366789639) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.31274351340835) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.4201366789639) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:346 - timing_s/agent_loop/slowest/response_length:1343 - timing_s/gen:15.389248048886657 - timing_s/reward:0.059297624975442886 - timing_s/old_log_prob:2.4455661196261644 - timing_s/adv:0.5371290631592274 - timing_s/update_actor:16.235428664833307 - timing_s/step:34.75727551802993 - timing_s/stop_profile:3.674998879432678e-05 - timing_per_token_ms/update_actor:0.09179040941248513 - timing_per_token_ms/adv:0.003036772088532734 - timing_per_token_ms/gen:0.14588209467050892 - perf/total_num_tokens:176875 - perf/time_per_step:34.75727551802993 - perf/throughput:636.107826936292 (TaskRunner pid=2074004) Training Progress: 13%|█▎ | 13/100 [11:09<1:20:06, 55.24s/it] (TaskRunner pid=2074004) step:14 - global_seqlen/min:15836 - global_seqlen/max:27136 - global_seqlen/minmax_diff:11300 - global_seqlen/balanced_min:21152 - global_seqlen/balanced_max:21214 - global_seqlen/mean:21164.375 - actor/entropy:0.13442379236221313 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967105984687805 - training/rollout_probs_diff_mean:0.003770587034523487 - training/rollout_probs_diff_std:0.013384245336055756 - training/rollout_actor_probs_pearson_corr:0.9973688125610352 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.890625 - self_distillation/correct_sample_fraction:0.671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.890625 - rollout_corr/rollout_is_mean:1.0001509189605713 - rollout_corr/rollout_is_ratio_fraction_high:3.0175318897818215e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012070127559127286 - rollout_corr/rollout_is_max:3.7234184741973877 - rollout_corr/rollout_is_min:0.041150957345962524 - rollout_corr/rollout_is_std:0.03739259019494057 - rollout_corr/rollout_is_eff_sample_size:0.9986041031225663 - rollout_corr/rollout_is_seq_mean:1.000020146369934 - rollout_corr/rollout_is_seq_std:0.002401510952040553 - rollout_corr/rollout_is_seq_max:1.0093960762023926 - rollout_corr/rollout_is_seq_min:0.9911119341850281 - rollout_corr/rollout_is_seq_max_deviation:0.009396076202392578 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1523799053393304) - rollout_corr/training_log_ppl:np.float64(0.13728270668070763) - rollout_corr/kl:np.float64(0.0008854879360526979) - rollout_corr/k3_kl:np.float64(0.0008427418540151166) - rollout_corr/rollout_ppl:np.float64(1.1513059278950095) - rollout_corr/rollout_log_ppl:np.float64(0.13639721802610438) - rollout_corr/log_ppl_diff:np.float64(0.0008854886546032503) - rollout_corr/log_ppl_abs_diff:np.float64(0.001983832713449374) - rollout_corr/log_ppl_diff_max:np.float64(0.011519268155097961) - rollout_corr/log_ppl_diff_min:np.float64(-0.00734923779964447) - rollout_corr/ppl_ratio:np.float64(1.0008891641627997) - rollout_corr/chi2_token:np.float64(0.001656870823353529) - rollout_corr/chi2_seq:np.float64(1.6609382745809853) - actor/pg_loss:np.float64(0.004066150169819593) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009369107503971463) - actor/ppo_kl:np.float64(0.00011594514723167748) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.890625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.890625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.671875) - self_distillation/token_reweight_w_mean:np.float64(220119.85480736103) - self_distillation/token_reweight_w_std:np.float64(3045461.0947162565) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001637001987547) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08081251988187432) - self_distillation/empty_target_batch:np.float64(0.109375) - actor/grad_norm:np.float64(0.4899965673685074) - perf/mfu/actor:np.float64(0.03798268804792023) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.0521469116211) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005462990142405033 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005462990142405033 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:388.35546875 - response_length/max:1228.0 - response_length/min:63.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:388.35546875 - response_length_non_aborted/max:1228.0 - response_length_non_aborted/min:63.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.03125 - prompt_length/max:375.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.705312550067902e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7120892163366079) - timing_s/agent_loop/generate_sequences/max:np.float64(8.587548367679119) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.029178587690694) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.587548367679119) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:301 - timing_s/agent_loop/slowest/response_length:1228 - timing_s/gen:14.909624475985765 - timing_s/reward:0.05932668782770634 - timing_s/old_log_prob:2.404671125113964 - timing_s/adv:0.5401296932250261 - timing_s/update_actor:16.007855907082558 - timing_s/step:34.01097614504397 - timing_s/stop_profile:0.00011675432324409485 - timing_per_token_ms/update_actor:0.09454481828002574 - timing_per_token_ms/adv:0.003190087666332139 - timing_per_token_ms/gen:0.14996755626173836 - perf/total_num_tokens:169315 - perf/time_per_step:34.01097614504397 - perf/throughput:622.2807281314695 (TaskRunner pid=2074004) Training Progress: 14%|█▍ | 14/100 [11:43<1:10:00, 48.85s/it] (TaskRunner pid=2074004) step:15 - global_seqlen/min:13000 - global_seqlen/max:27017 - global_seqlen/minmax_diff:14017 - global_seqlen/balanced_min:21238 - global_seqlen/balanced_max:21268 - global_seqlen/mean:21253.875 - actor/entropy:0.11480943113565445 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2772899866104126 - training/rollout_probs_diff_mean:0.0029744282364845276 - training/rollout_probs_diff_std:0.01138862781226635 - training/rollout_actor_probs_pearson_corr:0.9978798031806946 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.69140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:0.9999694228172302 - rollout_corr/rollout_is_ratio_fraction_high:2.9455948606482707e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.9455948606482707e-05 - rollout_corr/rollout_is_max:2.554089307785034 - rollout_corr/rollout_is_min:0.44323766231536865 - rollout_corr/rollout_is_std:0.032444074749946594 - rollout_corr/rollout_is_eff_sample_size:0.998948250895511 - rollout_corr/rollout_is_seq_mean:1.000088095664978 - rollout_corr/rollout_is_seq_std:0.001991908298805356 - rollout_corr/rollout_is_seq_max:1.0099899768829346 - rollout_corr/rollout_is_seq_min:0.9942845106124878 - rollout_corr/rollout_is_seq_max_deviation:0.00998997688293457 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.126635740045458) - rollout_corr/training_log_ppl:np.float64(0.11509814600140089) - rollout_corr/kl:np.float64(0.0005166397006295398) - rollout_corr/k3_kl:np.float64(0.0006518633542142993) - rollout_corr/rollout_ppl:np.float64(1.126022334676236) - rollout_corr/rollout_log_ppl:np.float64(0.11458150647194998) - rollout_corr/log_ppl_diff:np.float64(0.000516639529450913) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016528939577256097) - rollout_corr/log_ppl_diff_max:np.float64(0.007337689399719238) - rollout_corr/log_ppl_diff_min:np.float64(-0.009172983467578888) - rollout_corr/ppl_ratio:np.float64(1.0005192304961383) - rollout_corr/chi2_token:np.float64(0.0016447578091174364) - rollout_corr/chi2_seq:np.float64(0.39781012921594083) - actor/pg_loss:np.float64(0.002455260371789336) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006204967883149948) - actor/ppo_kl:np.float64(4.5773536659687863e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69140625) - self_distillation/token_reweight_w_mean:np.float64(192128.32348678238) - self_distillation/token_reweight_w_std:np.float64(3233259.402355935) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002948227338493) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06099457883101422) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.3918348103761673) - perf/mfu/actor:np.float64(0.03843189897453754) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(142.9206657409668) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0015439826529473066 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0015439826529473066 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:397.83984375 - response_length/max:1327.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:397.83984375 - response_length_non_aborted/max:1327.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:460.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.262189269065857e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7463704198598862) - timing_s/agent_loop/generate_sequences/max:np.float64(8.988874156028032) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.964175615350541) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.988874156028032) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:1327 - timing_s/gen:14.802029374986887 - timing_s/reward:0.06032639183104038 - timing_s/old_log_prob:2.3973333202302456 - timing_s/adv:0.5653208959847689 - timing_s/update_actor:15.675101241096854 - timing_s/step:33.58521797135472 - timing_s/stop_profile:3.232806921005249e-05 - timing_per_token_ms/update_actor:0.09218966683191214 - timing_per_token_ms/adv:0.0033248107461861004 - timing_per_token_ms/gen:0.14533593895732705 - perf/total_num_tokens:170031 - perf/time_per_step:33.58521797135472 - perf/throughput:632.8342134961789 (TaskRunner pid=2074004) Training Progress: 15%|█▌ | 15/100 [12:17<1:02:41, 44.25s/it] (TaskRunner pid=2074004) step:16 - global_seqlen/min:16611 - global_seqlen/max:28011 - global_seqlen/minmax_diff:11400 - global_seqlen/balanced_min:20244 - global_seqlen/balanced_max:26855 - global_seqlen/mean:21153.75 - actor/entropy:0.12396589666604996 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8040750622749329 - training/rollout_probs_diff_mean:0.0032445986289530993 - training/rollout_probs_diff_std:0.012475800700485706 - training/rollout_actor_probs_pearson_corr:0.9975688457489014 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:0.9998238682746887 - rollout_corr/rollout_is_ratio_fraction_high:3.8920348742976785e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.730087185744196e-05 - rollout_corr/rollout_is_max:2.290397882461548 - rollout_corr/rollout_is_min:3.1829338240640936e-06 - rollout_corr/rollout_is_std:0.03366485983133316 - rollout_corr/rollout_is_eff_sample_size:0.9988676034321022 - rollout_corr/rollout_is_seq_mean:0.9998279809951782 - rollout_corr/rollout_is_seq_std:0.002493756590411067 - rollout_corr/rollout_is_seq_max:1.0092169046401978 - rollout_corr/rollout_is_seq_min:0.9886294007301331 - rollout_corr/rollout_is_seq_max_deviation:0.011370599269866943 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1432740236632526) - rollout_corr/training_log_ppl:np.float64(0.12944419973246113) - rollout_corr/kl:np.float64(0.00073082422317583) - rollout_corr/k3_kl:np.float64(0.00073855356768604) - rollout_corr/rollout_ppl:np.float64(1.1423929436132312) - rollout_corr/rollout_log_ppl:np.float64(0.1287133757614356) - rollout_corr/log_ppl_diff:np.float64(0.0007308239710255293) - rollout_corr/log_ppl_abs_diff:np.float64(0.001978938538741204) - rollout_corr/log_ppl_diff_max:np.float64(0.010752767324447632) - rollout_corr/log_ppl_diff_min:np.float64(-0.0069452449679374695) - rollout_corr/ppl_ratio:np.float64(1.000734378118068) - rollout_corr/chi2_token:np.float64(0.001494420226663351) - rollout_corr/chi2_seq:np.float64(0.7517771443817765) - actor/pg_loss:np.float64(0.002636288059875369) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006587515040337166) - actor/ppo_kl:np.float64(-0.0003202963638955225) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(219386.07391313673) - self_distillation/token_reweight_w_std:np.float64(3319235.12259777) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0016510419081897) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05755699300789274) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.43999863415956497) - perf/mfu/actor:np.float64(0.03636736138421752) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(142.8370819091797) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.04519382491707802 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.04519382491707802 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:401.4609375 - response_length/max:8192.0 - response_length/min:97.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:401.4609375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:259.59375 - prompt_length/max:344.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.450495958328247e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.158904004842043) - timing_s/agent_loop/generate_sequences/max:np.float64(50.02395389974117) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.92313740587997) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.02395389974117) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:284 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.74822269193828 - timing_s/reward:0.05993708409368992 - timing_s/old_log_prob:2.6834859494119883 - timing_s/adv:0.5374118015170097 - timing_s/update_actor:16.872027203440666 - timing_s/step:75.98889152519405 - timing_s/stop_profile:0.00010964088141918182 - timing_per_token_ms/update_actor:0.09969879574213003 - timing_per_token_ms/adv:0.003175629625462446 - timing_per_token_ms/gen:0.5424350778595587 - perf/total_num_tokens:169230 - perf/time_per_step:75.98889152519405 - perf/throughput:278.37950489100757 (TaskRunner pid=2074004) Training Progress: 16%|█▌ | 16/100 [13:33<1:15:20, 53.82s/it] (TaskRunner pid=2074004) step:17 - global_seqlen/min:16383 - global_seqlen/max:22557 - global_seqlen/minmax_diff:6174 - global_seqlen/balanced_min:19789 - global_seqlen/balanced_max:19793 - global_seqlen/mean:19791.0 - actor/entropy:0.1373070627450943 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3535119295120239 - training/rollout_probs_diff_mean:0.0037673222832381725 - training/rollout_probs_diff_std:0.013155417516827583 - training/rollout_actor_probs_pearson_corr:0.997542679309845 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.890625 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.890625 - rollout_corr/rollout_is_mean:1.0000176429748535 - rollout_corr/rollout_is_ratio_fraction_high:5.546680768020451e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.765353075228631e-05 - rollout_corr/rollout_is_max:2.7404403686523438 - rollout_corr/rollout_is_min:0.32417601346969604 - rollout_corr/rollout_is_std:0.037032198160886765 - rollout_corr/rollout_is_eff_sample_size:0.9986302566846117 - rollout_corr/rollout_is_seq_mean:1.0000929832458496 - rollout_corr/rollout_is_seq_std:0.0029761020559817553 - rollout_corr/rollout_is_seq_max:1.0141609907150269 - rollout_corr/rollout_is_seq_min:0.9883509874343872 - rollout_corr/rollout_is_seq_max_deviation:0.014160990715026855 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1427205740474164) - rollout_corr/training_log_ppl:np.float64(0.13067101137858117) - rollout_corr/kl:np.float64(0.001165712662167806) - rollout_corr/k3_kl:np.float64(0.0011212053367444241) - rollout_corr/rollout_ppl:np.float64(1.1413688631728292) - rollout_corr/rollout_log_ppl:np.float64(0.12950529891531914) - rollout_corr/log_ppl_diff:np.float64(0.0011657124632620253) - rollout_corr/log_ppl_abs_diff:np.float64(0.002574236517830286) - rollout_corr/log_ppl_diff_max:np.float64(0.018865570425987244) - rollout_corr/log_ppl_diff_min:np.float64(-0.009756885468959808) - rollout_corr/ppl_ratio:np.float64(1.0011729013640434) - rollout_corr/chi2_token:np.float64(0.001890765968710184) - rollout_corr/chi2_seq:np.float64(1.113262691302225) - actor/pg_loss:np.float64(0.004973054630681872) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013291486598063784) - actor/ppo_kl:np.float64(0.000477771908172997) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.890625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.890625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(430373.50576436426) - self_distillation/token_reweight_w_std:np.float64(5256699.274065455) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005029244348407) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08690081464010291) - self_distillation/empty_target_batch:np.float64(0.109375) - actor/grad_norm:np.float64(0.691158652305603) - perf/mfu/actor:np.float64(0.03633210791443406) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(142.84893798828125) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0020605919416993856 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0020605919416993856 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:352.125 - response_length/max:919.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:352.125 - response_length_non_aborted/max:919.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.005889296531677e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8571333661675453) - timing_s/agent_loop/generate_sequences/max:np.float64(7.130504677072167) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.622363231239433) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.130504677072167) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:274 - timing_s/agent_loop/slowest/response_length:919 - timing_s/gen:12.924050455912948 - timing_s/reward:0.059431292116642 - timing_s/old_log_prob:2.4258709643036127 - timing_s/adv:0.542781762778759 - timing_s/update_actor:15.836449747905135 - timing_s/step:31.876996461302042 - timing_s/stop_profile:0.0001156199723482132 - timing_per_token_ms/update_actor:0.10002305181588307 - timing_per_token_ms/adv:0.003428210820440851 - timing_per_token_ms/gen:0.14337116675444786 - perf/total_num_tokens:158328 - perf/time_per_step:31.876996461302042 - perf/throughput:620.8552309508152 (TaskRunner pid=2074004) Training Progress: 17%|█▋ | 17/100 [14:05<1:05:20, 47.23s/it] (TaskRunner pid=2074004) step:18 - global_seqlen/min:15614 - global_seqlen/max:27961 - global_seqlen/minmax_diff:12347 - global_seqlen/balanced_min:22588 - global_seqlen/balanced_max:22618 - global_seqlen/mean:22607.75 - actor/entropy:0.11930368840694427 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7529500722885132 - training/rollout_probs_diff_mean:0.0031947942916303873 - training/rollout_probs_diff_std:0.01253251638263464 - training/rollout_actor_probs_pearson_corr:0.9974825382232666 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:0.9999328255653381 - rollout_corr/rollout_is_ratio_fraction_high:3.530138565110974e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015885622997302562 - rollout_corr/rollout_is_max:2.3141205310821533 - rollout_corr/rollout_is_min:0.1902111917734146 - rollout_corr/rollout_is_std:0.03377268835902214 - rollout_corr/rollout_is_eff_sample_size:0.998860586053378 - rollout_corr/rollout_is_seq_mean:1.000009298324585 - rollout_corr/rollout_is_seq_std:0.0019673723727464676 - rollout_corr/rollout_is_seq_max:1.007437825202942 - rollout_corr/rollout_is_seq_min:0.9938141703605652 - rollout_corr/rollout_is_seq_max_deviation:0.0074378252029418945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.129554508253932) - rollout_corr/training_log_ppl:np.float64(0.11883334071171703) - rollout_corr/kl:np.float64(0.0006264003223446224) - rollout_corr/k3_kl:np.float64(0.0007624520011759728) - rollout_corr/rollout_ppl:np.float64(1.1288568419404328) - rollout_corr/rollout_log_ppl:np.float64(0.1182069395945291) - rollout_corr/log_ppl_diff:np.float64(0.0006264011171879247) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016409209492849186) - rollout_corr/log_ppl_diff_max:np.float64(0.008434589952230453) - rollout_corr/log_ppl_diff_min:np.float64(-0.006128266453742981) - rollout_corr/ppl_ratio:np.float64(1.0006289719603956) - rollout_corr/chi2_token:np.float64(0.0018175377044826746) - rollout_corr/chi2_seq:np.float64(0.6837903573177755) - actor/pg_loss:np.float64(0.004627972492016852) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010802133826928184) - actor/ppo_kl:np.float64(-1.8316878190205443e-05) - actor/pg_clipfrac_lower:np.float64(1.7377642052451847e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.390625) - self_distillation/token_reweight_w_mean:np.float64(205932.63544678479) - self_distillation/token_reweight_w_std:np.float64(3275553.812427452) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015711192972958) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07417638991319109) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.49010825902223587) - perf/mfu/actor:np.float64(0.04100974753741407) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.18482971191406) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013242431916296482 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013242431916296482 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:442.6171875 - response_length/max:1530.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:442.6171875 - response_length_non_aborted/max:1530.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.875 - prompt_length/max:349.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014469213783740997 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7575890514999628) - timing_s/agent_loop/generate_sequences/max:np.float64(10.56420186534524) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.3792500576455495) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.56420186534524) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:236 - timing_s/agent_loop/slowest/response_length:1530 - timing_s/gen:16.62187496200204 - timing_s/reward:0.0622586477547884 - timing_s/old_log_prob:2.5925257727503777 - timing_s/adv:0.5586317237466574 - timing_s/update_actor:15.944112984463573 - timing_s/step:35.868158884346485 - timing_s/stop_profile:0.00011105090379714966 - timing_per_token_ms/update_actor:0.08815623505470233 - timing_per_token_ms/adv:0.0030887180488253885 - timing_per_token_ms/gen:0.14669380427148565 - perf/total_num_tokens:180862 - perf/time_per_step:35.868158884346485 - perf/throughput:630.3013788049889 (TaskRunner pid=2074004) Training Progress: 18%|█▊ | 18/100 [14:41<59:54, 43.83s/it] (TaskRunner pid=2074004) step:19 - global_seqlen/min:16296 - global_seqlen/max:29019 - global_seqlen/minmax_diff:12723 - global_seqlen/balanced_min:21192 - global_seqlen/balanced_max:21199 - global_seqlen/mean:21195.375 - actor/entropy:0.10421572625637054 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5834612846374512 - training/rollout_probs_diff_mean:0.002985131461173296 - training/rollout_probs_diff_std:0.011980867013335228 - training/rollout_actor_probs_pearson_corr:0.9974684119224548 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.953125 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.953125 - rollout_corr/rollout_is_mean:1.0000929832458496 - rollout_corr/rollout_is_ratio_fraction_high:3.9236074371729046e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010789919906528667 - rollout_corr/rollout_is_max:3.10373592376709 - rollout_corr/rollout_is_min:0.06492556631565094 - rollout_corr/rollout_is_std:0.033075422048568726 - rollout_corr/rollout_is_eff_sample_size:0.9989074497400088 - rollout_corr/rollout_is_seq_mean:1.0000252723693848 - rollout_corr/rollout_is_seq_std:0.0022536139003932476 - rollout_corr/rollout_is_seq_max:1.0099234580993652 - rollout_corr/rollout_is_seq_min:0.9871556758880615 - rollout_corr/rollout_is_seq_max_deviation:0.012844324111938477 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1145465495064855) - rollout_corr/training_log_ppl:np.float64(0.10453275293184561) - rollout_corr/kl:np.float64(0.0007910890604869714) - rollout_corr/k3_kl:np.float64(0.0006869278773535825) - rollout_corr/rollout_ppl:np.float64(1.11366256326437) - rollout_corr/rollout_log_ppl:np.float64(0.10374166368637816) - rollout_corr/log_ppl_diff:np.float64(0.0007910892454674467) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018341713002882898) - rollout_corr/log_ppl_diff_max:np.float64(0.020886391401290894) - rollout_corr/log_ppl_diff_min:np.float64(-0.008934102952480316) - rollout_corr/ppl_ratio:np.float64(1.0007949848659337) - rollout_corr/chi2_token:np.float64(0.0011295359581708908) - rollout_corr/chi2_seq:np.float64(0.5068342476151884) - actor/pg_loss:np.float64(0.004318527295254171) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010698111041165248) - actor/ppo_kl:np.float64(0.00017291589237800054) - actor/pg_clipfrac_lower:np.float64(1.0065717106044758e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(318408.1419896148) - self_distillation/token_reweight_w_std:np.float64(4400530.746190617) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.003058377886191) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06988180117332377) - self_distillation/empty_target_batch:np.float64(0.046875) - actor/grad_norm:np.float64(0.41591396927833557) - perf/mfu/actor:np.float64(0.03846046736330761) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.08876419067383) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0076731047593057156 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0076731047593057156 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:398.23046875 - response_length/max:1107.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:398.23046875 - response_length_non_aborted/max:1107.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.125 - prompt_length/max:339.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.310990571975708e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9409778341650963) - timing_s/agent_loop/generate_sequences/max:np.float64(8.202672267332673) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.057653636649775) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.202672267332673) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:282 - timing_s/agent_loop/slowest/response_length:1064 - timing_s/gen:14.027731182053685 - timing_s/reward:0.059779345989227295 - timing_s/old_log_prob:2.6110658906400204 - timing_s/adv:0.565576946362853 - timing_s/update_actor:15.789859525859356 - timing_s/step:33.141286708414555 - timing_s/stop_profile:0.00012139417231082916 - timing_per_token_ms/update_actor:0.09312090211814698 - timing_per_token_ms/adv:0.003335497404285446 - timing_per_token_ms/gen:0.13759827343672384 - perf/total_num_tokens:169563 - perf/time_per_step:33.141286708414555 - perf/throughput:639.5459291150124 (TaskRunner pid=2074004) Training Progress: 19%|█▉ | 19/100 [15:14<54:51, 40.64s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) We need to solve for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substituting the given values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest answer is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10/actor (TaskRunner pid=2074004) step:20 - global_seqlen/min:16557 - global_seqlen/max:25344 - global_seqlen/minmax_diff:8787 - global_seqlen/balanced_min:20112 - global_seqlen/balanced_max:20126 - global_seqlen/mean:20123.125 - actor/entropy:0.1016768291592598 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7593052387237549 - training/rollout_probs_diff_mean:0.0031040834728628397 - training/rollout_probs_diff_std:0.012476732954382896 - training/rollout_actor_probs_pearson_corr:0.9970778822898865 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.58984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9998236298561096 - rollout_corr/rollout_is_ratio_fraction_high:1.0585034942778293e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012702042295131832 - rollout_corr/rollout_is_max:3.1519858837127686 - rollout_corr/rollout_is_min:0.22677943110466003 - rollout_corr/rollout_is_std:0.0336710549890995 - rollout_corr/rollout_is_eff_sample_size:0.9988670087351662 - rollout_corr/rollout_is_seq_mean:0.9997695684432983 - rollout_corr/rollout_is_seq_std:0.00212579732760787 - rollout_corr/rollout_is_seq_max:1.0059138536453247 - rollout_corr/rollout_is_seq_min:0.9923778176307678 - rollout_corr/rollout_is_seq_max_deviation:0.007622182369232178 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1031243042089045) - rollout_corr/training_log_ppl:np.float64(0.09565349744298146) - rollout_corr/kl:np.float64(0.0009412075490131677) - rollout_corr/k3_kl:np.float64(0.0007612467196338457) - rollout_corr/rollout_ppl:np.float64(1.1020435886457562) - rollout_corr/rollout_log_ppl:np.float64(0.09471228851907654) - rollout_corr/log_ppl_diff:np.float64(0.0009412089239049237) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019292085235065315) - rollout_corr/log_ppl_diff_max:np.float64(0.011152833700180054) - rollout_corr/log_ppl_diff_min:np.float64(-0.005067870020866394) - rollout_corr/ppl_ratio:np.float64(1.000944792991504) - rollout_corr/chi2_token:np.float64(0.001142266672104597) - rollout_corr/chi2_seq:np.float64(0.5473281410522759) - actor/pg_loss:np.float64(0.0028149266727268696) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000719407254791804) - actor/ppo_kl:np.float64(9.32751950681876e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_w_mean:np.float64(257358.8287920449) - self_distillation/token_reweight_w_std:np.float64(3547259.863764613) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000823934096843) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04804125138616655) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.4549396261572838) - perf/mfu/actor:np.float64(0.03647682227242746) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.2781219482422) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6) - val-aux/sciknoweval/reward/std@16:np.float64(0.1854085386899461) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6780125) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15731648141387625) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5234624999999999) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1564494677502724) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6009) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.18569702583933162) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7384999999999999) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11774162185268273) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4601125) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11507313107226662) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.615125) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.15177662910926504) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7845125000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07917893866380923) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.41328750000000003) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07234460911845528) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6239874999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.11172766129349468) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8170249999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04611981709792097) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.38732500000000003) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.0340747398319288) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6269750000000001) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07494075368453937) - val-aux/sciknoweval/score/mean@16:np.float64(0.6) - val-aux/sciknoweval/score/std@16:np.float64(0.1854085386899461) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6780125) - val-aux/sciknoweval/score/best@2/std:np.float64(0.15731648141387625) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5234624999999999) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1564494677502724) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6009) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.18569702583933162) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7384999999999999) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11774162185268273) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4601125) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11507313107226662) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.615125) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.15177662910926504) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7845125000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07917893866380923) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.41328750000000003) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07234460911845528) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6239874999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.11172766129349468) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8170249999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04611981709792097) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.38732500000000003) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.0340747398319288) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6269750000000001) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07494075368453937) - val-core/sciknoweval/acc/mean@16:np.float64(0.6) - val-aux/sciknoweval/acc/std@16:np.float64(0.1854085386899461) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6780125) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.15731648141387625) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5234624999999999) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1564494677502724) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6009) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.18569702583933162) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7384999999999999) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11774162185268273) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4601125) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11507313107226662) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.615125) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.15177662910926504) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7845125000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07917893866380923) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.41328750000000003) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07234460911845528) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6239874999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.11172766129349468) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8170249999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04611981709792097) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.38732500000000003) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.0340747398319288) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6269750000000001) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07494075368453937) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.024206145913796356) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9995875000000002) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.006276698746323264) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9878875) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.03259636494557759) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9940750000000002) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.023588209728692643) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9772874999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.04188475149905875) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9979250000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.01420822710607692) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9603625000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0489078128065511) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9996875) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0053028861809800276) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9365375) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.04813723029580622) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0023961872793734074 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0023961872793734074 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:369.03515625 - response_length/max:1235.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:369.03515625 - response_length_non_aborted/max:1235.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.8125 - prompt_length/max:355.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.98199325799942e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1489538233727217) - timing_s/agent_loop/generate_sequences/max:np.float64(8.91213802434504) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7332966142930673) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.91213802434504) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:279 - timing_s/agent_loop/slowest/response_length:1235 - timing_s/gen:14.779135162010789 - timing_s/reward:0.05472232215106487 - timing_s/old_log_prob:2.653603931888938 - timing_s/adv:0.5555432084947824 - timing_s/update_actor:15.782804861664772 - timing_s/step:33.897329518571496 - timing_s/testing:88.8644837513566 - timing_s/save_checkpoint:6.838149785995483 - timing_s/stop_profile:0.00017092563211917877 - timing_per_token_ms/update_actor:0.09803897792753842 - timing_per_token_ms/adv:0.0034509004472142277 - timing_per_token_ms/gen:0.15643766115197769 - perf/total_num_tokens:160985 - perf/time_per_step:33.897329518571496 - perf/throughput:593.649272252407 (TaskRunner pid=2074004) Training Progress: 20%|██ | 20/100 [17:24<1:29:48, 67.36s/it] (TaskRunner pid=2074004) step:21 - global_seqlen/min:18517 - global_seqlen/max:25867 - global_seqlen/minmax_diff:7350 - global_seqlen/balanced_min:21107 - global_seqlen/balanced_max:21412 - global_seqlen/mean:21162.875 - actor/entropy:0.09419602900743484 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5948364734649658 - training/rollout_probs_diff_mean:0.002783035859465599 - training/rollout_probs_diff_std:0.011772817932069302 - training/rollout_actor_probs_pearson_corr:0.9972338080406189 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.6640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.0000512599945068 - rollout_corr/rollout_is_ratio_fraction_high:1.9861960026901215e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012910272926092148 - rollout_corr/rollout_is_max:2.5543158054351807 - rollout_corr/rollout_is_min:0.1329210251569748 - rollout_corr/rollout_is_std:0.03238707408308983 - rollout_corr/rollout_is_eff_sample_size:0.9989522955064933 - rollout_corr/rollout_is_seq_mean:1.0000442266464233 - rollout_corr/rollout_is_seq_std:0.0020267271902412176 - rollout_corr/rollout_is_seq_max:1.0177290439605713 - rollout_corr/rollout_is_seq_min:0.9942728877067566 - rollout_corr/rollout_is_seq_max_deviation:0.01772904396057129 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0939804124645889) - rollout_corr/training_log_ppl:np.float64(0.08795880044999649) - rollout_corr/kl:np.float64(0.0004990964827804589) - rollout_corr/k3_kl:np.float64(0.0006357246575703357) - rollout_corr/rollout_ppl:np.float64(1.093429901637137) - rollout_corr/rollout_log_ppl:np.float64(0.08745970321979257) - rollout_corr/log_ppl_diff:np.float64(0.0004990972302039154) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015424156808876432) - rollout_corr/log_ppl_diff_max:np.float64(0.005901060998439789) - rollout_corr/log_ppl_diff_min:np.float64(-0.0117645263671875) - rollout_corr/ppl_ratio:np.float64(1.000501276459545) - rollout_corr/chi2_token:np.float64(0.0017355745658278465) - rollout_corr/chi2_seq:np.float64(0.9698769934475422) - actor/pg_loss:np.float64(0.0027807088335976005) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005495589271049539) - actor/ppo_kl:np.float64(-4.407575040854539e-05) - actor/pg_clipfrac_lower:np.float64(1.0252624633722007e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6640625) - self_distillation/token_reweight_w_mean:np.float64(141096.08209011448) - self_distillation/token_reweight_w_std:np.float64(1977869.731875727) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000101735349745) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04118481201294344) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.3234409838914871) - perf/mfu/actor:np.float64(0.03855656297649644) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.51013565063477) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002361090388149023 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002361090388149023 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:393.33984375 - response_length/max:1529.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:393.33984375 - response_length_non_aborted/max:1529.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.0 - prompt_length/max:364.0 - prompt_length/min:195.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015509501099586487 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4505607336759567) - timing_s/agent_loop/generate_sequences/max:np.float64(10.29461688362062) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.021393945564341) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.29461688362062) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1529 - timing_s/gen:15.94972218759358 - timing_s/reward:0.0574874933809042 - timing_s/old_log_prob:2.583099478855729 - timing_s/adv:0.6336547415703535 - timing_s/update_actor:15.871118316426873 - timing_s/step:35.191847467795014 - timing_s/stop_profile:0.00011814944446086884 - timing_per_token_ms/update_actor:0.09374386937282195 - timing_per_token_ms/adv:0.003742726009405347 - timing_per_token_ms/gen:0.15839636712442107 - perf/total_num_tokens:169303 - perf/time_per_step:35.191847467795014 - perf/throughput:601.3573177528319 (TaskRunner pid=2074004) Training Progress: 21%|██ | 21/100 [17:59<1:16:00, 57.72s/it] (TaskRunner pid=2074004) step:22 - global_seqlen/min:15901 - global_seqlen/max:25415 - global_seqlen/minmax_diff:9514 - global_seqlen/balanced_min:20195 - global_seqlen/balanced_max:20210 - global_seqlen/mean:20203.0 - actor/entropy:0.10593846440315247 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5313484072685242 - training/rollout_probs_diff_mean:0.003251798916608095 - training/rollout_probs_diff_std:0.012875033542513847 - training/rollout_actor_probs_pearson_corr:0.9971089959144592 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.00010347366333 - rollout_corr/rollout_is_ratio_fraction_high:4.197447924525477e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010493619629414752 - rollout_corr/rollout_is_max:2.9541661739349365 - rollout_corr/rollout_is_min:0.09866257756948471 - rollout_corr/rollout_is_std:0.03540460020303726 - rollout_corr/rollout_is_eff_sample_size:0.9987482024906889 - rollout_corr/rollout_is_seq_mean:1.0001643896102905 - rollout_corr/rollout_is_seq_std:0.0024298520293086767 - rollout_corr/rollout_is_seq_max:1.0121618509292603 - rollout_corr/rollout_is_seq_min:0.9910787343978882 - rollout_corr/rollout_is_seq_max_deviation:0.012161850929260254 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1144308936782181) - rollout_corr/training_log_ppl:np.float64(0.10470682490995387) - rollout_corr/kl:np.float64(0.000816442931922623) - rollout_corr/k3_kl:np.float64(0.0008207170265279728) - rollout_corr/rollout_ppl:np.float64(1.1135047189891338) - rollout_corr/rollout_log_ppl:np.float64(0.10389038186986) - rollout_corr/log_ppl_diff:np.float64(0.0008164430400938727) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018727829810814) - rollout_corr/log_ppl_diff_max:np.float64(0.0158127099275589) - rollout_corr/log_ppl_diff_min:np.float64(-0.007389448583126068) - rollout_corr/ppl_ratio:np.float64(1.0008202025201172) - rollout_corr/chi2_token:np.float64(0.0015751214232295752) - rollout_corr/chi2_seq:np.float64(0.42517587705515325) - actor/pg_loss:np.float64(0.0036053532967343926) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010071134870486276) - actor/ppo_kl:np.float64(0.00023022429938990285) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(218233.54828879703) - self_distillation/token_reweight_w_std:np.float64(3210707.643729917) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001665785908699) - self_distillation/token_reweight_w_clip_frac:np.float64(0.058951567800249904) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.4596095457673073) - perf/mfu/actor:np.float64(0.036942325232477896) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.46227645874023) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006771008018404245 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006771008018404245 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:372.25 - response_length/max:1292.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:372.25 - response_length_non_aborted/max:1292.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.09375 - prompt_length/max:375.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001899413764476776 - timing_s/agent_loop/generate_sequences/min:np.float64(1.149434920400381) - timing_s/agent_loop/generate_sequences/max:np.float64(8.853524098172784) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8520722229222883) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.853524098172784) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:1292 - timing_s/gen:14.650895247235894 - timing_s/reward:0.04885207116603851 - timing_s/old_log_prob:2.5285760313272476 - timing_s/adv:0.5984222404658794 - timing_s/update_actor:15.83981061168015 - timing_s/step:33.70206049270928 - timing_s/stop_profile:0.00015066377818584442 - timing_per_token_ms/update_actor:0.09800407496213527 - timing_per_token_ms/adv:0.0037025580388177462 - timing_per_token_ms/gen:0.15374092561320407 - perf/total_num_tokens:161624 - perf/time_per_step:33.70206049270928 - perf/throughput:599.4588967155432 (TaskRunner pid=2074004) Training Progress: 22%|██▏ | 22/100 [18:33<1:05:41, 50.53s/it] (TaskRunner pid=2074004) step:23 - global_seqlen/min:16336 - global_seqlen/max:30208 - global_seqlen/minmax_diff:13872 - global_seqlen/balanced_min:23332 - global_seqlen/balanced_max:23529 - global_seqlen/mean:23365.625 - actor/entropy:0.13051186501979828 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.46211716532707214 - training/rollout_probs_diff_mean:0.003518641460686922 - training/rollout_probs_diff_std:0.012740753591060638 - training/rollout_actor_probs_pearson_corr:0.9975020289421082 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.44140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:0.9999224543571472 - rollout_corr/rollout_is_ratio_fraction_high:2.5563025701558217e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013633613707497716 - rollout_corr/rollout_is_max:3.358520746231079 - rollout_corr/rollout_is_min:0.2698378264904022 - rollout_corr/rollout_is_std:0.03545843064785004 - rollout_corr/rollout_is_eff_sample_size:0.9987440406221719 - rollout_corr/rollout_is_seq_mean:0.9998674988746643 - rollout_corr/rollout_is_seq_std:0.002211816841736436 - rollout_corr/rollout_is_seq_max:1.0098694562911987 - rollout_corr/rollout_is_seq_min:0.9933016896247864 - rollout_corr/rollout_is_seq_max_deviation:0.00986945629119873 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1375553528778255) - rollout_corr/training_log_ppl:np.float64(0.12625176343863131) - rollout_corr/kl:np.float64(0.0009684273467929216) - rollout_corr/k3_kl:np.float64(0.0008128368172322098) - rollout_corr/rollout_ppl:np.float64(1.1364499349147081) - rollout_corr/rollout_log_ppl:np.float64(0.12528333482623566) - rollout_corr/log_ppl_diff:np.float64(0.0009684286123956554) - rollout_corr/log_ppl_abs_diff:np.float64(0.001832397589168977) - rollout_corr/log_ppl_diff_max:np.float64(0.011887356638908386) - rollout_corr/log_ppl_diff_min:np.float64(-0.008548229932785034) - rollout_corr/ppl_ratio:np.float64(1.000971759436652) - rollout_corr/chi2_token:np.float64(0.0012992981355637312) - rollout_corr/chi2_seq:np.float64(0.5191449248231947) - actor/pg_loss:np.float64(0.005724000162445009) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016155378229996131) - actor/ppo_kl:np.float64(0.00011277476574633738) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.44140625) - self_distillation/token_reweight_w_mean:np.float64(304747.9913651231) - self_distillation/token_reweight_w_std:np.float64(4644227.323464029) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019670443143696) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08922295937372837) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.5448018610477448) - perf/mfu/actor:np.float64(0.04195048253218733) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.12269973754883) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:1 - critic/score/mean:0.44140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004490571562200785 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004490571562200785 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:458.42578125 - response_length/max:1832.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:458.42578125 - response_length_non_aborted/max:1832.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.75 - prompt_length/max:375.0 - prompt_length/min:203.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0005329102277755737 - timing_s/agent_loop/generate_sequences/min:np.float64(1.116574076935649) - timing_s/agent_loop/generate_sequences/max:np.float64(11.822160206735134) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.7243306759992265) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.822160206735134) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:1832 - timing_s/gen:18.11809141561389 - timing_s/reward:0.05629480630159378 - timing_s/old_log_prob:2.732456060126424 - timing_s/adv:0.6987331956624985 - timing_s/update_actor:15.840048659592867 - timing_s/step:37.55766446888447 - timing_s/stop_profile:6.579048931598663e-05 - timing_per_token_ms/update_actor:0.08474012924752101 - timing_per_token_ms/adv:0.00373804036732646 - timing_per_token_ms/gen:0.154384411800011 - perf/total_num_tokens:186925 - perf/time_per_step:37.55766446888447 - perf/throughput:622.1266772154537 (TaskRunner pid=2074004) Training Progress: 23%|██▎ | 23/100 [19:11<59:59, 46.75s/it] (TaskRunner pid=2074004) step:24 - global_seqlen/min:17135 - global_seqlen/max:25434 - global_seqlen/minmax_diff:8299 - global_seqlen/balanced_min:20734 - global_seqlen/balanced_max:20748 - global_seqlen/mean:20742.75 - actor/entropy:0.10690584033727646 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3519520163536072 - training/rollout_probs_diff_mean:0.0030989660881459713 - training/rollout_probs_diff_std:0.012050447054207325 - training/rollout_actor_probs_pearson_corr:0.9973185062408447 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.76171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.0002549886703491 - rollout_corr/rollout_is_ratio_fraction_high:2.0413579477462918e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.165431790985167e-05 - rollout_corr/rollout_is_max:2.1403613090515137 - rollout_corr/rollout_is_min:0.21778209507465363 - rollout_corr/rollout_is_std:0.033476658165454865 - rollout_corr/rollout_is_eff_sample_size:0.9988811627143783 - rollout_corr/rollout_is_seq_mean:1.0003387928009033 - rollout_corr/rollout_is_seq_std:0.0020440113730728626 - rollout_corr/rollout_is_seq_max:1.0078518390655518 - rollout_corr/rollout_is_seq_min:0.9929835200309753 - rollout_corr/rollout_is_seq_max_deviation:0.007851839065551758 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.109124246519059) - rollout_corr/training_log_ppl:np.float64(0.10045992501545697) - rollout_corr/kl:np.float64(0.00014452066502457228) - rollout_corr/k3_kl:np.float64(0.0006306728797227379) - rollout_corr/rollout_ppl:np.float64(1.1089712120592594) - rollout_corr/rollout_log_ppl:np.float64(0.10031540389900329) - rollout_corr/log_ppl_diff:np.float64(0.0001445211164536886) - rollout_corr/log_ppl_abs_diff:np.float64(0.001574805770360399) - rollout_corr/log_ppl_diff_max:np.float64(0.011304967105388641) - rollout_corr/log_ppl_diff_min:np.float64(-0.011325351893901825) - rollout_corr/ppl_ratio:np.float64(1.000147320330143) - rollout_corr/chi2_token:np.float64(0.0022659352980554104) - rollout_corr/chi2_seq:np.float64(0.813906122231856) - actor/pg_loss:np.float64(0.002153943874873221) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006025306415722298) - actor/ppo_kl:np.float64(-0.00010819188584676454) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_w_mean:np.float64(135130.44865711615) - self_distillation/token_reweight_w_std:np.float64(2006584.000496325) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000246322248131) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04379224638978485) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.33335286378860474) - perf/mfu/actor:np.float64(0.037579949707391716) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.17943572998047) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:1 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005197807680815458 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005197807680815458 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:382.7109375 - response_length/max:1139.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:382.7109375 - response_length_non_aborted/max:1139.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.5 - prompt_length/max:383.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.317960262298584e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8356650900095701) - timing_s/agent_loop/generate_sequences/max:np.float64(8.357717419043183) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8592212262228713) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.357717419043183) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:353 - timing_s/agent_loop/slowest/response_length:1139 - timing_s/gen:14.09934650734067 - timing_s/reward:0.05855764448642731 - timing_s/old_log_prob:2.4800929073244333 - timing_s/adv:0.5536916963756084 - timing_s/update_actor:15.75769667699933 - timing_s/step:33.040523858740926 - timing_s/stop_profile:0.0001149345189332962 - timing_per_token_ms/update_actor:0.0949590620638496 - timing_per_token_ms/adv:0.003336657967094578 - timing_per_token_ms/gen:0.1439090626833718 - perf/total_num_tokens:165942 - perf/time_per_step:33.040523858740926 - perf/throughput:627.7972494831516 (TaskRunner pid=2074004) Training Progress: 24%|██▍ | 24/100 [19:44<54:01, 42.65s/it] (TaskRunner pid=2074004) step:25 - global_seqlen/min:15092 - global_seqlen/max:31297 - global_seqlen/minmax_diff:16205 - global_seqlen/balanced_min:21652 - global_seqlen/balanced_max:28332 - global_seqlen/mean:22490.0 - actor/entropy:0.10956007242202759 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6190921068191528 - training/rollout_probs_diff_mean:0.0033610411919653416 - training/rollout_probs_diff_std:0.013345292769372463 - training/rollout_actor_probs_pearson_corr:0.9969186782836914 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.65625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.000109314918518 - rollout_corr/rollout_is_ratio_fraction_high:3.6875877412967384e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.531381248147227e-05 - rollout_corr/rollout_is_max:2.4000637531280518 - rollout_corr/rollout_is_min:0.031893547624349594 - rollout_corr/rollout_is_std:0.03650045394897461 - rollout_corr/rollout_is_eff_sample_size:0.9986697273810201 - rollout_corr/rollout_is_seq_mean:1.0001757144927979 - rollout_corr/rollout_is_seq_std:0.002466355450451374 - rollout_corr/rollout_is_seq_max:1.009514331817627 - rollout_corr/rollout_is_seq_min:0.9879665374755859 - rollout_corr/rollout_is_seq_max_deviation:0.012033462524414062 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1296572694554925) - rollout_corr/training_log_ppl:np.float64(0.11874415549391415) - rollout_corr/kl:np.float64(0.0007365228179097016) - rollout_corr/k3_kl:np.float64(0.000829679726990662) - rollout_corr/rollout_ppl:np.float64(1.1288006519898772) - rollout_corr/rollout_log_ppl:np.float64(0.11800763171049766) - rollout_corr/log_ppl_diff:np.float64(0.0007365237834164873) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019228525488870218) - rollout_corr/log_ppl_diff_max:np.float64(0.01413746178150177) - rollout_corr/log_ppl_diff_min:np.float64(-0.0073609501123428345) - rollout_corr/ppl_ratio:np.float64(1.0007400074973702) - rollout_corr/chi2_token:np.float64(0.001852479763329029) - rollout_corr/chi2_seq:np.float64(0.793041022028774) - actor/pg_loss:np.float64(0.0035626995377242565) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000992866120668623) - actor/ppo_kl:np.float64(0.00013374969253865387) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_w_mean:np.float64(210967.00421022926) - self_distillation/token_reweight_w_std:np.float64(2976356.6710397173) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001523660030216) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0665886076312745) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.4494774639606476) - perf/mfu/actor:np.float64(0.03848721918082872) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.24299240112305) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:1 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.052680645138025284 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.052680645138025284 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:423.71875 - response_length/max:8192.0 - response_length/min:75.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:423.71875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:75.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:279.09375 - prompt_length/max:365.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015040487051010132 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8068552948534489) - timing_s/agent_loop/generate_sequences/max:np.float64(50.265202172100544) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.223772806173656) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.265202172100544) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:251 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.000157160684466 - timing_s/reward:0.062263449653983116 - timing_s/old_log_prob:2.6894225608557463 - timing_s/adv:0.575139531865716 - timing_s/update_actor:16.96509013324976 - timing_s/step:76.38079810328782 - timing_s/stop_profile:0.00012892484664916992 - timing_per_token_ms/update_actor:0.09429240847737749 - timing_per_token_ms/adv:0.003196640350520876 - timing_per_token_ms/gen:0.5162637100881745 - perf/total_num_tokens:179920 - perf/time_per_step:76.38079810328782 - perf/throughput:294.44573189176873 (TaskRunner pid=2074004) Training Progress: 25%|██▌ | 25/100 [21:00<1:05:59, 52.79s/it] (TaskRunner pid=2074004) step:26 - global_seqlen/min:17856 - global_seqlen/max:22321 - global_seqlen/minmax_diff:4465 - global_seqlen/balanced_min:19619 - global_seqlen/balanced_max:19629 - global_seqlen/mean:19624.625 - actor/entropy:0.10051669180393219 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30771172046661377 - training/rollout_probs_diff_mean:0.0031875635031610727 - training/rollout_probs_diff_std:0.012896761298179626 - training/rollout_actor_probs_pearson_corr:0.9968799948692322 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90625 - self_distillation/correct_sample_fraction:0.76171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90625 - rollout_corr/rollout_is_mean:1.0000146627426147 - rollout_corr/rollout_is_ratio_fraction_high:2.208553814853076e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.93849171209149e-05 - rollout_corr/rollout_is_max:2.9205586910247803 - rollout_corr/rollout_is_min:0.3763677477836609 - rollout_corr/rollout_is_std:0.03457672521471977 - rollout_corr/rollout_is_eff_sample_size:0.9988058776680073 - rollout_corr/rollout_is_seq_mean:1.0000602006912231 - rollout_corr/rollout_is_seq_std:0.0020706949289888144 - rollout_corr/rollout_is_seq_max:1.0055534839630127 - rollout_corr/rollout_is_seq_min:0.991748571395874 - rollout_corr/rollout_is_seq_max_deviation:0.008251428604125977 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0977428681217134) - rollout_corr/training_log_ppl:np.float64(0.09058145447852439) - rollout_corr/kl:np.float64(0.0006791098424780984) - rollout_corr/k3_kl:np.float64(0.0007086409321068743) - rollout_corr/rollout_ppl:np.float64(1.0969782588072121) - rollout_corr/rollout_log_ppl:np.float64(0.0899023435304116) - rollout_corr/log_ppl_diff:np.float64(0.0006791109481127933) - rollout_corr/log_ppl_abs_diff:np.float64(0.001920343165693339) - rollout_corr/log_ppl_diff_max:np.float64(0.024022668600082397) - rollout_corr/log_ppl_diff_min:np.float64(-0.005489669740200043) - rollout_corr/ppl_ratio:np.float64(1.0006832026410848) - rollout_corr/chi2_token:np.float64(0.001474759541451931) - rollout_corr/chi2_seq:np.float64(0.6005796948447824) - actor/pg_loss:np.float64(0.003621779615059495) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007727315569354687) - actor/ppo_kl:np.float64(0.000100072484177538) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_w_mean:np.float64(201738.92587617878) - self_distillation/token_reweight_w_std:np.float64(2411880.3968902696) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.00004991912283) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04766093760554213) - self_distillation/empty_target_batch:np.float64(0.09375) - actor/grad_norm:np.float64(0.45207298547029495) - perf/mfu/actor:np.float64(0.03555715209782605) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.1187286376953) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:1 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-5.383349707699381e-05 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-5.383349707699381e-05 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:353.73828125 - response_length/max:1273.0 - response_length/min:79.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:353.73828125 - response_length_non_aborted/max:1273.0 - response_length_non_aborted/min:79.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.53125 - prompt_length/max:363.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011607818305492401 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5807047858834267) - timing_s/agent_loop/generate_sequences/max:np.float64(8.703462390229106) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3116236440982902) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.703462390229106) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:236 - timing_s/agent_loop/slowest/response_length:1273 - timing_s/gen:14.44053604081273 - timing_s/reward:0.058792874217033386 - timing_s/old_log_prob:2.4093342386186123 - timing_s/adv:0.5303562916815281 - timing_s/update_actor:15.99789859354496 - timing_s/step:33.523452650755644 - timing_s/stop_profile:2.9999762773513794e-05 - timing_per_token_ms/update_actor:0.10189939039309642 - timing_per_token_ms/adv:0.0033781301023683773 - timing_per_token_ms/gen:0.15946349857893627 - perf/total_num_tokens:156997 - perf/time_per_step:33.523452650755644 - perf/throughput:585.3998752589001 (TaskRunner pid=2074004) Training Progress: 26%|██▌ | 26/100 [21:34<57:59, 47.02s/it] (TaskRunner pid=2074004) step:27 - global_seqlen/min:19536 - global_seqlen/max:27649 - global_seqlen/minmax_diff:8113 - global_seqlen/balanced_min:24156 - global_seqlen/balanced_max:24592 - global_seqlen/mean:24227.0 - actor/entropy:0.08968006819486618 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35172876715660095 - training/rollout_probs_diff_mean:0.0026838507037609816 - training/rollout_probs_diff_std:0.01177211757749319 - training/rollout_actor_probs_pearson_corr:0.9971114993095398 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.96484375 - self_distillation/correct_sample_fraction:0.73046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.96484375 - rollout_corr/rollout_is_mean:1.000118613243103 - rollout_corr/rollout_is_ratio_fraction_high:3.249707515351474e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.68698815186508e-05 - rollout_corr/rollout_is_max:9.403421401977539 - rollout_corr/rollout_is_min:0.415858656167984 - rollout_corr/rollout_is_std:0.03169693797826767 - rollout_corr/rollout_is_eff_sample_size:0.9989963125083661 - rollout_corr/rollout_is_seq_mean:1.0000813007354736 - rollout_corr/rollout_is_seq_std:0.0021606753580272198 - rollout_corr/rollout_is_seq_max:1.014573335647583 - rollout_corr/rollout_is_seq_min:0.991974413394928 - rollout_corr/rollout_is_seq_max_deviation:0.014573335647583008 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0907162227667868) - rollout_corr/training_log_ppl:np.float64(0.08525840124275419) - rollout_corr/kl:np.float64(0.0006036972056477907) - rollout_corr/k3_kl:np.float64(0.0006888124125836725) - rollout_corr/rollout_ppl:np.float64(1.0900488472543657) - rollout_corr/rollout_log_ppl:np.float64(0.08465470290684607) - rollout_corr/log_ppl_diff:np.float64(0.0006036983359081205) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015368564454547595) - rollout_corr/log_ppl_diff_max:np.float64(0.013522587716579437) - rollout_corr/log_ppl_diff_min:np.float64(-0.0061819590628147125) - rollout_corr/ppl_ratio:np.float64(1.0006064060144126) - rollout_corr/chi2_token:np.float64(0.0026917706709355116) - rollout_corr/chi2_seq:np.float64(5.136045180493966) - actor/pg_loss:np.float64(0.0036772675812244415) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007991504742221878) - actor/ppo_kl:np.float64(0.00012214449816383421) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.96484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.96484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_w_mean:np.float64(157488.76363183185) - self_distillation/token_reweight_w_std:np.float64(2501365.0777522195) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000293314224109) - self_distillation/token_reweight_w_clip_frac:np.float64(0.055393004906363785) - self_distillation/empty_target_batch:np.float64(0.03515625) - actor/grad_norm:np.float64(0.4007357805967331) - perf/mfu/actor:np.float64(0.043591956964482634) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.22241592407227) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:1 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009190579294227064 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009190579294227064 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:480.8125 - response_length/max:2125.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:480.8125 - response_length_non_aborted/max:2125.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.28125 - prompt_length/max:375.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.861744284629822e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1442433521151543) - timing_s/agent_loop/generate_sequences/max:np.float64(13.537522902712226) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.885370632960985) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.537522902712226) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:2125 - timing_s/gen:19.567774968221784 - timing_s/reward:0.06262494809925556 - timing_s/old_log_prob:2.4061235673725605 - timing_s/adv:0.5981502141803503 - timing_s/update_actor:16.023667991161346 - timing_s/step:38.744677659124136 - timing_s/stop_profile:0.00011847354471683502 - timing_per_token_ms/update_actor:0.08267463981901053 - timing_per_token_ms/adv:0.0030861756211063604 - timing_per_token_ms/gen:0.15897386396904478 - perf/total_num_tokens:193816 - perf/time_per_step:38.744677659124136 - perf/throughput:625.2987884722971 (TaskRunner pid=2074004) Training Progress: 27%|██▋ | 27/100 [22:13<54:12, 44.55s/it] (TaskRunner pid=2074004) step:28 - global_seqlen/min:15689 - global_seqlen/max:28335 - global_seqlen/minmax_diff:12646 - global_seqlen/balanced_min:22049 - global_seqlen/balanced_max:22067 - global_seqlen/mean:22059.625 - actor/entropy:0.11999335139989853 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3807188868522644 - training/rollout_probs_diff_mean:0.00325206876732409 - training/rollout_probs_diff_std:0.012448713183403015 - training/rollout_actor_probs_pearson_corr:0.9974731802940369 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.0001393556594849 - rollout_corr/rollout_is_ratio_fraction_high:5.649025843013078e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.590530392713845e-05 - rollout_corr/rollout_is_max:3.7063121795654297 - rollout_corr/rollout_is_min:0.3865959644317627 - rollout_corr/rollout_is_std:0.03557758033275604 - rollout_corr/rollout_is_eff_sample_size:0.998736192621627 - rollout_corr/rollout_is_seq_mean:1.0002413988113403 - rollout_corr/rollout_is_seq_std:0.002391271060332656 - rollout_corr/rollout_is_seq_max:1.0125643014907837 - rollout_corr/rollout_is_seq_min:0.9939977526664734 - rollout_corr/rollout_is_seq_max_deviation:0.012564301490783691 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1245770892128348) - rollout_corr/training_log_ppl:np.float64(0.1141112526966026) - rollout_corr/kl:np.float64(0.0005681062639624201) - rollout_corr/k3_kl:np.float64(0.0007897823702087692) - rollout_corr/rollout_ppl:np.float64(1.1239187237806618) - rollout_corr/rollout_log_ppl:np.float64(0.11354314523669018) - rollout_corr/log_ppl_diff:np.float64(0.0005681074599124258) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016994898305711104) - rollout_corr/log_ppl_diff_max:np.float64(0.01184430718421936) - rollout_corr/log_ppl_diff_min:np.float64(-0.006551269441843033) - rollout_corr/ppl_ratio:np.float64(1.0005709801334888) - rollout_corr/chi2_token:np.float64(0.002181260846555233) - rollout_corr/chi2_seq:np.float64(0.662524172803387) - actor/pg_loss:np.float64(0.002658500336110592) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007556931734598038) - actor/ppo_kl:np.float64(8.9224213090211e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(108684.6212300947) - self_distillation/token_reweight_w_std:np.float64(1726174.4731562997) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000196180306375) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04746657198120374) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.3256382420659065) - perf/mfu/actor:np.float64(0.039955600042607634) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.3211898803711) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:1 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.012464340776205063 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.012464340776205063 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:414.89453125 - response_length/max:1399.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:414.89453125 - response_length_non_aborted/max:1399.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.46875 - prompt_length/max:363.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016648322343826294 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5349066201597452) - timing_s/agent_loop/generate_sequences/max:np.float64(9.329830139875412) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.712864897781401) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.329830139875412) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:302 - timing_s/agent_loop/slowest/response_length:1399 - timing_s/gen:15.155654072761536 - timing_s/reward:0.061105452477931976 - timing_s/old_log_prob:2.4498131908476353 - timing_s/adv:0.594944417476654 - timing_s/update_actor:15.982728907838464 - timing_s/step:34.33417861536145 - timing_s/stop_profile:0.00012039020657539368 - timing_per_token_ms/update_actor:0.09056550659767824 - timing_per_token_ms/adv:0.003371229211039705 - timing_per_token_ms/gen:0.14269114018775042 - perf/total_num_tokens:176477 - perf/time_per_step:34.33417861536145 - perf/throughput:642.4975313121457 (TaskRunner pid=2074004) Training Progress: 28%|██▊ | 28/100 [22:47<49:47, 41.50s/it] (TaskRunner pid=2074004) step:29 - global_seqlen/min:18998 - global_seqlen/max:31893 - global_seqlen/minmax_diff:12895 - global_seqlen/balanced_min:22424 - global_seqlen/balanced_max:28754 - global_seqlen/mean:23272.0 - actor/entropy:0.09004729986190796 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6166199445724487 - training/rollout_probs_diff_mean:0.0027836794033646584 - training/rollout_probs_diff_std:0.011670615524053574 - training/rollout_actor_probs_pearson_corr:0.9972294569015503 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.70703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:0.9999050498008728 - rollout_corr/rollout_is_ratio_fraction_high:1.6847496226546355e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.423747931374237e-05 - rollout_corr/rollout_is_max:2.463893175125122 - rollout_corr/rollout_is_min:0.2046324759721756 - rollout_corr/rollout_is_std:0.03296531364321709 - rollout_corr/rollout_is_eff_sample_size:0.9989144677772216 - rollout_corr/rollout_is_seq_mean:0.9998564124107361 - rollout_corr/rollout_is_seq_std:0.0022072712890803814 - rollout_corr/rollout_is_seq_max:1.006649136543274 - rollout_corr/rollout_is_seq_min:0.99166339635849 - rollout_corr/rollout_is_seq_max_deviation:0.00833660364151001 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1013299119658768) - rollout_corr/training_log_ppl:np.float64(0.09491147637072572) - rollout_corr/kl:np.float64(0.0012193648983345184) - rollout_corr/k3_kl:np.float64(0.0009733567474512483) - rollout_corr/rollout_ppl:np.float64(1.099963684566319) - rollout_corr/rollout_log_ppl:np.float64(0.09369211093780905) - rollout_corr/log_ppl_diff:np.float64(0.0012193654329166748) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020960377587471157) - rollout_corr/log_ppl_diff_max:np.float64(0.05848722159862518) - rollout_corr/log_ppl_diff_min:np.float64(-0.007673636078834534) - rollout_corr/ppl_ratio:np.float64(1.001229693647474) - rollout_corr/chi2_token:np.float64(0.0014654879923909903) - rollout_corr/chi2_seq:np.float64(0.6356173786334693) - actor/pg_loss:np.float64(0.003317820141091943) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006650509092196444) - actor/ppo_kl:np.float64(0.0003957474698959018) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_w_mean:np.float64(62520.58305266849) - self_distillation/token_reweight_w_std:np.float64(1281642.6764327008) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001606633886695) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03943118227471132) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.3942992687225342) - perf/mfu/actor:np.float64(0.039518949200975054) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.29967880249023) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:1 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0459052175283432 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0459052175283432 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:463.71875 - response_length/max:8192.0 - response_length/min:80.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:463.71875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:263.53125 - prompt_length/max:375.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.063258767127991e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0605478826910257) - timing_s/agent_loop/generate_sequences/max:np.float64(49.57040260732174) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.396626845227729) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.57040260732174) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:375 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.50206731632352 - timing_s/reward:0.05697246454656124 - timing_s/old_log_prob:2.633130395784974 - timing_s/adv:0.6030823420733213 - timing_s/update_actor:17.1904456615448 - timing_s/step:76.02233720384538 - timing_s/stop_profile:0.00010943599045276642 - timing_per_token_ms/update_actor:0.09233438070183482 - timing_per_token_ms/adv:0.003239313026777465 - timing_per_token_ms/gen:0.46753544137343755 - perf/total_num_tokens:186176 - perf/time_per_step:76.02233720384538 - perf/throughput:306.1205542470858 (TaskRunner pid=2074004) Training Progress: 29%|██▉ | 29/100 [24:03<1:01:22, 51.87s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) The electric field $ E $ due to a point charge $ q $ at a distance $ r $ is given by: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) We are given: (TaskRunner pid=2074004) - $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2074004) - $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2074004) - $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $ (TaskRunner pid=2074004) (TaskRunner pid=2074004) We need to solve for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substitute the values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest option is **B: 56 pC**. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20/actor (TaskRunner pid=2074004) step:30 - global_seqlen/min:16055 - global_seqlen/max:23382 - global_seqlen/minmax_diff:7327 - global_seqlen/balanced_min:20479 - global_seqlen/balanced_max:20784 - global_seqlen/mean:20529.125 - actor/entropy:0.11640558391809464 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8577249050140381 - training/rollout_probs_diff_mean:0.0033130382653325796 - training/rollout_probs_diff_std:0.013030188158154488 - training/rollout_actor_probs_pearson_corr:0.9971984624862671 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.6640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:1.0003026723861694 - rollout_corr/rollout_is_ratio_fraction_high:6.133902934379876e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010223171557299793 - rollout_corr/rollout_is_max:9.797200202941895 - rollout_corr/rollout_is_min:0.02608746662735939 - rollout_corr/rollout_is_std:0.03469718620181084 - rollout_corr/rollout_is_eff_sample_size:0.9987981476185248 - rollout_corr/rollout_is_seq_mean:1.0002856254577637 - rollout_corr/rollout_is_seq_std:0.0022839626763015985 - rollout_corr/rollout_is_seq_max:1.0106697082519531 - rollout_corr/rollout_is_seq_min:0.992483377456665 - rollout_corr/rollout_is_seq_max_deviation:0.010669708251953125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.116787108592689) - rollout_corr/training_log_ppl:np.float64(0.10795232591772219) - rollout_corr/kl:np.float64(0.000616683089219805) - rollout_corr/k3_kl:np.float64(0.0007727639311809753) - rollout_corr/rollout_ppl:np.float64(1.1160784345120192) - rollout_corr/rollout_log_ppl:np.float64(0.10733564237307291) - rollout_corr/log_ppl_diff:np.float64(0.0006166835446492769) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018158721577492543) - rollout_corr/log_ppl_diff_max:np.float64(0.01455879956483841) - rollout_corr/log_ppl_diff_min:np.float64(-0.007083818316459656) - rollout_corr/ppl_ratio:np.float64(1.000619996106252) - rollout_corr/chi2_token:np.float64(0.0019336091354489326) - rollout_corr/chi2_seq:np.float64(1.0694307961966842) - actor/pg_loss:np.float64(0.002601322717964649) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00044001030278195685) - actor/ppo_kl:np.float64(0.00017719261101882544) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6640625) - self_distillation/token_reweight_w_mean:np.float64(32765.776810475392) - self_distillation/token_reweight_w_std:np.float64(708157.8602350631) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9986124283168465) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04477454177686013) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.21477733738720417) - perf/mfu/actor:np.float64(0.03686629124421728) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.2654571533203) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.62421875) - val-aux/sciknoweval/reward/std@16:np.float64(0.18777951588879277) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.699725) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.14925101134863997) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5497249999999999) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1686963837105089) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6241125000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1881880107415588) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7547625) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.10614923952325535) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.48316250000000005) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1385217733706531) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6387875) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.14554909217773004) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7954375) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07429913902055749) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4257375) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10329608095410872) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.646375) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10428943394865786) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8249500000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04785468179260445) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.38275000000000003) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06501528454070343) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6504875) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07057889100574483) - val-aux/sciknoweval/score/mean@16:np.float64(0.62421875) - val-aux/sciknoweval/score/std@16:np.float64(0.18777951588879277) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.699725) - val-aux/sciknoweval/score/best@2/std:np.float64(0.14925101134863997) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5497249999999999) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1686963837105089) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6241125000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1881880107415588) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7547625) - val-aux/sciknoweval/score/best@4/std:np.float64(0.10614923952325535) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.48316250000000005) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1385217733706531) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6387875) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.14554909217773004) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7954375) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07429913902055749) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4257375) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10329608095410872) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.646375) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10428943394865786) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8249500000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04785468179260445) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.38275000000000003) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06501528454070343) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6504875) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07057889100574483) - val-core/sciknoweval/acc/mean@16:np.float64(0.62421875) - val-aux/sciknoweval/acc/std@16:np.float64(0.18777951588879277) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.699725) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.14925101134863997) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5497249999999999) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1686963837105089) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6241125000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1881880107415588) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7547625) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.10614923952325535) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.48316250000000005) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1385217733706531) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6387875) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.14554909217773004) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7954375) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07429913902055749) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4257375) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10329608095410872) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.646375) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10428943394865786) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8249500000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04785468179260445) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.38275000000000003) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06501528454070343) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6504875) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07057889100574483) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99296875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.02339681404319957) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9992374999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.007777136517816352) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9865124999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.03086320051797221) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9931000000000001) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.022986682202438165) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.97585) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.038177039681755176) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9976375000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.013409115755367873) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9586375) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.04250866307068464) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9995375) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.005306554090650112) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.93815) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03807095702890277) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999625) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.000683625445693766) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00048432277981191874 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00048432277981191874 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:382.09765625 - response_length/max:1653.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:382.09765625 - response_length_non_aborted/max:1653.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.4375 - prompt_length/max:330.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011514127254486084 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9901416879147291) - timing_s/agent_loop/generate_sequences/max:np.float64(10.789292139932513) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9146284311136696) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.789292139932513) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:1653 - timing_s/gen:16.85866062529385 - timing_s/reward:0.059875741600990295 - timing_s/old_log_prob:2.412396924570203 - timing_s/adv:0.6014310400933027 - timing_s/update_actor:15.988812597468495 - timing_s/step:36.00774179585278 - timing_s/testing:91.78712977468967 - timing_s/save_checkpoint:6.680276766419411 - timing_s/stop_profile:0.00010959990322589874 - timing_per_token_ms/update_actor:0.09735444519352685 - timing_per_token_ms/adv:0.003662059635355274 - timing_per_token_ms/gen:0.17234898458646095 - perf/total_num_tokens:164233 - perf/time_per_step:36.00774179585278 - perf/throughput:570.1308656452446 (TaskRunner pid=2074004) Training Progress: 30%|███ | 30/100 [26:18<1:29:26, 76.67s/it] (TaskRunner pid=2074004) step:31 - global_seqlen/min:18767 - global_seqlen/max:37361 - global_seqlen/minmax_diff:18594 - global_seqlen/balanced_min:23793 - global_seqlen/balanced_max:23854 - global_seqlen/mean:23813.5 - actor/entropy:0.09906209260225296 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35506775975227356 - training/rollout_probs_diff_mean:0.0027171920519322157 - training/rollout_probs_diff_std:0.01121894083917141 - training/rollout_actor_probs_pearson_corr:0.997601330280304 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.0000742673873901 - rollout_corr/rollout_is_ratio_fraction_high:2.5133205781457946e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.188867751508951e-05 - rollout_corr/rollout_is_max:2.3356175422668457 - rollout_corr/rollout_is_min:0.32783398032188416 - rollout_corr/rollout_is_std:0.030455991625785828 - rollout_corr/rollout_is_eff_sample_size:0.9990735300790685 - rollout_corr/rollout_is_seq_mean:1.0001554489135742 - rollout_corr/rollout_is_seq_std:0.0015059742145240307 - rollout_corr/rollout_is_seq_max:1.0055322647094727 - rollout_corr/rollout_is_seq_min:0.9959016442298889 - rollout_corr/rollout_is_seq_max_deviation:0.005532264709472656 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.09525554953143) - rollout_corr/training_log_ppl:np.float64(0.08918771221215138) - rollout_corr/kl:np.float64(0.0004833923444760302) - rollout_corr/k3_kl:np.float64(0.0005884723202669306) - rollout_corr/rollout_ppl:np.float64(1.094714654609561) - rollout_corr/rollout_log_ppl:np.float64(0.08870431894683861) - rollout_corr/log_ppl_diff:np.float64(0.0004833932653127704) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014353545957419556) - rollout_corr/log_ppl_diff_max:np.float64(0.007750939577817917) - rollout_corr/log_ppl_diff_min:np.float64(-0.0094594806432724) - rollout_corr/ppl_ratio:np.float64(1.0004852761048824) - rollout_corr/chi2_token:np.float64(0.0013451201375573874) - rollout_corr/chi2_seq:np.float64(0.6003347004298121) - actor/pg_loss:np.float64(0.0021815065992996097) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000496987612450539) - actor/ppo_kl:np.float64(0.0001865752313179314) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(91561.24236924388) - self_distillation/token_reweight_w_std:np.float64(1730255.973181627) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005300368648022) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04233101321733557) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.2640515901148319) - perf/mfu/actor:np.float64(0.04318160738783638) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.296875) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013344685547053814 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013344685547053814 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:466.265625 - response_length/max:1752.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:466.265625 - response_length_non_aborted/max:1752.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.90625 - prompt_length/max:364.0 - prompt_length/min:188.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.933486580848694e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.500905817374587) - timing_s/agent_loop/generate_sequences/max:np.float64(11.73058369755745) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.724863474373706) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.73058369755745) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:296 - timing_s/agent_loop/slowest/response_length:1752 - timing_s/gen:17.499854369089007 - timing_s/reward:0.06082160025835037 - timing_s/old_log_prob:2.452167185023427 - timing_s/adv:0.642735943198204 - timing_s/update_actor:15.802205985412002 - timing_s/step:36.550422716885805 - timing_s/stop_profile:3.2784417271614075e-05 - timing_per_token_ms/update_actor:0.08294772915264452 - timing_per_token_ms/adv:0.003373800277144288 - timing_per_token_ms/gen:0.14660914822801688 - perf/total_num_tokens:190508 - perf/time_per_step:36.550422716885805 - perf/throughput:651.5246125730437 (TaskRunner pid=2074004) Training Progress: 31%|███ | 31/100 [26:54<1:14:20, 64.64s/it] (TaskRunner pid=2074004) step:32 - global_seqlen/min:14634 - global_seqlen/max:23574 - global_seqlen/minmax_diff:8940 - global_seqlen/balanced_min:18859 - global_seqlen/balanced_max:18877 - global_seqlen/mean:18872.25 - actor/entropy:0.10486175119876862 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43836408853530884 - training/rollout_probs_diff_mean:0.003058877307921648 - training/rollout_probs_diff_std:0.012399275787174702 - training/rollout_actor_probs_pearson_corr:0.9971876740455627 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9999085068702698 - rollout_corr/rollout_is_ratio_fraction_high:2.3329055693466216e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001399743341607973 - rollout_corr/rollout_is_max:2.0753023624420166 - rollout_corr/rollout_is_min:0.2297990471124649 - rollout_corr/rollout_is_std:0.032623618841171265 - rollout_corr/rollout_is_eff_sample_size:0.998936593082401 - rollout_corr/rollout_is_seq_mean:0.9999740123748779 - rollout_corr/rollout_is_seq_std:0.0029045569244772196 - rollout_corr/rollout_is_seq_max:1.014037847518921 - rollout_corr/rollout_is_seq_min:0.9840235710144043 - rollout_corr/rollout_is_seq_max_deviation:0.015976428985595703 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0999524341896176) - rollout_corr/training_log_ppl:np.float64(0.09279653291014256) - rollout_corr/kl:np.float64(0.0007732100730457603) - rollout_corr/k3_kl:np.float64(0.000771619481270136) - rollout_corr/rollout_ppl:np.float64(1.0990605875849724) - rollout_corr/rollout_log_ppl:np.float64(0.09202332204586128) - rollout_corr/log_ppl_diff:np.float64(0.0007732108642812818) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020941188995493576) - rollout_corr/log_ppl_diff_max:np.float64(0.02633790671825409) - rollout_corr/log_ppl_diff_min:np.float64(-0.011187225580215454) - rollout_corr/ppl_ratio:np.float64(1.0007794103585184) - rollout_corr/chi2_token:np.float64(0.0014043895062059164) - rollout_corr/chi2_seq:np.float64(0.37330453749746084) - actor/pg_loss:np.float64(0.0019935755990445614) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005535308105208969) - actor/ppo_kl:np.float64(0.00010920122786206576) - actor/pg_clipfrac_lower:np.float64(7.207103408291005e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(222890.26285817288) - self_distillation/token_reweight_w_std:np.float64(2919852.600875127) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000309090828523) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03591290376789402) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.3092605173587799) - perf/mfu/actor:np.float64(0.03432852611920223) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.16852188110352) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:1 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0020937828812748194 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0020937828812748194 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:334.8828125 - response_length/max:1186.0 - response_length/min:51.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:334.8828125 - response_length_non_aborted/max:1186.0 - response_length_non_aborted/min:51.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.875 - prompt_length/max:375.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.379349648952484e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.595955204218626) - timing_s/agent_loop/generate_sequences/max:np.float64(8.028898818418384) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3284817362364265) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.028898818418384) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:1142 - timing_s/gen:13.935343297198415 - timing_s/reward:0.058442361652851105 - timing_s/old_log_prob:2.4146821219474077 - timing_s/adv:0.5538248158991337 - timing_s/update_actor:15.78230369463563 - timing_s/step:32.8320482596755 - timing_s/stop_profile:5.641952157020569e-05 - timing_per_token_ms/update_actor:0.10453379760386036 - timing_per_token_ms/adv:0.0036682484593724496 - timing_per_token_ms/gen:0.16254920444649965 - perf/total_num_tokens:150978 - perf/time_per_step:32.8320482596755 - perf/throughput:574.811837834041 (TaskRunner pid=2074004) Training Progress: 32%|███▏ | 32/100 [27:27<1:02:27, 55.11s/it] (TaskRunner pid=2074004) step:33 - global_seqlen/min:19167 - global_seqlen/max:32062 - global_seqlen/minmax_diff:12895 - global_seqlen/balanced_min:24910 - global_seqlen/balanced_max:31094 - global_seqlen/mean:25699.125 - actor/entropy:0.0963616594672203 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5835636854171753 - training/rollout_probs_diff_mean:0.002623476553708315 - training/rollout_probs_diff_std:0.011050577275454998 - training/rollout_actor_probs_pearson_corr:0.9976565837860107 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0000323057174683 - rollout_corr/rollout_is_ratio_fraction_high:2.285731716256123e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.571463432512246e-05 - rollout_corr/rollout_is_max:3.547874927520752 - rollout_corr/rollout_is_min:0.27111193537712097 - rollout_corr/rollout_is_std:0.030399184674024582 - rollout_corr/rollout_is_eff_sample_size:0.9990767427790138 - rollout_corr/rollout_is_seq_mean:1.0000334978103638 - rollout_corr/rollout_is_seq_std:0.0019103900995105505 - rollout_corr/rollout_is_seq_max:1.0073792934417725 - rollout_corr/rollout_is_seq_min:0.9880338907241821 - rollout_corr/rollout_is_seq_max_deviation:0.011966109275817871 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1100693135522306) - rollout_corr/training_log_ppl:np.float64(0.10072795890664565) - rollout_corr/kl:np.float64(0.0004943630928924136) - rollout_corr/k3_kl:np.float64(0.0005992158173953044) - rollout_corr/rollout_ppl:np.float64(1.1094494266435504) - rollout_corr/rollout_log_ppl:np.float64(0.10023359479600913) - rollout_corr/log_ppl_diff:np.float64(0.0004943641106365249) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015578279126202688) - rollout_corr/log_ppl_diff_max:np.float64(0.022818326950073242) - rollout_corr/log_ppl_diff_min:np.float64(-0.005478024482727051) - rollout_corr/ppl_ratio:np.float64(1.0004977090284228) - rollout_corr/chi2_token:np.float64(0.001456292113289237) - rollout_corr/chi2_seq:np.float64(0.8751170034520328) - actor/pg_loss:np.float64(0.0023588838521391153) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005614726733256248) - actor/ppo_kl:np.float64(6.841486872133373e-06) - actor/pg_clipfrac_lower:np.float64(4.124868155486183e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_w_mean:np.float64(73720.56168391788) - self_distillation/token_reweight_w_std:np.float64(1263082.4041759328) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995736547280103) - self_distillation/token_reweight_w_clip_frac:np.float64(0.038523146067745984) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.31921055912971497) - perf/mfu/actor:np.float64(0.04383612915635905) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.20793533325195) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:1 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010148649103939533 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010148649103939533 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:512.69140625 - response_length/max:8192.0 - response_length/min:90.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:512.69140625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:290.40625 - prompt_length/max:437.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001917630434036255 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0906069297343493) - timing_s/agent_loop/generate_sequences/max:np.float64(50.3866535294801) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.971057376002136) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.3866535294801) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:346 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.46107683330774 - timing_s/reward:0.06383568607270718 - timing_s/old_log_prob:2.6003563329577446 - timing_s/adv:0.6247836276888847 - timing_s/update_actor:17.063110006973147 - timing_s/step:76.9013549759984 - timing_s/stop_profile:0.00010899640619754791 - timing_per_token_ms/update_actor:0.0829946058813926 - timing_per_token_ms/adv:0.003038934339636489 - timing_per_token_ms/gen:0.4301829105997588 - perf/total_num_tokens:205593 - perf/time_per_step:76.9013549759984 - perf/throughput:334.1829933688545 (TaskRunner pid=2074004) Training Progress: 33%|███▎ | 33/100 [28:44<1:08:51, 61.66s/it] (TaskRunner pid=2074004) step:34 - global_seqlen/min:17957 - global_seqlen/max:26538 - global_seqlen/minmax_diff:8581 - global_seqlen/balanced_min:21718 - global_seqlen/balanced_max:21733 - global_seqlen/mean:21729.125 - actor/entropy:0.09093478322029114 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35351693630218506 - training/rollout_probs_diff_mean:0.002648385940119624 - training/rollout_probs_diff_std:0.01128794439136982 - training/rollout_actor_probs_pearson_corr:0.9973158240318298 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0001046657562256 - rollout_corr/rollout_is_ratio_fraction_high:3.717990330187604e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.29497618926689e-05 - rollout_corr/rollout_is_max:3.9687960147857666 - rollout_corr/rollout_is_min:0.3704526722431183 - rollout_corr/rollout_is_std:0.03138892725110054 - rollout_corr/rollout_is_eff_sample_size:0.9990158239877898 - rollout_corr/rollout_is_seq_mean:0.9999611377716064 - rollout_corr/rollout_is_seq_std:0.002279487904161215 - rollout_corr/rollout_is_seq_max:1.009277582168579 - rollout_corr/rollout_is_seq_min:0.9918825626373291 - rollout_corr/rollout_is_seq_max_deviation:0.009277582168579102 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0867361915297806) - rollout_corr/training_log_ppl:np.float64(0.08140930149238557) - rollout_corr/kl:np.float64(0.0006834588668453989) - rollout_corr/k3_kl:np.float64(0.0006041397766978207) - rollout_corr/rollout_ppl:np.float64(1.0859681307338178) - rollout_corr/rollout_log_ppl:np.float64(0.08072584179171827) - rollout_corr/log_ppl_diff:np.float64(0.0006834597006672993) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016792384558357298) - rollout_corr/log_ppl_diff_max:np.float64(0.011221528053283691) - rollout_corr/log_ppl_diff_min:np.float64(-0.004778027534484863) - rollout_corr/ppl_ratio:np.float64(1.000686518382281) - rollout_corr/chi2_token:np.float64(0.0010883505456149578) - rollout_corr/chi2_seq:np.float64(0.5784096252173185) - actor/pg_loss:np.float64(0.002307310584001243) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005154957439117425) - actor/ppo_kl:np.float64(9.805531538464862e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(82543.4987949559) - self_distillation/token_reweight_w_std:np.float64(1531421.5975041706) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002939573023468) - self_distillation/token_reweight_w_clip_frac:np.float64(0.032446051089209504) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.2603403516113758) - perf/mfu/actor:np.float64(0.0392491945722704) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.23517227172852) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:1 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008258585818111897 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008258585818111897 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:420.25390625 - response_length/max:1275.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:420.25390625 - response_length_non_aborted/max:1275.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.78125 - prompt_length/max:382.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014248304069042206 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9343549367040396) - timing_s/agent_loop/generate_sequences/max:np.float64(9.114851927384734) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.215552492758434) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.114851927384734) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1275 - timing_s/gen:14.953994173556566 - timing_s/reward:0.05961340852081776 - timing_s/old_log_prob:2.531619004905224 - timing_s/adv:0.5816438850015402 - timing_s/update_actor:15.924356224015355 - timing_s/step:34.133271373808384 - timing_s/stop_profile:4.3351203203201294e-05 - timing_per_token_ms/update_actor:0.09160721050672402 - timing_per_token_ms/adv:0.003345992331729535 - timing_per_token_ms/gen:0.13899701792588712 - perf/total_num_tokens:173833 - perf/time_per_step:34.133271373808384 - perf/throughput:636.5966145475729 (TaskRunner pid=2074004) Training Progress: 34%|███▍ | 34/100 [29:18<58:45, 53.41s/it] (TaskRunner pid=2074004) step:35 - global_seqlen/min:17441 - global_seqlen/max:33644 - global_seqlen/minmax_diff:16203 - global_seqlen/balanced_min:21811 - global_seqlen/balanced_max:28467 - global_seqlen/mean:22649.75 - actor/entropy:0.10204526782035828 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.293226420879364 - training/rollout_probs_diff_mean:0.002843219321221113 - training/rollout_probs_diff_std:0.011386570520699024 - training/rollout_actor_probs_pearson_corr:0.9975972771644592 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.000219702720642 - rollout_corr/rollout_is_ratio_fraction_high:1.7400079741491936e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011310051922919229 - rollout_corr/rollout_is_max:2.0903584957122803 - rollout_corr/rollout_is_min:0.10869106650352478 - rollout_corr/rollout_is_std:0.03217843919992447 - rollout_corr/rollout_is_eff_sample_size:0.9989663329398729 - rollout_corr/rollout_is_seq_mean:1.0003238916397095 - rollout_corr/rollout_is_seq_std:0.002396445255726576 - rollout_corr/rollout_is_seq_max:1.0162620544433594 - rollout_corr/rollout_is_seq_min:0.9926619529724121 - rollout_corr/rollout_is_seq_max_deviation:0.016262054443359375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1133964317850769) - rollout_corr/training_log_ppl:np.float64(0.10447760274473694) - rollout_corr/kl:np.float64(0.000506502414840071) - rollout_corr/k3_kl:np.float64(0.0008595675468114905) - rollout_corr/rollout_ppl:np.float64(1.1128244805149734) - rollout_corr/rollout_log_ppl:np.float64(0.10397110023404821) - rollout_corr/log_ppl_diff:np.float64(0.0005065025106887333) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018531291279941797) - rollout_corr/log_ppl_diff_max:np.float64(0.026727454736828804) - rollout_corr/log_ppl_diff_min:np.float64(-0.01405394822359085) - rollout_corr/ppl_ratio:np.float64(1.000511220889166) - rollout_corr/chi2_token:np.float64(0.0022407646756619215) - rollout_corr/chi2_seq:np.float64(1.0442664730362594) - actor/pg_loss:np.float64(0.002358463010750711) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00048152222007047385) - actor/ppo_kl:np.float64(0.0001821644597370664) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(110187.43466792861) - self_distillation/token_reweight_w_std:np.float64(1515968.7875927906) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997799128759652) - self_distillation/token_reweight_w_clip_frac:np.float64(0.034502132126362994) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.34709084033966064) - perf/mfu/actor:np.float64(0.0391226831248303) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.17388153076172) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:1 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.053757548332214355 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.053757548332214355 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:448.9921875 - response_length/max:8192.0 - response_length/min:68.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:448.9921875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:68.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:258.8125 - prompt_length/max:365.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.4147873520851135e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8165026474744081) - timing_s/agent_loop/generate_sequences/max:np.float64(50.60485251620412) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.408925204807019) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.60485251620412) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.32023567520082 - timing_s/reward:0.0652729794383049 - timing_s/old_log_prob:2.546900112181902 - timing_s/adv:0.5270522274076939 - timing_s/update_actor:16.483777582645416 - timing_s/step:76.03851772285998 - timing_s/stop_profile:0.00011594034731388092 - timing_per_token_ms/update_actor:0.0909710790552071 - timing_per_token_ms/adv:0.0029087088566523574 - timing_per_token_ms/gen:0.489988304320447 - perf/total_num_tokens:181198 - perf/time_per_step:76.03851772285998 - perf/throughput:297.87206113817564 (TaskRunner pid=2074004) Training Progress: 35%|███▌ | 35/100 [30:34<1:05:13, 60.21s/it] (TaskRunner pid=2074004) step:36 - global_seqlen/min:18954 - global_seqlen/max:33911 - global_seqlen/minmax_diff:14957 - global_seqlen/balanced_min:26378 - global_seqlen/balanced_max:32662 - global_seqlen/mean:27187.75 - actor/entropy:0.1043354868888855 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4853993356227875 - training/rollout_probs_diff_mean:0.0027279683854430914 - training/rollout_probs_diff_std:0.011042559519410133 - training/rollout_actor_probs_pearson_corr:0.9978550672531128 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:1.000059723854065 - rollout_corr/rollout_is_ratio_fraction_high:2.6941470423480496e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.082440763246268e-05 - rollout_corr/rollout_is_max:7.419532775878906 - rollout_corr/rollout_is_min:0.13505788147449493 - rollout_corr/rollout_is_std:0.03132620081305504 - rollout_corr/rollout_is_eff_sample_size:0.9990196311945923 - rollout_corr/rollout_is_seq_mean:1.000086784362793 - rollout_corr/rollout_is_seq_std:0.0020494619384407997 - rollout_corr/rollout_is_seq_max:1.015174388885498 - rollout_corr/rollout_is_seq_min:0.99062579870224 - rollout_corr/rollout_is_seq_max_deviation:0.015174388885498047 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1137358797714114) - rollout_corr/training_log_ppl:np.float64(0.10603692273616616) - rollout_corr/kl:np.float64(0.0006563753371509051) - rollout_corr/k3_kl:np.float64(0.0007457188598607445) - rollout_corr/rollout_ppl:np.float64(1.1129927351139486) - rollout_corr/rollout_log_ppl:np.float64(0.10538054626522353) - rollout_corr/log_ppl_diff:np.float64(0.0006563764709426323) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015180988721112953) - rollout_corr/log_ppl_diff_max:np.float64(0.011238820850849152) - rollout_corr/log_ppl_diff_min:np.float64(-0.007395319640636444) - rollout_corr/ppl_ratio:np.float64(1.0006589936092496) - rollout_corr/chi2_token:np.float64(0.002004736103117466) - rollout_corr/chi2_seq:np.float64(1.2244051673915237) - actor/pg_loss:np.float64(0.0032588703325018287) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008896479857867234) - actor/ppo_kl:np.float64(8.400913770501006e-05) - actor/pg_clipfrac_lower:np.float64(5.440459517558338e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(142463.33054139256) - self_distillation/token_reweight_w_std:np.float64(2692305.3041795073) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015312230680138) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0585212862060871) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.31766650825738907) - perf/mfu/actor:np.float64(0.04633240200426601) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.1927490234375) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:1 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.04938371479511261 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.04938371479511261 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:579.9609375 - response_length/max:8192.0 - response_length/min:82.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:579.9609375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:269.65625 - prompt_length/max:375.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.186511695384979e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9956839792430401) - timing_s/agent_loop/generate_sequences/max:np.float64(51.712549494579434) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.89511316808057) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(51.712549494579434) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:233 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:57.78715647570789 - timing_s/reward:0.06627766788005829 - timing_s/old_log_prob:2.5614236313849688 - timing_s/adv:0.6225856561213732 - timing_s/update_actor:17.082536520436406 - timing_s/step:78.20577510073781 - timing_s/stop_profile:0.00011078454554080963 - timing_per_token_ms/update_actor:0.07853967559119643 - timing_per_token_ms/adv:0.0028624364655100787 - timing_per_token_ms/gen:0.3892177306911018 - perf/total_num_tokens:217502 - perf/time_per_step:78.20577510073781 - perf/throughput:347.6437637115562 (TaskRunner pid=2074004) Training Progress: 36%|███▌ | 36/100 [31:52<1:09:59, 65.62s/it] (TaskRunner pid=2074004) step:37 - global_seqlen/min:18791 - global_seqlen/max:27840 - global_seqlen/minmax_diff:9049 - global_seqlen/balanced_min:23579 - global_seqlen/balanced_max:23755 - global_seqlen/mean:23627.625 - actor/entropy:0.10102436691522598 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39139458537101746 - training/rollout_probs_diff_mean:0.002805094001814723 - training/rollout_probs_diff_std:0.011440390720963478 - training/rollout_actor_probs_pearson_corr:0.9976106882095337 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.6640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:0.9998602271080017 - rollout_corr/rollout_is_ratio_fraction_high:1.6350957594113424e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.9052869144361466e-05 - rollout_corr/rollout_is_max:2.243018388748169 - rollout_corr/rollout_is_min:0.0117238974198699 - rollout_corr/rollout_is_std:0.03203921392560005 - rollout_corr/rollout_is_eff_sample_size:0.9989743035225582 - rollout_corr/rollout_is_seq_mean:0.9999796152114868 - rollout_corr/rollout_is_seq_std:0.002273126505315304 - rollout_corr/rollout_is_seq_max:1.0130821466445923 - rollout_corr/rollout_is_seq_min:0.9926750659942627 - rollout_corr/rollout_is_seq_max_deviation:0.013082146644592285 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0996112595312297) - rollout_corr/training_log_ppl:np.float64(0.09333795910788467) - rollout_corr/kl:np.float64(0.001013359834161065) - rollout_corr/k3_kl:np.float64(0.0008460905967808685) - rollout_corr/rollout_ppl:np.float64(1.0984738464467227) - rollout_corr/rollout_log_ppl:np.float64(0.09232459875784116) - rollout_corr/log_ppl_diff:np.float64(0.0010133603500435129) - rollout_corr/log_ppl_abs_diff:np.float64(0.00191976049245568) - rollout_corr/log_ppl_diff_max:np.float64(0.014345422387123108) - rollout_corr/log_ppl_diff_min:np.float64(-0.009604066610336304) - rollout_corr/ppl_ratio:np.float64(1.001017474103719) - rollout_corr/chi2_token:np.float64(0.001261905301362276) - rollout_corr/chi2_seq:np.float64(0.5114637641236186) - actor/pg_loss:np.float64(0.003147437470033765) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007599365783335088) - actor/ppo_kl:np.float64(0.0003716098390409428) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6640625) - self_distillation/token_reweight_w_mean:np.float64(192101.5823432547) - self_distillation/token_reweight_w_std:np.float64(2797782.555044028) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008915911894292) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04522743531560991) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.35525741800665855) - perf/mfu/actor:np.float64(0.04261905670880045) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.2181911468506) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008520892821252346 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008520892821252346 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:477.80078125 - response_length/max:2318.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:477.80078125 - response_length_non_aborted/max:2318.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:260.5625 - prompt_length/max:350.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.429400622844696e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.102793700993061) - timing_s/agent_loop/generate_sequences/max:np.float64(14.33990266919136) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.5264077913961955) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.33990266919136) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:344 - timing_s/agent_loop/slowest/response_length:2318 - timing_s/gen:20.240060921758413 - timing_s/reward:0.06487942114472389 - timing_s/old_log_prob:2.515492904931307 - timing_s/adv:0.5939651485532522 - timing_s/update_actor:16.080622201785445 - timing_s/step:39.57463042251766 - timing_s/stop_profile:0.00011385232210159302 - timing_per_token_ms/update_actor:0.08507320457401794 - timing_per_token_ms/adv:0.0031423235966017123 - timing_per_token_ms/gen:0.16547218229484384 - perf/total_num_tokens:189021 - perf/time_per_step:39.57463042251766 - perf/throughput:597.0396879955716 (TaskRunner pid=2074004) Training Progress: 37%|███▋ | 37/100 [32:32<1:00:42, 57.82s/it] (TaskRunner pid=2074004) step:38 - global_seqlen/min:17600 - global_seqlen/max:27502 - global_seqlen/minmax_diff:9902 - global_seqlen/balanced_min:21161 - global_seqlen/balanced_max:21168 - global_seqlen/mean:21164.5 - actor/entropy:0.09618508070707321 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6138811707496643 - training/rollout_probs_diff_mean:0.0026226078625768423 - training/rollout_probs_diff_std:0.011445230804383755 - training/rollout_actor_probs_pearson_corr:0.9974937438964844 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.6015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.0000364780426025 - rollout_corr/rollout_is_ratio_fraction_high:1.979805892915465e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.899029828375205e-05 - rollout_corr/rollout_is_max:2.0751965045928955 - rollout_corr/rollout_is_min:0.013780022040009499 - rollout_corr/rollout_is_std:0.03070547617971897 - rollout_corr/rollout_is_eff_sample_size:0.9990580618130113 - rollout_corr/rollout_is_seq_mean:1.0000827312469482 - rollout_corr/rollout_is_seq_std:0.0019129818538203835 - rollout_corr/rollout_is_seq_max:1.0122212171554565 - rollout_corr/rollout_is_seq_min:0.9918540716171265 - rollout_corr/rollout_is_seq_max_deviation:0.012221217155456543 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0946312695741653) - rollout_corr/training_log_ppl:np.float64(0.08725073869572952) - rollout_corr/kl:np.float64(0.0006405033833267737) - rollout_corr/k3_kl:np.float64(0.0005329421040762838) - rollout_corr/rollout_ppl:np.float64(1.0939017231576145) - rollout_corr/rollout_log_ppl:np.float64(0.08661023495005793) - rollout_corr/log_ppl_diff:np.float64(0.0006405037456715945) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014858933245704975) - rollout_corr/log_ppl_diff_max:np.float64(0.014180347323417664) - rollout_corr/log_ppl_diff_min:np.float64(-0.004229940474033356) - rollout_corr/ppl_ratio:np.float64(1.0006429618224502) - rollout_corr/chi2_token:np.float64(0.0008146455511450768) - rollout_corr/chi2_seq:np.float64(0.17395253549329937) - actor/pg_loss:np.float64(0.002348349546082318) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007172689474828076) - actor/ppo_kl:np.float64(0.00023065414274014984) - actor/pg_clipfrac_lower:np.float64(1.249999968422344e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6015625) - self_distillation/token_reweight_w_mean:np.float64(223555.59363828856) - self_distillation/token_reweight_w_std:np.float64(3377666.5284842458) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001167273381725) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04519080827594735) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.2977074235677719) - perf/mfu/actor:np.float64(0.03844596025288242) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.3445816040039) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:1 - critic/score/mean:0.6015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0034980697091668844 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0034980697091668844 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:394.609375 - response_length/max:1200.0 - response_length/min:84.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:394.609375 - response_length_non_aborted/max:1200.0 - response_length_non_aborted/min:84.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.78125 - prompt_length/max:328.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.56140798330307e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0009879916906357) - timing_s/agent_loop/generate_sequences/max:np.float64(8.732898579910398) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.088401710039761) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.732898579910398) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:1200 - timing_s/gen:14.707298658788204 - timing_s/reward:0.05728561617434025 - timing_s/old_log_prob:2.3959624897688627 - timing_s/adv:0.5882252883166075 - timing_s/update_actor:15.931603338569403 - timing_s/step:33.75794658437371 - timing_s/stop_profile:0.0001205597072839737 - timing_per_token_ms/update_actor:0.09409390334386238 - timing_per_token_ms/adv:0.0034741270069964295 - timing_per_token_ms/gen:0.1455879890990715 - perf/total_num_tokens:169316 - perf/time_per_step:33.75794658437371 - perf/throughput:626.9486785015793 (TaskRunner pid=2074004) Training Progress: 38%|███▊ | 38/100 [33:06<52:18, 50.62s/it] (TaskRunner pid=2074004) step:39 - global_seqlen/min:14192 - global_seqlen/max:21395 - global_seqlen/minmax_diff:7203 - global_seqlen/balanced_min:18859 - global_seqlen/balanced_max:18868 - global_seqlen/mean:18864.75 - actor/entropy:0.08581894636154175 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8518780469894409 - training/rollout_probs_diff_mean:0.0027508889324963093 - training/rollout_probs_diff_std:0.012479580007493496 - training/rollout_actor_probs_pearson_corr:0.9966431260108948 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.82421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:0.9999743700027466 - rollout_corr/rollout_is_ratio_fraction_high:2.358101301069837e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010611455945763737 - rollout_corr/rollout_is_max:2.9851624965667725 - rollout_corr/rollout_is_min:0.14805817604064941 - rollout_corr/rollout_is_std:0.03196844086050987 - rollout_corr/rollout_is_eff_sample_size:0.9989789431740315 - rollout_corr/rollout_is_seq_mean:1.0000545978546143 - rollout_corr/rollout_is_seq_std:0.0020085289143025875 - rollout_corr/rollout_is_seq_max:1.0090090036392212 - rollout_corr/rollout_is_seq_min:0.9939286708831787 - rollout_corr/rollout_is_seq_max_deviation:0.009009003639221191 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.090072206221521) - rollout_corr/training_log_ppl:np.float64(0.08314310060450225) - rollout_corr/kl:np.float64(0.0007816835961591551) - rollout_corr/k3_kl:np.float64(0.0008022212134584095) - rollout_corr/rollout_ppl:np.float64(1.0891828034073114) - rollout_corr/rollout_log_ppl:np.float64(0.08236141606539604) - rollout_corr/log_ppl_diff:np.float64(0.0007816845391062088) - rollout_corr/log_ppl_abs_diff:np.float64(0.00177329323196318) - rollout_corr/log_ppl_diff_max:np.float64(0.045620523393154144) - rollout_corr/log_ppl_diff_min:np.float64(-0.008172988891601562) - rollout_corr/ppl_ratio:np.float64(1.0007883713115007) - rollout_corr/chi2_token:np.float64(0.0012875364627689123) - rollout_corr/chi2_seq:np.float64(0.3463961035013199) - actor/pg_loss:np.float64(0.0016197055811062455) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00035407407904131105) - actor/ppo_kl:np.float64(0.00027624938138259836) - actor/pg_clipfrac_lower:np.float64(1.398360836901702e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.82421875) - self_distillation/token_reweight_w_mean:np.float64(47769.31919008936) - self_distillation/token_reweight_w_std:np.float64(767346.1878132067) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993690950796008) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02541799924802035) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.3479584529995918) - perf/mfu/actor:np.float64(0.0347302901051022) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.15287017822266) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:1 - critic/score/mean:0.82421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.82421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007721307687461376 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007721307687461376 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:331.3046875 - response_length/max:879.0 - response_length/min:65.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:331.3046875 - response_length_non_aborted/max:879.0 - response_length_non_aborted/min:65.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.21875 - prompt_length/max:326.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011703930795192719 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9549711402505636) - timing_s/agent_loop/generate_sequences/max:np.float64(6.732800455763936) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4961698439146858) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.732800455763936) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:317 - timing_s/agent_loop/slowest/response_length:879 - timing_s/gen:12.607269439846277 - timing_s/reward:0.05702884867787361 - timing_s/old_log_prob:2.344845812767744 - timing_s/adv:0.5173551607877016 - timing_s/update_actor:15.629758136346936 - timing_s/step:31.24427343159914 - timing_s/stop_profile:0.000109916552901268 - timing_per_token_ms/update_actor:0.10356457239260351 - timing_per_token_ms/adv:0.003428054710423552 - timing_per_token_ms/gen:0.14864608955887326 - perf/total_num_tokens:150918 - perf/time_per_step:31.24427343159914 - perf/throughput:603.7826432833924 (TaskRunner pid=2074004) Training Progress: 39%|███▉ | 39/100 [33:37<45:34, 44.83s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. We need to find the charge $ q $ using the formula: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Solving for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substitute the values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest option is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30/actor (TaskRunner pid=2074004) step:40 - global_seqlen/min:17338 - global_seqlen/max:29776 - global_seqlen/minmax_diff:12438 - global_seqlen/balanced_min:22374 - global_seqlen/balanced_max:22391 - global_seqlen/mean:22384.625 - actor/entropy:0.089930459856987 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3171451985836029 - training/rollout_probs_diff_mean:0.0025929557159543037 - training/rollout_probs_diff_std:0.011268679052591324 - training/rollout_actor_probs_pearson_corr:0.9974833726882935 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.95703125 - self_distillation/correct_sample_fraction:0.80078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.95703125 - rollout_corr/rollout_is_mean:1.000073790550232 - rollout_corr/rollout_is_ratio_fraction_high:1.7818165360949934e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.4545413402374834e-05 - rollout_corr/rollout_is_max:2.4635725021362305 - rollout_corr/rollout_is_min:0.2820615768432617 - rollout_corr/rollout_is_std:0.03070935793220997 - rollout_corr/rollout_is_eff_sample_size:0.9990580618130113 - rollout_corr/rollout_is_seq_mean:0.9999992847442627 - rollout_corr/rollout_is_seq_std:0.001931522972881794 - rollout_corr/rollout_is_seq_max:1.0066325664520264 - rollout_corr/rollout_is_seq_min:0.9905868768692017 - rollout_corr/rollout_is_seq_max_deviation:0.00941312313079834 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0896218931302428) - rollout_corr/training_log_ppl:np.float64(0.08464303463551914) - rollout_corr/kl:np.float64(0.0005157805179152319) - rollout_corr/k3_kl:np.float64(0.0005807906663477524) - rollout_corr/rollout_ppl:np.float64(1.0890426081605256) - rollout_corr/rollout_log_ppl:np.float64(0.08412725336529547) - rollout_corr/log_ppl_diff:np.float64(0.0005157812702236697) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015912332892185077) - rollout_corr/log_ppl_diff_max:np.float64(0.009794257581233978) - rollout_corr/log_ppl_diff_min:np.float64(-0.0059789493680000305) - rollout_corr/ppl_ratio:np.float64(1.0005182197783142) - rollout_corr/chi2_token:np.float64(0.0013064609374850988) - rollout_corr/chi2_seq:np.float64(1.130385466851294) - actor/pg_loss:np.float64(0.002167836995795369) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005737871229030134) - actor/ppo_kl:np.float64(1.1175101938931675e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.95703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.95703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_w_mean:np.float64(140985.0619100898) - self_distillation/token_reweight_w_std:np.float64(2265311.5997434272) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001305928453803) - self_distillation/token_reweight_w_clip_frac:np.float64(0.036739757866598666) - self_distillation/empty_target_batch:np.float64(0.04296875) - actor/grad_norm:np.float64(0.2651498392224312) - perf/mfu/actor:np.float64(0.04031842311642975) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.09762573242188) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.625) - val-aux/sciknoweval/reward/std@16:np.float64(0.17935140746688946) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6972125) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15230234946062082) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5531125) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15011488729407602) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6247875) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.17933428616120461) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.757625) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11678637676576642) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.49601249999999997) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11323686397372593) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6350625) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1464125828088421) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8041375000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08263344427572242) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4525625) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0819621762464158) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6428) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1088052134913493) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8369375000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.051110544163678794) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4195875) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05406867605156187) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.648425) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.08121353362515363) - val-aux/sciknoweval/score/mean@16:np.float64(0.625) - val-aux/sciknoweval/score/std@16:np.float64(0.17935140746688946) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6972125) - val-aux/sciknoweval/score/best@2/std:np.float64(0.15230234946062082) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5531125) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15011488729407602) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6247875) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.17933428616120461) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.757625) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11678637676576642) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.49601249999999997) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11323686397372593) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6350625) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1464125828088421) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8041375000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08263344427572242) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4525625) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0819621762464158) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6428) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.1088052134913493) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8369375000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.051110544163678794) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4195875) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05406867605156187) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.648425) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.08121353362515363) - val-core/sciknoweval/acc/mean@16:np.float64(0.625) - val-aux/sciknoweval/acc/std@16:np.float64(0.17935140746688946) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6972125) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.15230234946062082) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5531125) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15011488729407602) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6247875) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.17933428616120461) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.757625) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11678637676576642) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.49601249999999997) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11323686397372593) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6350625) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1464125828088421) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8041375000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08263344427572242) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4525625) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0819621762464158) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6428) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.1088052134913493) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8369375000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.051110544163678794) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4195875) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05406867605156187) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.648425) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.08121353362515363) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9921875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.024141732305909894) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9989125000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.008527445661552757) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.985275) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.03137280736233215) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9916750000000001) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.024734393444018433) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999874999999999) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.00039508701573197775) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9734375) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.03851008868178078) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9960000000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.01673449238498939) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.956475) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.04152269363307004) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9987) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.00904727412664771) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.936625) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03653224445364453) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.999775) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.002781121632849242) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:1 - critic/score/mean:0.80078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.000915408250875771 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.000915408250875771 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:438.45703125 - response_length/max:1467.0 - response_length/min:64.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:438.45703125 - response_length_non_aborted/max:1467.0 - response_length_non_aborted/min:64.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.0625 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014247745275497437 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8024486973881721) - timing_s/agent_loop/generate_sequences/max:np.float64(10.08868633955717) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.353498277778272) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.08868633955717) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1467 - timing_s/gen:16.001727301627398 - timing_s/reward:0.061644136905670166 - timing_s/old_log_prob:2.4333597384393215 - timing_s/adv:0.5891087763011456 - timing_s/update_actor:15.919175075367093 - timing_s/step:35.09279179573059 - timing_s/testing:93.67140794359148 - timing_s/save_checkpoint:6.656575288623571 - timing_s/stop_profile:0.00014066509902477264 - timing_per_token_ms/update_actor:0.08889569891927547 - timing_per_token_ms/adv:0.0032896953617781487 - timing_per_token_ms/gen:0.14256071363203168 - perf/total_num_tokens:179077 - perf/time_per_step:35.09279179573059 - perf/throughput:637.8695981299306 (TaskRunner pid=2074004) Training Progress: 40%|████ | 40/100 [35:53<1:12:01, 72.03s/it] (TaskRunner pid=2074004) step:41 - global_seqlen/min:20646 - global_seqlen/max:30800 - global_seqlen/minmax_diff:10154 - global_seqlen/balanced_min:23788 - global_seqlen/balanced_max:23817 - global_seqlen/mean:23809.0 - actor/entropy:0.08412408828735352 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5083496570587158 - training/rollout_probs_diff_mean:0.002506777411326766 - training/rollout_probs_diff_std:0.010917266830801964 - training/rollout_actor_probs_pearson_corr:0.9974930882453918 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.70703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.0000280141830444 - rollout_corr/rollout_is_ratio_fraction_high:1.6747051631682552e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.698820652673021e-05 - rollout_corr/rollout_is_max:2.611849784851074 - rollout_corr/rollout_is_min:0.2993384003639221 - rollout_corr/rollout_is_std:0.030081849545240402 - rollout_corr/rollout_is_eff_sample_size:0.999095900418749 - rollout_corr/rollout_is_seq_mean:0.9999955296516418 - rollout_corr/rollout_is_seq_std:0.00155645702034235 - rollout_corr/rollout_is_seq_max:1.0051690340042114 - rollout_corr/rollout_is_seq_min:0.9948093891143799 - rollout_corr/rollout_is_seq_max_deviation:0.005190610885620117 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.084649627096951) - rollout_corr/training_log_ppl:np.float64(0.08000787740093074) - rollout_corr/kl:np.float64(0.00042501650833115523) - rollout_corr/k3_kl:np.float64(0.0005248165592366405) - rollout_corr/rollout_ppl:np.float64(1.084169534035027) - rollout_corr/rollout_log_ppl:np.float64(0.07958286030770978) - rollout_corr/log_ppl_diff:np.float64(0.0004250170932209585) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013505659480870236) - rollout_corr/log_ppl_diff_max:np.float64(0.006395198404788971) - rollout_corr/log_ppl_diff_min:np.float64(-0.0090675950050354) - rollout_corr/ppl_ratio:np.float64(1.0004267231561244) - rollout_corr/chi2_token:np.float64(0.0012936093844473362) - rollout_corr/chi2_seq:np.float64(1.8906177775934339) - actor/pg_loss:np.float64(0.002206359291449189) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006284501350819482) - actor/ppo_kl:np.float64(-4.702512273624393e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_w_mean:np.float64(146167.46774941264) - self_distillation/token_reweight_w_std:np.float64(2535003.0120101087) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000315876677632) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04627777053974569) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.32163915038108826) - perf/mfu/actor:np.float64(0.04315536156757415) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.23479080200195) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:1 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009857733733952045 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009857733733952045 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:466.5 - response_length/max:1590.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:466.5 - response_length_non_aborted/max:1590.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:361.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.379800081253052e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2317185122519732) - timing_s/agent_loop/generate_sequences/max:np.float64(10.718674024567008) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.755297555726429) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.718674024567008) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1590 - timing_s/gen:16.80528495274484 - timing_s/reward:0.0609660129994154 - timing_s/old_log_prob:2.4625763557851315 - timing_s/adv:0.5879520736634731 - timing_s/update_actor:15.929620191454887 - timing_s/step:35.9443339202553 - timing_s/stop_profile:0.00010867416858673096 - timing_per_token_ms/update_actor:0.08363234591674833 - timing_per_token_ms/adv:0.003086816296691761 - timing_per_token_ms/gen:0.14071949484814478 - perf/total_num_tokens:190472 - perf/time_per_step:35.9443339202553 - perf/throughput:662.3853443166236 (TaskRunner pid=2074004) Training Progress: 41%|████ | 41/100 [36:29<1:00:11, 61.22s/it] (TaskRunner pid=2074004) step:42 - global_seqlen/min:20360 - global_seqlen/max:28679 - global_seqlen/minmax_diff:8319 - global_seqlen/balanced_min:24195 - global_seqlen/balanced_max:24221 - global_seqlen/mean:24215.75 - actor/entropy:0.09680385887622833 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6651126146316528 - training/rollout_probs_diff_mean:0.00268166302703321 - training/rollout_probs_diff_std:0.011481569148600101 - training/rollout_actor_probs_pearson_corr:0.9974598288536072 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.56640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9999688863754272 - rollout_corr/rollout_is_ratio_fraction_high:2.390247755101882e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001195123913930729 - rollout_corr/rollout_is_max:3.0732650756835938 - rollout_corr/rollout_is_min:0.28940898180007935 - rollout_corr/rollout_is_std:0.030898984521627426 - rollout_corr/rollout_is_eff_sample_size:0.99904604449077 - rollout_corr/rollout_is_seq_mean:0.999945342540741 - rollout_corr/rollout_is_seq_std:0.001563522731885314 - rollout_corr/rollout_is_seq_max:1.0046355724334717 - rollout_corr/rollout_is_seq_min:0.9948148131370544 - rollout_corr/rollout_is_seq_max_deviation:0.005185186862945557 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0920318001881242) - rollout_corr/training_log_ppl:np.float64(0.0861618878916488) - rollout_corr/kl:np.float64(0.0007285167887651767) - rollout_corr/k3_kl:np.float64(0.0005457256108627462) - rollout_corr/rollout_ppl:np.float64(1.0912344371899962) - rollout_corr/rollout_log_ppl:np.float64(0.08543337112496374) - rollout_corr/log_ppl_diff:np.float64(0.0007285167666850612) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013972546003060415) - rollout_corr/log_ppl_diff_max:np.float64(0.014810048043727875) - rollout_corr/log_ppl_diff_min:np.float64(-0.0045769959688186646) - rollout_corr/ppl_ratio:np.float64(1.0007309333886951) - rollout_corr/chi2_token:np.float64(0.0006357615347951651) - rollout_corr/chi2_seq:np.float64(0.37331055640242994) - actor/pg_loss:np.float64(0.002819624845869839) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005845594248512498) - actor/ppo_kl:np.float64(0.00019738566773241928) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.56640625) - self_distillation/token_reweight_w_mean:np.float64(118144.65402136883) - self_distillation/token_reweight_w_std:np.float64(2489355.6096611773) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007247163448483) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0432515628344845) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.2655489929020405) - perf/mfu/actor:np.float64(0.043623625719631715) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.27901077270508) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:1 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0013405305799096823 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0013405305799096823 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:490.2734375 - response_length/max:1590.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:490.2734375 - response_length_non_aborted/max:1590.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:355.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001036711037158966 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1733751446008682) - timing_s/agent_loop/generate_sequences/max:np.float64(11.362804541364312) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.0733919305566815) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.362804541364312) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1590 - timing_s/gen:17.271388456225395 - timing_s/reward:0.06370862200856209 - timing_s/old_log_prob:2.4125118106603622 - timing_s/adv:0.5772267878055573 - timing_s/update_actor:16.07286419905722 - timing_s/step:36.48842146247625 - timing_s/stop_profile:0.00012125447392463684 - timing_per_token_ms/update_actor:0.08296699564878861 - timing_per_token_ms/adv:0.002979604120281001 - timing_per_token_ms/gen:0.13760966023604013 - perf/total_num_tokens:193726 - perf/time_per_step:36.48842146247625 - perf/throughput:663.6557304870765 (TaskRunner pid=2074004) Training Progress: 42%|████▏ | 42/100 [37:05<52:01, 53.81s/it] (TaskRunner pid=2074004) step:43 - global_seqlen/min:18165 - global_seqlen/max:29195 - global_seqlen/minmax_diff:11030 - global_seqlen/balanced_min:23675 - global_seqlen/balanced_max:23697 - global_seqlen/mean:23692.125 - actor/entropy:0.1023547351360321 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8032844066619873 - training/rollout_probs_diff_mean:0.0026671267114579678 - training/rollout_probs_diff_std:0.01125924102962017 - training/rollout_actor_probs_pearson_corr:0.9977118372917175 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.89453125 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.89453125 - rollout_corr/rollout_is_mean:1.0000076293945312 - rollout_corr/rollout_is_ratio_fraction_high:1.6602332834736444e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012451749353203923 - rollout_corr/rollout_is_max:2.4638938903808594 - rollout_corr/rollout_is_min:0.01720614731311798 - rollout_corr/rollout_is_std:0.03037957102060318 - rollout_corr/rollout_is_eff_sample_size:0.9990778136835871 - rollout_corr/rollout_is_seq_mean:1.0000402927398682 - rollout_corr/rollout_is_seq_std:0.0017050451133400202 - rollout_corr/rollout_is_seq_max:1.0078315734863281 - rollout_corr/rollout_is_seq_min:0.9947986006736755 - rollout_corr/rollout_is_seq_max_deviation:0.007831573486328125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1006618356332183) - rollout_corr/training_log_ppl:np.float64(0.09350979735609144) - rollout_corr/kl:np.float64(0.000630945518594217) - rollout_corr/k3_kl:np.float64(0.0005851791151911812) - rollout_corr/rollout_ppl:np.float64(1.0999378017149866) - rollout_corr/rollout_log_ppl:np.float64(0.09287885023513809) - rollout_corr/log_ppl_diff:np.float64(0.0006309471209533513) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015661002835258842) - rollout_corr/log_ppl_diff_max:np.float64(0.007478758692741394) - rollout_corr/log_ppl_diff_min:np.float64(-0.007118046283721924) - rollout_corr/ppl_ratio:np.float64(1.000633135670796) - rollout_corr/chi2_token:np.float64(0.0010510697029531002) - rollout_corr/chi2_seq:np.float64(0.48250076407566667) - actor/pg_loss:np.float64(0.0034622400999069214) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007893792012509948) - actor/ppo_kl:np.float64(0.00017401028439456567) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.89453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.89453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(130333.681850672) - self_distillation/token_reweight_w_std:np.float64(2574542.6272122907) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005807324778289) - self_distillation/token_reweight_w_clip_frac:np.float64(0.056435555168718565) - self_distillation/empty_target_batch:np.float64(0.10546875) - actor/grad_norm:np.float64(0.34318234771490097) - perf/mfu/actor:np.float64(0.04306781395749705) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.32390975952148) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:1 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010272692888975143 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010272692888975143 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:470.56640625 - response_length/max:1498.0 - response_length/min:117.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:470.56640625 - response_length_non_aborted/max:1498.0 - response_length_non_aborted/min:117.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.8125 - prompt_length/max:365.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.386613965034485e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1191709153354168) - timing_s/agent_loop/generate_sequences/max:np.float64(10.657457020133734) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.579246542722103) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.657457020133734) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:346 - timing_s/agent_loop/slowest/response_length:1498 - timing_s/gen:16.503861967474222 - timing_s/reward:0.05490053817629814 - timing_s/old_log_prob:2.4478375520557165 - timing_s/adv:0.5797670539468527 - timing_s/update_actor:15.963536273688078 - timing_s/step:35.651620699092746 - timing_s/stop_profile:3.4825876355171204e-05 - timing_per_token_ms/update_actor:0.08422385219607822 - timing_per_token_ms/adv:0.003058859504723894 - timing_per_token_ms/gen:0.1370013030131094 - perf/total_num_tokens:189537 - perf/time_per_step:35.651620699092746 - perf/throughput:664.545525152042 (TaskRunner pid=2074004) Training Progress: 43%|████▎ | 43/100 [37:41<45:57, 48.37s/it] (TaskRunner pid=2074004) step:44 - global_seqlen/min:18372 - global_seqlen/max:30473 - global_seqlen/minmax_diff:12101 - global_seqlen/balanced_min:24351 - global_seqlen/balanced_max:24371 - global_seqlen/mean:24365.0 - actor/entropy:0.0982256531715393 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3797517716884613 - training/rollout_probs_diff_mean:0.0026915573980659246 - training/rollout_probs_diff_std:0.011107339523732662 - training/rollout_actor_probs_pearson_corr:0.9976074695587158 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0001195669174194 - rollout_corr/rollout_is_ratio_fraction_high:1.613319545867853e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.0332990465685725e-05 - rollout_corr/rollout_is_max:3.7719850540161133 - rollout_corr/rollout_is_min:0.08076421916484833 - rollout_corr/rollout_is_std:0.030701594427227974 - rollout_corr/rollout_is_eff_sample_size:0.9990586567374643 - rollout_corr/rollout_is_seq_mean:1.000076413154602 - rollout_corr/rollout_is_seq_std:0.0018174588913097978 - rollout_corr/rollout_is_seq_max:1.0099468231201172 - rollout_corr/rollout_is_seq_min:0.9935795664787292 - rollout_corr/rollout_is_seq_max_deviation:0.009946823120117188 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0983206499367952) - rollout_corr/training_log_ppl:np.float64(0.091351964041678) - rollout_corr/kl:np.float64(0.0005401147962658115) - rollout_corr/k3_kl:np.float64(0.0005354781641990769) - rollout_corr/rollout_ppl:np.float64(1.0977034606039524) - rollout_corr/rollout_log_ppl:np.float64(0.09081184863316594) - rollout_corr/log_ppl_diff:np.float64(0.0005401154085120652) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013354642323974986) - rollout_corr/log_ppl_diff_max:np.float64(0.010618716478347778) - rollout_corr/log_ppl_diff_min:np.float64(-0.006194006651639938) - rollout_corr/ppl_ratio:np.float64(1.000541973626241) - rollout_corr/chi2_token:np.float64(0.0011658316943794489) - rollout_corr/chi2_seq:np.float64(0.8551819822750986) - actor/pg_loss:np.float64(0.00199379853438586) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002786078871395148) - actor/ppo_kl:np.float64(0.00011415650995871829) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(42518.53975814115) - self_distillation/token_reweight_w_std:np.float64(780929.215723624) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994724853895605) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03163288858195301) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.21826045215129852) - perf/mfu/actor:np.float64(0.04429235206402637) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.40694427490234) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:1 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01199503056704998 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01199503056704998 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:484.25 - response_length/max:1568.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:484.25 - response_length_non_aborted/max:1568.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.15625 - prompt_length/max:460.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.6304274797439575e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7051511611789465) - timing_s/agent_loop/generate_sequences/max:np.float64(11.325649071484804) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.876759573868185) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.325649071484804) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:1568 - timing_s/gen:17.214315151795745 - timing_s/reward:0.06438344530761242 - timing_s/old_log_prob:2.4776456244289875 - timing_s/adv:0.5917312819510698 - timing_s/update_actor:15.933018373325467 - timing_s/step:36.368578501045704 - timing_s/stop_profile:4.634261131286621e-05 - timing_per_token_ms/update_actor:0.08174132143097408 - timing_per_token_ms/adv:0.003035764836605119 - timing_per_token_ms/gen:0.13886095727765024 - perf/total_num_tokens:194920 - perf/time_per_step:36.368578501045704 - perf/throughput:669.946448396366 (TaskRunner pid=2074004) Training Progress: 44%|████▍ | 44/100 [38:17<41:47, 44.78s/it] (TaskRunner pid=2074004) step:45 - global_seqlen/min:18847 - global_seqlen/max:30065 - global_seqlen/minmax_diff:11218 - global_seqlen/balanced_min:23771 - global_seqlen/balanced_max:23792 - global_seqlen/mean:23779.375 - actor/entropy:0.11290297657251358 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7185672521591187 - training/rollout_probs_diff_mean:0.0027959917206317186 - training/rollout_probs_diff_std:0.011578291654586792 - training/rollout_actor_probs_pearson_corr:0.9977681040763855 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0000535249710083 - rollout_corr/rollout_is_ratio_fraction_high:8.408802386838943e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0001177232334157452 - rollout_corr/rollout_is_max:2.5759212970733643 - rollout_corr/rollout_is_min:0.01617681048810482 - rollout_corr/rollout_is_std:0.031018352136015892 - rollout_corr/rollout_is_eff_sample_size:0.9990390246046145 - rollout_corr/rollout_is_seq_mean:1.0000488758087158 - rollout_corr/rollout_is_seq_std:0.001919785514473915 - rollout_corr/rollout_is_seq_max:1.009157657623291 - rollout_corr/rollout_is_seq_min:0.9913671612739563 - rollout_corr/rollout_is_seq_max_deviation:0.009157657623291016 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1144616291858256) - rollout_corr/training_log_ppl:np.float64(0.10538233329680224) - rollout_corr/kl:np.float64(0.0005276777127427579) - rollout_corr/k3_kl:np.float64(0.000598768252578985) - rollout_corr/rollout_ppl:np.float64(1.1138749476522207) - rollout_corr/rollout_log_ppl:np.float64(0.1048546542024269) - rollout_corr/log_ppl_diff:np.float64(0.000527679094375344) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015542648143309634) - rollout_corr/log_ppl_diff_max:np.float64(0.010761316865682602) - rollout_corr/log_ppl_diff_min:np.float64(-0.007235199213027954) - rollout_corr/ppl_ratio:np.float64(1.000530268996954) - rollout_corr/chi2_token:np.float64(0.001289344858378172) - rollout_corr/chi2_seq:np.float64(0.600404967321083) - actor/pg_loss:np.float64(0.003451700205914676) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003801528589519876) - actor/ppo_kl:np.float64(-1.3993248870747266e-07) - actor/pg_clipfrac_lower:np.float64(1.1601573987718439e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(70150.05770411994) - self_distillation/token_reweight_w_std:np.float64(1347604.018264451) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003975154832006) - self_distillation/token_reweight_w_clip_frac:np.float64(0.047006876673549414) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.3081394359469414) - perf/mfu/actor:np.float64(0.04282063737580456) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.50539779663086) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:2 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011149020865559578 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011149020865559578 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:464.54296875 - response_length/max:1638.0 - response_length/min:73.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:464.54296875 - response_length_non_aborted/max:1638.0 - response_length_non_aborted/min:73.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.5625 - prompt_length/max:437.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003325045108795166 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6967908591032028) - timing_s/agent_loop/generate_sequences/max:np.float64(11.396734926849604) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.698892642998544) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.396734926849604) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:375 - timing_s/agent_loop/slowest/response_length:1638 - timing_s/gen:17.557129818946123 - timing_s/reward:0.05820976942777634 - timing_s/old_log_prob:2.601362394168973 - timing_s/adv:0.6350415386259556 - timing_s/update_actor:16.227487675845623 - timing_s/step:37.15261420793831 - timing_s/stop_profile:0.00015691295266151428 - timing_per_token_ms/update_actor:0.08530232436641849 - timing_per_token_ms/adv:0.003338195067290223 - timing_per_token_ms/gen:0.1476344342048731 - perf/total_num_tokens:190235 - perf/time_per_step:37.15261420793831 - perf/throughput:640.0458085374546 (TaskRunner pid=2074004) Training Progress: 45%|████▌ | 45/100 [38:55<39:05, 42.64s/it] (TaskRunner pid=2074004) step:46 - global_seqlen/min:14903 - global_seqlen/max:28076 - global_seqlen/minmax_diff:13173 - global_seqlen/balanced_min:20663 - global_seqlen/balanced_max:20685 - global_seqlen/mean:20676.625 - actor/entropy:0.09572755545377731 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6351487040519714 - training/rollout_probs_diff_mean:0.0026079132221639156 - training/rollout_probs_diff_std:0.01122567243874073 - training/rollout_actor_probs_pearson_corr:0.9975275993347168 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.7890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:1.0001404285430908 - rollout_corr/rollout_is_ratio_fraction_high:3.958632441936061e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.9379482991062105e-05 - rollout_corr/rollout_is_max:5.243876934051514 - rollout_corr/rollout_is_min:0.22313041985034943 - rollout_corr/rollout_is_std:0.030579039826989174 - rollout_corr/rollout_is_eff_sample_size:0.9990660338595477 - rollout_corr/rollout_is_seq_mean:1.0001130104064941 - rollout_corr/rollout_is_seq_std:0.002018552739173174 - rollout_corr/rollout_is_seq_max:1.0071282386779785 - rollout_corr/rollout_is_seq_min:0.9935094714164734 - rollout_corr/rollout_is_seq_max_deviation:0.007128238677978516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0899349767714739) - rollout_corr/training_log_ppl:np.float64(0.08403534088211018) - rollout_corr/kl:np.float64(0.0005288002676930859) - rollout_corr/k3_kl:np.float64(0.0005856861293978) - rollout_corr/rollout_ppl:np.float64(1.089340966194868) - rollout_corr/rollout_log_ppl:np.float64(0.08350654012610903) - rollout_corr/log_ppl_diff:np.float64(0.0005288007560011465) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017128633153333794) - rollout_corr/log_ppl_diff_max:np.float64(0.009660359472036362) - rollout_corr/log_ppl_diff_min:np.float64(-0.006222076714038849) - rollout_corr/ppl_ratio:np.float64(1.000531758647412) - rollout_corr/chi2_token:np.float64(0.0013514722231775522) - rollout_corr/chi2_seq:np.float64(0.6378451467026025) - actor/pg_loss:np.float64(0.001488825073465705) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003111429728051007) - actor/ppo_kl:np.float64(0.00012299883569388825) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7890625) - self_distillation/token_reweight_w_mean:np.float64(29391.911929843947) - self_distillation/token_reweight_w_std:np.float64(617711.6287392898) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997125186491758) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02729684312362224) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.15495813265442848) - perf/mfu/actor:np.float64(0.03730698959695489) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.52793884277344) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:2 - critic/score/mean:0.7890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007471918594092131 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007471918594092131 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:394.70703125 - response_length/max:1505.0 - response_length/min:67.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:394.70703125 - response_length_non_aborted/max:1505.0 - response_length_non_aborted/min:67.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.4375 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011683255434036255 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8251722864806652) - timing_s/agent_loop/generate_sequences/max:np.float64(10.226146340370178) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7836934844599455) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.226146340370178) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:1505 - timing_s/gen:16.00960760936141 - timing_s/reward:0.0563168041408062 - timing_s/old_log_prob:2.468111554160714 - timing_s/adv:0.5863060262054205 - timing_s/update_actor:15.916377088055015 - timing_s/step:35.136502992361784 - timing_s/stop_profile:0.00013731792569160461 - timing_per_token_ms/update_actor:0.09622204474893155 - timing_per_token_ms/adv:0.0035444978702122596 - timing_per_token_ms/gen:0.1584403741833976 - perf/total_num_tokens:165413 - perf/time_per_step:35.136502992361784 - perf/throughput:588.4656479472311 (TaskRunner pid=2074004) Training Progress: 46%|████▌ | 46/100 [39:30<36:21, 40.40s/it] (TaskRunner pid=2074004) step:47 - global_seqlen/min:18667 - global_seqlen/max:30924 - global_seqlen/minmax_diff:12257 - global_seqlen/balanced_min:24432 - global_seqlen/balanced_max:24607 - global_seqlen/mean:24464.125 - actor/entropy:0.093343086540699 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28589993715286255 - training/rollout_probs_diff_mean:0.0024511476512998343 - training/rollout_probs_diff_std:0.010416332632303238 - training/rollout_actor_probs_pearson_corr:0.997839629650116 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.78125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:1.0000395774841309 - rollout_corr/rollout_is_ratio_fraction_high:1.5632695067324676e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.471443000715226e-05 - rollout_corr/rollout_is_max:3.1934804916381836 - rollout_corr/rollout_is_min:0.2297939509153366 - rollout_corr/rollout_is_std:0.028684185817837715 - rollout_corr/rollout_is_eff_sample_size:0.9991780129114268 - rollout_corr/rollout_is_seq_mean:0.9999590516090393 - rollout_corr/rollout_is_seq_std:0.0018708055140450597 - rollout_corr/rollout_is_seq_max:1.0055705308914185 - rollout_corr/rollout_is_seq_min:0.9900708198547363 - rollout_corr/rollout_is_seq_max_deviation:0.009929180145263672 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.095050911884755) - rollout_corr/training_log_ppl:np.float64(0.08821879087918205) - rollout_corr/kl:np.float64(0.00046318759317642844) - rollout_corr/k3_kl:np.float64(0.0005760851533125333) - rollout_corr/rollout_ppl:np.float64(1.0945486696437001) - rollout_corr/rollout_log_ppl:np.float64(0.0877556023770012) - rollout_corr/log_ppl_diff:np.float64(0.0004631885021808557) - rollout_corr/log_ppl_abs_diff:np.float64(0.001563526107929647) - rollout_corr/log_ppl_diff_max:np.float64(0.01143985241651535) - rollout_corr/log_ppl_diff_min:np.float64(-0.00824648141860962) - rollout_corr/ppl_ratio:np.float64(1.0004658417310566) - rollout_corr/chi2_token:np.float64(0.001373328734189272) - rollout_corr/chi2_seq:np.float64(0.9047737107612193) - actor/pg_loss:np.float64(0.0018651984864845872) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004548156155124161) - actor/ppo_kl:np.float64(-4.627550882929654e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_w_mean:np.float64(51240.47999361623) - self_distillation/token_reweight_w_std:np.float64(1059937.9959096338) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991865828633308) - self_distillation/token_reweight_w_clip_frac:np.float64(0.035300534596899524) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.20760727860033512) - perf/mfu/actor:np.float64(0.04301014037057002) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.6991195678711) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:2 - critic/score/mean:0.78125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010366430506110191 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010366430506110191 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:499.75390625 - response_length/max:2319.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:499.75390625 - response_length_non_aborted/max:2319.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.75 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.209476411342621e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8284339234232903) - timing_s/agent_loop/generate_sequences/max:np.float64(14.41846733354032) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.81055764901248) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.41846733354032) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:2319 - timing_s/gen:20.333935126662254 - timing_s/reward:0.06512773968279362 - timing_s/old_log_prob:2.491063568741083 - timing_s/adv:0.754039952531457 - timing_s/update_actor:16.0744774825871 - timing_s/step:39.806080570444465 - timing_s/stop_profile:0.00010729581117630005 - timing_per_token_ms/update_actor:0.08213290625858834 - timing_per_token_ms/adv:0.0038527841918087045 - timing_per_token_ms/gen:0.1589370950285082 - perf/total_num_tokens:195713 - perf/time_per_step:39.806080570444465 - perf/throughput:614.5826127419417 (TaskRunner pid=2074004) Training Progress: 47%|████▋ | 47/100 [40:10<35:32, 40.24s/it] (TaskRunner pid=2074004) step:48 - global_seqlen/min:14563 - global_seqlen/max:26256 - global_seqlen/minmax_diff:11693 - global_seqlen/balanced_min:21802 - global_seqlen/balanced_max:21832 - global_seqlen/mean:21823.375 - actor/entropy:0.08697269856929779 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8820101022720337 - training/rollout_probs_diff_mean:0.0027279374189674854 - training/rollout_probs_diff_std:0.01222990918904543 - training/rollout_actor_probs_pearson_corr:0.9968819618225098 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.7265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:1.0002572536468506 - rollout_corr/rollout_is_ratio_fraction_high:4.7129349695751444e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013196218060329556 - rollout_corr/rollout_is_max:2.5243570804595947 - rollout_corr/rollout_is_min:0.09723177552223206 - rollout_corr/rollout_is_std:0.03248079866170883 - rollout_corr/rollout_is_eff_sample_size:0.9989465854769791 - rollout_corr/rollout_is_seq_mean:1.0003472566604614 - rollout_corr/rollout_is_seq_std:0.002232569269835949 - rollout_corr/rollout_is_seq_max:1.0138044357299805 - rollout_corr/rollout_is_seq_min:0.9910656213760376 - rollout_corr/rollout_is_seq_max_deviation:0.013804435729980469 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0930587616749108) - rollout_corr/training_log_ppl:np.float64(0.08755744152949774) - rollout_corr/kl:np.float64(0.0005523046467796888) - rollout_corr/k3_kl:np.float64(0.00072532679760684) - rollout_corr/rollout_ppl:np.float64(1.0924336458556354) - rollout_corr/rollout_log_ppl:np.float64(0.0870051367310225) - rollout_corr/log_ppl_diff:np.float64(0.0005523047984752338) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017461000934417825) - rollout_corr/log_ppl_diff_max:np.float64(0.012880250811576843) - rollout_corr/log_ppl_diff_min:np.float64(-0.008728347718715668) - rollout_corr/ppl_ratio:np.float64(1.0005556710530072) - rollout_corr/chi2_token:np.float64(0.0017671133391559124) - rollout_corr/chi2_seq:np.float64(0.6951510151848197) - actor/pg_loss:np.float64(0.002507986268028617) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006494858009773452) - actor/ppo_kl:np.float64(0.0003249875776454303) - actor/pg_clipfrac_lower:np.float64(4.164445726928534e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_w_mean:np.float64(161986.24940627953) - self_distillation/token_reweight_w_std:np.float64(2150402.851084696) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997133247088641) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04499576549278572) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.378052793443203) - perf/mfu/actor:np.float64(0.04008844180647516) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.73404788970947) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:2 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010018521919846535 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010018521919846535 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:414.41796875 - response_length/max:1405.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:414.41796875 - response_length_non_aborted/max:1405.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.5625 - prompt_length/max:365.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017635710537433624 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7887936737388372) - timing_s/agent_loop/generate_sequences/max:np.float64(9.995722759515047) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.310525228465849) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.995722759515047) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1405 - timing_s/gen:16.018609369173646 - timing_s/reward:0.06094110943377018 - timing_s/old_log_prob:2.350804327055812 - timing_s/adv:0.5859208330512047 - timing_s/update_actor:15.523810938000679 - timing_s/step:34.628727208822966 - timing_s/stop_profile:0.00015043653547763824 - timing_per_token_ms/update_actor:0.08891733598721943 - timing_per_token_ms/adv:0.0033560392987519384 - timing_per_token_ms/gen:0.15098933339466727 - perf/total_num_tokens:174587 - perf/time_per_step:34.628727208822966 - perf/throughput:630.2101393561956 (TaskRunner pid=2074004) Training Progress: 48%|████▊ | 48/100 [40:45<33:25, 38.57s/it] (TaskRunner pid=2074004) step:49 - global_seqlen/min:15971 - global_seqlen/max:23148 - global_seqlen/minmax_diff:7177 - global_seqlen/balanced_min:19974 - global_seqlen/balanced_max:20001 - global_seqlen/mean:19989.25 - actor/entropy:0.08722288906574249 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4621162414550781 - training/rollout_probs_diff_mean:0.0027316466439515352 - training/rollout_probs_diff_std:0.012118976563215256 - training/rollout_actor_probs_pearson_corr:0.9968992471694946 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9296875 - self_distillation/correct_sample_fraction:0.76171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9296875 - rollout_corr/rollout_is_mean:1.0001474618911743 - rollout_corr/rollout_is_ratio_fraction_high:4.222527059027925e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.500686428509653e-05 - rollout_corr/rollout_is_max:2.718278646469116 - rollout_corr/rollout_is_min:0.33812370896339417 - rollout_corr/rollout_is_std:0.03343567997217178 - rollout_corr/rollout_is_eff_sample_size:0.9988835415737872 - rollout_corr/rollout_is_seq_mean:1.0000269412994385 - rollout_corr/rollout_is_seq_std:0.0024658842012286186 - rollout_corr/rollout_is_seq_max:1.0119198560714722 - rollout_corr/rollout_is_seq_min:0.9833274483680725 - rollout_corr/rollout_is_seq_max_deviation:0.01667255163192749 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0898574301972985) - rollout_corr/training_log_ppl:np.float64(0.084673610603204) - rollout_corr/kl:np.float64(0.0007291726103133556) - rollout_corr/k3_kl:np.float64(0.0008918259389361083) - rollout_corr/rollout_ppl:np.float64(1.0890288641676307) - rollout_corr/rollout_log_ppl:np.float64(0.08394443713768851) - rollout_corr/log_ppl_diff:np.float64(0.0007291734655154869) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020636038680095226) - rollout_corr/log_ppl_diff_max:np.float64(0.021112143993377686) - rollout_corr/log_ppl_diff_min:np.float64(-0.007896922528743744) - rollout_corr/ppl_ratio:np.float64(1.0007340346928686) - rollout_corr/chi2_token:np.float64(0.002316090976819396) - rollout_corr/chi2_seq:np.float64(1.0663579376414418) - actor/pg_loss:np.float64(0.0028226502472534776) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006949842643280135) - actor/ppo_kl:np.float64(0.00011967911359889172) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_w_mean:np.float64(197467.6685416242) - self_distillation/token_reweight_w_std:np.float64(2992394.127208988) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000012104865164) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04289876340772025) - self_distillation/empty_target_batch:np.float64(0.0703125) - actor/grad_norm:np.float64(0.39689869433641434) - perf/mfu/actor:np.float64(0.03667143020489837) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.6230697631836) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:2 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0072983745485544205 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0072983745485544205 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:370.0390625 - response_length/max:1377.0 - response_length/min:75.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:370.0390625 - response_length_non_aborted/max:1377.0 - response_length_non_aborted/min:75.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.625 - prompt_length/max:362.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011089816689491272 - timing_s/agent_loop/generate_sequences/min:np.float64(0.894102580845356) - timing_s/agent_loop/generate_sequences/max:np.float64(9.03606103733182) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6704066668535233) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.03606103733182) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1377 - timing_s/gen:15.202601235359907 - timing_s/reward:0.05952438898384571 - timing_s/old_log_prob:2.3819591104984283 - timing_s/adv:0.6096546649932861 - timing_s/update_actor:15.689758809283376 - timing_s/step:34.03324189968407 - timing_s/stop_profile:0.0001423526555299759 - timing_per_token_ms/update_actor:0.09811372868719047 - timing_per_token_ms/adv:0.0038123908162717842 - timing_per_token_ms/gen:0.16048349240325036 - perf/total_num_tokens:159914 - perf/time_per_step:34.03324189968407 - perf/throughput:587.34486884676 (TaskRunner pid=2074004) Training Progress: 49%|████▉ | 49/100 [41:19<31:38, 37.22s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. We need to find the charge $ q $ using the formula: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Solving for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substitute the values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest answer is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40/actor (TaskRunner pid=2074004) step:50 - global_seqlen/min:18143 - global_seqlen/max:30858 - global_seqlen/minmax_diff:12715 - global_seqlen/balanced_min:24044 - global_seqlen/balanced_max:26361 - global_seqlen/mean:24342.0 - actor/entropy:0.10675854980945587 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5083586573600769 - training/rollout_probs_diff_mean:0.0026588926557451487 - training/rollout_probs_diff_std:0.011075353249907494 - training/rollout_actor_probs_pearson_corr:0.9978457689285278 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.58203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:0.9999751448631287 - rollout_corr/rollout_is_ratio_fraction_high:2.371916525589768e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.115749758668244e-05 - rollout_corr/rollout_is_max:3.3773059844970703 - rollout_corr/rollout_is_min:0.2692754864692688 - rollout_corr/rollout_is_std:0.030405066907405853 - rollout_corr/rollout_is_eff_sample_size:0.9990761478330206 - rollout_corr/rollout_is_seq_mean:0.9999284744262695 - rollout_corr/rollout_is_seq_std:0.002185159595683217 - rollout_corr/rollout_is_seq_max:1.0124692916870117 - rollout_corr/rollout_is_seq_min:0.9912188649177551 - rollout_corr/rollout_is_seq_max_deviation:0.012469291687011719 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1083718882873654) - rollout_corr/training_log_ppl:np.float64(0.10039229939866345) - rollout_corr/kl:np.float64(0.0004886855121668532) - rollout_corr/k3_kl:np.float64(0.0005853448297905572) - rollout_corr/rollout_ppl:np.float64(1.1078288545832038) - rollout_corr/rollout_log_ppl:np.float64(0.09990361310701701) - rollout_corr/log_ppl_diff:np.float64(0.0004886862916464452) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016839853378769476) - rollout_corr/log_ppl_diff_max:np.float64(0.010547831654548645) - rollout_corr/log_ppl_diff_min:np.float64(-0.011270195245742798) - rollout_corr/ppl_ratio:np.float64(1.000491572311148) - rollout_corr/chi2_token:np.float64(0.0013913614675402641) - rollout_corr/chi2_seq:np.float64(1.0534377871081233) - actor/pg_loss:np.float64(0.0014425351982936263) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00040900465501181316) - actor/ppo_kl:np.float64(-0.00011984414344468064) - actor/pg_clipfrac_lower:np.float64(3.814697265625e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58203125) - self_distillation/token_reweight_w_mean:np.float64(121532.27847940498) - self_distillation/token_reweight_w_std:np.float64(2120513.0500425613) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004689684137702) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03253905474412022) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.20551946759223938) - perf/mfu/actor:np.float64(0.04243176899328058) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.7705078125) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.63984375) - val-aux/sciknoweval/reward/std@16:np.float64(0.15617237535672698) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.70185) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.12792221226476658) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5791125) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.13673480051872064) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6415) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1561483190784494) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7504125) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09557563742116812) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5252625) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10832359511761105) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6525000000000001) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.12038866118646771) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7888999999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06876976879223133) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.47989999999999994) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0796137581343784) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6589125000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07946446968361667) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8169375000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04228047841467854) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.448425) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.047402393710538074) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6619124999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.05008887435519682) - val-aux/sciknoweval/score/mean@16:np.float64(0.63984375) - val-aux/sciknoweval/score/std@16:np.float64(0.15617237535672698) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.70185) - val-aux/sciknoweval/score/best@2/std:np.float64(0.12792221226476658) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5791125) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.13673480051872064) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6415) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1561483190784494) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7504125) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09557563742116812) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5252625) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10832359511761105) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6525000000000001) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.12038866118646771) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7888999999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06876976879223133) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.47989999999999994) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0796137581343784) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6589125000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07946446968361667) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8169375000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04228047841467854) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.448425) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.047402393710538074) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6619124999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.05008887435519682) - val-core/sciknoweval/acc/mean@16:np.float64(0.63984375) - val-aux/sciknoweval/acc/std@16:np.float64(0.15617237535672698) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.70185) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.12792221226476658) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5791125) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.13673480051872064) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6415) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1561483190784494) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7504125) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09557563742116812) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5252625) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10832359511761105) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6525000000000001) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.12038866118646771) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7888999999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06876976879223133) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.47989999999999994) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0796137581343784) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6589125000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07946446968361667) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8169375000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04228047841467854) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.448425) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.047402393710538074) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6619124999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.05008887435519682) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9921875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.026059282360820557) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9991375000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.008110749235580297) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.985425) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.03423627902458588) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.992475) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.025689990721416346) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999625) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.000683625445693766) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9733875) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.04275106884274511) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9975375) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.014926669851977312) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.954525) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.047782282724323125) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9996125000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.005601383627055409) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9313375) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.04368822856243866) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999874999999999) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.00039508701573197775) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:2 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0210300050675869 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0210300050675869 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:494.0625 - response_length/max:4102.0 - response_length/min:70.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:494.0625 - response_length_non_aborted/max:4102.0 - response_length_non_aborted/min:70.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.625 - prompt_length/max:355.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.833462536334991e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6969410367310047) - timing_s/agent_loop/generate_sequences/max:np.float64(22.719568610191345) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.79131419971236) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.719568610191345) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:295 - timing_s/agent_loop/slowest/response_length:4102 - timing_s/gen:28.734515454620123 - timing_s/reward:0.06417730078101158 - timing_s/old_log_prob:2.4680988006293774 - timing_s/adv:0.8539764694869518 - timing_s/update_actor:16.26316487416625 - timing_s/step:48.47273708879948 - timing_s/testing:90.87557928822935 - timing_s/save_checkpoint:6.687707267701626 - timing_s/stop_profile:0.0001324862241744995 - timing_per_token_ms/update_actor:0.08351391049506127 - timing_per_token_ms/adv:0.004385303536515856 - timing_per_token_ms/gen:0.22718623857226536 - perf/total_num_tokens:194736 - perf/time_per_step:48.47273708879948 - perf/throughput:502.17919312884584 (TaskRunner pid=2074004) Training Progress: 50%|█████ | 50/100 [43:45<58:14, 69.89s/it] (TaskRunner pid=2074004) step:51 - global_seqlen/min:16973 - global_seqlen/max:21967 - global_seqlen/minmax_diff:4994 - global_seqlen/balanced_min:19550 - global_seqlen/balanced_max:19572 - global_seqlen/mean:19567.375 - actor/entropy:0.11901509016752243 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5083246231079102 - training/rollout_probs_diff_mean:0.003160488326102495 - training/rollout_probs_diff_std:0.012160852551460266 - training/rollout_actor_probs_pearson_corr:0.9976589679718018 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0001776218414307 - rollout_corr/rollout_is_ratio_fraction_high:5.6728575145825744e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.807428871979937e-05 - rollout_corr/rollout_is_max:2.820579767227173 - rollout_corr/rollout_is_min:0.30462947487831116 - rollout_corr/rollout_is_std:0.03463872894644737 - rollout_corr/rollout_is_eff_sample_size:0.9988020720902324 - rollout_corr/rollout_is_seq_mean:1.000294804573059 - rollout_corr/rollout_is_seq_std:0.0022349499631673098 - rollout_corr/rollout_is_seq_max:1.0152208805084229 - rollout_corr/rollout_is_seq_min:0.995274543762207 - rollout_corr/rollout_is_seq_max_deviation:0.015220880508422852 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1107426239177585) - rollout_corr/training_log_ppl:np.float64(0.10169691975897877) - rollout_corr/kl:np.float64(0.0001784627847740694) - rollout_corr/k3_kl:np.float64(0.0006646001246366495) - rollout_corr/rollout_ppl:np.float64(1.1105284076184034) - rollout_corr/rollout_log_ppl:np.float64(0.10151845622021938) - rollout_corr/log_ppl_diff:np.float64(0.00017846353875938803) - rollout_corr/log_ppl_abs_diff:np.float64(0.001676651110756211) - rollout_corr/log_ppl_diff_max:np.float64(0.006288684904575348) - rollout_corr/log_ppl_diff_min:np.float64(-0.010029513388872147) - rollout_corr/ppl_ratio:np.float64(1.0001808677334338) - rollout_corr/chi2_token:np.float64(0.002416089177131653) - rollout_corr/chi2_seq:np.float64(2.1752094991970807) - actor/pg_loss:np.float64(0.0023010852746665478) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00047835509440119495) - actor/ppo_kl:np.float64(-0.0001440780888302129) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(79808.69256585464) - self_distillation/token_reweight_w_std:np.float64(1330398.761181703) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998520778026432) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04437096892797854) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.2404463179409504) - perf/mfu/actor:np.float64(0.03516023093043365) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.7251968383789) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:2 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004383700899779797 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004383700899779797 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:344.29296875 - response_length/max:1233.0 - response_length/min:73.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:344.29296875 - response_length_non_aborted/max:1233.0 - response_length_non_aborted/min:73.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.1875 - prompt_length/max:356.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.046704947948456e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8948994744569063) - timing_s/agent_loop/generate_sequences/max:np.float64(8.36026605591178) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4736403662464) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.36026605591178) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:310 - timing_s/agent_loop/slowest/response_length:1233 - timing_s/gen:14.1562640350312 - timing_s/reward:0.051813943311572075 - timing_s/old_log_prob:2.4304460305720568 - timing_s/adv:0.6007976606488228 - timing_s/update_actor:15.809248071163893 - timing_s/step:33.15378465689719 - timing_s/stop_profile:0.00011606328189373016 - timing_per_token_ms/update_actor:0.10099239212697086 - timing_per_token_ms/adv:0.0038380062517891565 - timing_per_token_ms/gen:0.1606129413203145 - perf/total_num_tokens:156539 - perf/time_per_step:33.15378465689719 - perf/throughput:590.2003407001462 (TaskRunner pid=2074004) Training Progress: 51%|█████ | 51/100 [44:18<48:05, 58.88s/it] (TaskRunner pid=2074004) step:52 - global_seqlen/min:16957 - global_seqlen/max:26308 - global_seqlen/minmax_diff:9351 - global_seqlen/balanced_min:22416 - global_seqlen/balanced_max:22423 - global_seqlen/mean:22420.25 - actor/entropy:0.11742626130580902 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6386399269104004 - training/rollout_probs_diff_mean:0.0029321101028472185 - training/rollout_probs_diff_std:0.011663330718874931 - training/rollout_actor_probs_pearson_corr:0.9977625608444214 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71875 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71875 - rollout_corr/rollout_is_mean:0.9999903440475464 - rollout_corr/rollout_is_ratio_fraction_high:3.6525008908938617e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.305001781787723e-05 - rollout_corr/rollout_is_max:3.6345746517181396 - rollout_corr/rollout_is_min:0.252546101808548 - rollout_corr/rollout_is_std:0.03316900134086609 - rollout_corr/rollout_is_eff_sample_size:0.9989007886433641 - rollout_corr/rollout_is_seq_mean:1.0000351667404175 - rollout_corr/rollout_is_seq_std:0.002520139329135418 - rollout_corr/rollout_is_seq_max:1.0145207643508911 - rollout_corr/rollout_is_seq_min:0.9895867109298706 - rollout_corr/rollout_is_seq_max_deviation:0.014520764350891113 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1248778235167265) - rollout_corr/training_log_ppl:np.float64(0.11452079268201487) - rollout_corr/kl:np.float64(0.0006545804253192955) - rollout_corr/k3_kl:np.float64(0.0007203538166162815) - rollout_corr/rollout_ppl:np.float64(1.1241182647645473) - rollout_corr/rollout_log_ppl:np.float64(0.11386621197380009) - rollout_corr/log_ppl_diff:np.float64(0.0006545807082147803) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016909198930079583) - rollout_corr/log_ppl_diff_max:np.float64(0.011592693626880646) - rollout_corr/log_ppl_diff_min:np.float64(-0.010950211435556412) - rollout_corr/ppl_ratio:np.float64(1.0006579770706594) - rollout_corr/chi2_token:np.float64(0.0015927678905427456) - rollout_corr/chi2_seq:np.float64(0.5052446881309152) - actor/pg_loss:np.float64(0.0011261987965554) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00031590935577696655) - actor/ppo_kl:np.float64(4.731501897392576e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(120173.51662478969) - self_distillation/token_reweight_w_std:np.float64(1645237.523866353) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993055597878993) - self_distillation/token_reweight_w_clip_frac:np.float64(0.026213745499262586) - self_distillation/empty_target_batch:np.float64(0.28125) - actor/grad_norm:np.float64(0.3443529047071934) - perf/mfu/actor:np.float64(0.04109087151546889) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.80995559692383) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:2 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003660490969195962 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003660490969195962 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:427.7890625 - response_length/max:1240.0 - response_length/min:50.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:427.7890625 - response_length_non_aborted/max:1240.0 - response_length_non_aborted/min:50.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.84375 - prompt_length/max:365.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.612925350666046e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6170574445277452) - timing_s/agent_loop/generate_sequences/max:np.float64(9.085840474814177) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.157990079183946) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.085840474814177) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:346 - timing_s/agent_loop/slowest/response_length:1240 - timing_s/gen:14.695505177602172 - timing_s/reward:0.06002468429505825 - timing_s/old_log_prob:2.512907398864627 - timing_s/adv:0.5510312523692846 - timing_s/update_actor:15.702309988439083 - timing_s/step:33.61664153076708 - timing_s/stop_profile:0.00011726468801498413 - timing_per_token_ms/update_actor:0.08754535513898754 - timing_per_token_ms/adv:0.0030721738850441266 - timing_per_token_ms/gen:0.13418837023213628 - perf/total_num_tokens:179362 - perf/time_per_step:33.61664153076708 - perf/throughput:666.9390212428042 (TaskRunner pid=2074004) Training Progress: 52%|█████▏ | 52/100 [44:52<41:03, 51.32s/it] (TaskRunner pid=2074004) step:53 - global_seqlen/min:15926 - global_seqlen/max:32544 - global_seqlen/minmax_diff:16618 - global_seqlen/balanced_min:21720 - global_seqlen/balanced_max:28240 - global_seqlen/mean:22614.625 - actor/entropy:0.09381777793169022 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6157671809196472 - training/rollout_probs_diff_mean:0.0025205647107213736 - training/rollout_probs_diff_std:0.011054250411689281 - training/rollout_actor_probs_pearson_corr:0.9975994229316711 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.73828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:1.0000879764556885 - rollout_corr/rollout_is_ratio_fraction_high:4.539635483524762e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.26341677363962e-05 - rollout_corr/rollout_is_max:2.6310083866119385 - rollout_corr/rollout_is_min:0.3212862014770508 - rollout_corr/rollout_is_std:0.029877066612243652 - rollout_corr/rollout_is_eff_sample_size:0.999108513924274 - rollout_corr/rollout_is_seq_mean:1.0001354217529297 - rollout_corr/rollout_is_seq_std:0.0017414779867976904 - rollout_corr/rollout_is_seq_max:1.0066789388656616 - rollout_corr/rollout_is_seq_min:0.9954082369804382 - rollout_corr/rollout_is_seq_max_deviation:0.006678938865661621 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.098151988349855) - rollout_corr/training_log_ppl:np.float64(0.09104150990060589) - rollout_corr/kl:np.float64(0.00047328945871338846) - rollout_corr/k3_kl:np.float64(0.0005634092783721734) - rollout_corr/rollout_ppl:np.float64(1.0976255191490054) - rollout_corr/rollout_log_ppl:np.float64(0.09056821986359864) - rollout_corr/log_ppl_diff:np.float64(0.0004732900370072457) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015623152985426714) - rollout_corr/log_ppl_diff_max:np.float64(0.005912806838750839) - rollout_corr/log_ppl_diff_min:np.float64(-0.005394577980041504) - rollout_corr/ppl_ratio:np.float64(1.0004753414541483) - rollout_corr/chi2_token:np.float64(0.0013314660172909498) - rollout_corr/chi2_seq:np.float64(0.4222771213389933) - actor/pg_loss:np.float64(0.002000119653530419) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005868266684956325) - actor/ppo_kl:np.float64(0.00012776876268461024) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_w_mean:np.float64(53422.7099313552) - self_distillation/token_reweight_w_std:np.float64(849382.4181836921) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993473722133785) - self_distillation/token_reweight_w_clip_frac:np.float64(0.037509667774429545) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.31460506841540337) - perf/mfu/actor:np.float64(0.03860267849970589) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.8161849975586) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:2 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.057434335350990295 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.057434335350990295 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:430.23828125 - response_length/max:8192.0 - response_length/min:95.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:430.23828125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:276.46875 - prompt_length/max:375.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011334381997585297 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0902089104056358) - timing_s/agent_loop/generate_sequences/max:np.float64(50.08467713370919) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.101931656085071) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.08467713370919) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:375 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.888266975060105 - timing_s/reward:0.061832696199417114 - timing_s/old_log_prob:2.569713741540909 - timing_s/adv:0.5755481589585543 - timing_s/update_actor:16.975983068346977 - timing_s/step:76.15641017071903 - timing_s/stop_profile:2.802535891532898e-05 - timing_per_token_ms/update_actor:0.09383299009129588 - timing_per_token_ms/adv:0.003181282902980672 - timing_per_token_ms/gen:0.5074247280763757 - perf/total_num_tokens:180917 - perf/time_per_step:76.15641017071903 - perf/throughput:296.94972424914766 (TaskRunner pid=2074004) Training Progress: 53%|█████▎ | 53/100 [46:08<46:02, 58.79s/it] (TaskRunner pid=2074004) step:54 - global_seqlen/min:16107 - global_seqlen/max:24016 - global_seqlen/minmax_diff:7909 - global_seqlen/balanced_min:19289 - global_seqlen/balanced_max:19501 - global_seqlen/mean:19320.75 - actor/entropy:0.09417933225631714 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39961040019989014 - training/rollout_probs_diff_mean:0.0026771980337798595 - training/rollout_probs_diff_std:0.011605466715991497 - training/rollout_actor_probs_pearson_corr:0.9973964095115662 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.71875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.0001647472381592 - rollout_corr/rollout_is_ratio_fraction_high:1.1300201549602207e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.040161239681765e-05 - rollout_corr/rollout_is_max:2.7947683334350586 - rollout_corr/rollout_is_min:0.2384055107831955 - rollout_corr/rollout_is_std:0.03145531937479973 - rollout_corr/rollout_is_eff_sample_size:0.9990118978361671 - rollout_corr/rollout_is_seq_mean:1.0001072883605957 - rollout_corr/rollout_is_seq_std:0.0022169763687998056 - rollout_corr/rollout_is_seq_max:1.009650707244873 - rollout_corr/rollout_is_seq_min:0.9886189699172974 - rollout_corr/rollout_is_seq_max_deviation:0.011381030082702637 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0925864586606622) - rollout_corr/training_log_ppl:np.float64(0.08678890572628006) - rollout_corr/kl:np.float64(0.0009032392683874235) - rollout_corr/k3_kl:np.float64(0.0007826630679801383) - rollout_corr/rollout_ppl:np.float64(1.0915671489201486) - rollout_corr/rollout_log_ppl:np.float64(0.08588566524122143) - rollout_corr/log_ppl_diff:np.float64(0.0009032404850586317) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018698280400712974) - rollout_corr/log_ppl_diff_max:np.float64(0.03783611208200455) - rollout_corr/log_ppl_diff_min:np.float64(-0.005891826003789902) - rollout_corr/ppl_ratio:np.float64(1.0009094115812331) - rollout_corr/chi2_token:np.float64(0.0010525539983063936) - rollout_corr/chi2_seq:np.float64(0.43362018815241754) - actor/pg_loss:np.float64(0.002331349765881896) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007941815301819588) - actor/ppo_kl:np.float64(0.0004760969166213158) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_w_mean:np.float64(237838.98983726162) - self_distillation/token_reweight_w_std:np.float64(3087923.700471521) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999815991614014) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04877196221787017) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.3219018206000328) - perf/mfu/actor:np.float64(0.03530995118577071) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.7523651123047) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:2 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.000819264561869204 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.000819264561869204 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:345.6796875 - response_length/max:1721.0 - response_length/min:64.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:345.6796875 - response_length_non_aborted/max:1721.0 - response_length_non_aborted/min:64.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.09375 - prompt_length/max:363.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.285946488380432e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.751097746193409) - timing_s/agent_loop/generate_sequences/max:np.float64(10.377681985497475) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.351018370616657) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.377681985497475) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:350 - timing_s/agent_loop/slowest/response_length:1721 - timing_s/gen:16.176042480394244 - timing_s/reward:0.05808785557746887 - timing_s/old_log_prob:2.318193769082427 - timing_s/adv:0.6144152693450451 - timing_s/update_actor:15.822785630822182 - timing_s/step:35.07668521627784 - timing_s/stop_profile:0.00010946020483970642 - timing_per_token_ms/update_actor:0.10236912148093488 - timing_per_token_ms/adv:0.00397509975897057 - timing_per_token_ms/gen:0.18279253373555546 - perf/total_num_tokens:154566 - perf/time_per_step:35.07668521627784 - perf/throughput:550.8145904001764 (TaskRunner pid=2074004) Training Progress: 54%|█████▍ | 54/100 [46:43<39:37, 51.69s/it] (TaskRunner pid=2074004) step:55 - global_seqlen/min:14652 - global_seqlen/max:26201 - global_seqlen/minmax_diff:11549 - global_seqlen/balanced_min:20421 - global_seqlen/balanced_max:21183 - global_seqlen/mean:20601.125 - actor/entropy:0.085003562271595 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7348287105560303 - training/rollout_probs_diff_mean:0.0025023941416293383 - training/rollout_probs_diff_std:0.01154287438839674 - training/rollout_actor_probs_pearson_corr:0.9971373081207275 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.89453125 - self_distillation/correct_sample_fraction:0.7421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.89453125 - rollout_corr/rollout_is_mean:1.0001779794692993 - rollout_corr/rollout_is_ratio_fraction_high:1.0315978215658106e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013410771498456597 - rollout_corr/rollout_is_max:2.935729742050171 - rollout_corr/rollout_is_min:0.038828011602163315 - rollout_corr/rollout_is_std:0.030352089554071426 - rollout_corr/rollout_is_eff_sample_size:0.999079955499621 - rollout_corr/rollout_is_seq_mean:1.0003106594085693 - rollout_corr/rollout_is_seq_std:0.002173425629734993 - rollout_corr/rollout_is_seq_max:1.0108247995376587 - rollout_corr/rollout_is_seq_min:0.9864917993545532 - rollout_corr/rollout_is_seq_max_deviation:0.013508200645446777 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.085447907447815) - rollout_corr/training_log_ppl:np.float64(0.0803521656380326) - rollout_corr/kl:np.float64(0.0008227331833889906) - rollout_corr/k3_kl:np.float64(0.0011211537594633114) - rollout_corr/rollout_ppl:np.float64(1.0845554950647056) - rollout_corr/rollout_log_ppl:np.float64(0.0795294317031221) - rollout_corr/log_ppl_diff:np.float64(0.0008227339349105023) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020583992882166058) - rollout_corr/log_ppl_diff_max:np.float64(0.04965774714946747) - rollout_corr/log_ppl_diff_min:np.float64(-0.009343862533569336) - rollout_corr/ppl_ratio:np.float64(1.0008315925952047) - rollout_corr/chi2_token:np.float64(0.002741375006735325) - rollout_corr/chi2_seq:np.float64(0.736898449016735) - actor/pg_loss:np.float64(0.001779418089427054) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009461770737289044) - actor/ppo_kl:np.float64(0.0004943441474765109) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.89453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.89453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_w_mean:np.float64(220652.25802971283) - self_distillation/token_reweight_w_std:np.float64(2705884.476536175) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000388199230656) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0328305515286047) - self_distillation/empty_target_batch:np.float64(0.10546875) - actor/grad_norm:np.float64(0.42648517340421677) - perf/mfu/actor:np.float64(0.037390062668015575) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.62273788452148) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:2 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006393327843397856 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006393327843397856 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:378.66015625 - response_length/max:2305.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:378.66015625 - response_length_non_aborted/max:2305.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.125 - prompt_length/max:383.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012177042663097382 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9361808206886053) - timing_s/agent_loop/generate_sequences/max:np.float64(13.556594530120492) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.681363452458754) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.556594530120492) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:318 - timing_s/agent_loop/slowest/response_length:2305 - timing_s/gen:19.531876169145107 - timing_s/reward:0.05353007838129997 - timing_s/old_log_prob:2.345628321170807 - timing_s/adv:0.5701816361397505 - timing_s/update_actor:15.82276027649641 - timing_s/step:38.36092114262283 - timing_s/stop_profile:0.00011600367724895477 - timing_per_token_ms/update_actor:0.0960066517999406 - timing_per_token_ms/adv:0.003459651087863833 - timing_per_token_ms/gen:0.20149041304295684 - perf/total_num_tokens:164809 - perf/time_per_step:38.36092114262283 - perf/throughput:537.0341583667052 (TaskRunner pid=2074004) Training Progress: 55%|█████▌ | 55/100 [47:22<35:46, 47.70s/it] (TaskRunner pid=2074004) step:56 - global_seqlen/min:16673 - global_seqlen/max:30667 - global_seqlen/minmax_diff:13994 - global_seqlen/balanced_min:21519 - global_seqlen/balanced_max:21670 - global_seqlen/mean:21542.25 - actor/entropy:0.0999220460653305 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9910814762115479 - training/rollout_probs_diff_mean:0.0026262723840773106 - training/rollout_probs_diff_std:0.011972086504101753 - training/rollout_actor_probs_pearson_corr:0.9973817467689514 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.92578125 - self_distillation/correct_sample_fraction:0.6953125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.92578125 - rollout_corr/rollout_is_mean:1.0003807544708252 - rollout_corr/rollout_is_ratio_fraction_high:4.651249400922097e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.372248703381047e-05 - rollout_corr/rollout_is_max:44.50603103637695 - rollout_corr/rollout_is_min:0.0011786288814619184 - rollout_corr/rollout_is_std:0.030854584649205208 - rollout_corr/rollout_is_eff_sample_size:0.9990490190485594 - rollout_corr/rollout_is_seq_mean:1.0000252723693848 - rollout_corr/rollout_is_seq_std:0.004327365662902594 - rollout_corr/rollout_is_seq_max:1.0554637908935547 - rollout_corr/rollout_is_seq_min:0.9875951409339905 - rollout_corr/rollout_is_seq_max_deviation:0.05546379089355469 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0925669269636273) - rollout_corr/training_log_ppl:np.float64(0.08702500175058958) - rollout_corr/kl:np.float64(0.0007599289466115877) - rollout_corr/k3_kl:np.float64(0.000812841809917586) - rollout_corr/rollout_ppl:np.float64(1.0917559866793454) - rollout_corr/rollout_log_ppl:np.float64(0.08626507179360487) - rollout_corr/log_ppl_diff:np.float64(0.0007599299569847062) - rollout_corr/log_ppl_abs_diff:np.float64(0.001818598095269408) - rollout_corr/log_ppl_diff_max:np.float64(0.024008244276046753) - rollout_corr/log_ppl_diff_min:np.float64(-0.014017723500728607) - rollout_corr/ppl_ratio:np.float64(1.0007652402855456) - rollout_corr/chi2_token:np.float64(0.007633969187736511) - rollout_corr/chi2_seq:np.float64(13.315856878180057) - actor/pg_loss:np.float64(0.0017298469319939613) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006172826681449806) - actor/ppo_kl:np.float64(-5.590297072188832e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.92578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.92578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6953125) - self_distillation/token_reweight_w_mean:np.float64(123974.65548743168) - self_distillation/token_reweight_w_std:np.float64(2170329.61304679) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005807334091514) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04290506925462978) - self_distillation/empty_target_batch:np.float64(0.07421875) - actor/grad_norm:np.float64(0.27438871562480927) - perf/mfu/actor:np.float64(0.03876809494412483) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.63893127441406) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:2 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.024676039814949036 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.024676039814949036 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:419.9140625 - response_length/max:2024.0 - response_length/min:40.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:419.9140625 - response_length_non_aborted/max:2024.0 - response_length_non_aborted/min:40.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:253.28125 - prompt_length/max:355.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012747198343276978 - timing_s/agent_loop/generate_sequences/min:np.float64(0.48513596691191196) - timing_s/agent_loop/generate_sequences/max:np.float64(13.075700936838984) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.0640050550719025) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.075700936838984) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:2024 - timing_s/gen:18.986181549727917 - timing_s/reward:0.06117998994886875 - timing_s/old_log_prob:2.4063704144209623 - timing_s/adv:0.5998788680881262 - timing_s/update_actor:15.829487767070532 - timing_s/step:37.97097833454609 - timing_s/stop_profile:0.00010865926742553711 - timing_per_token_ms/update_actor:0.09185140692749441 - timing_per_token_ms/adv:0.003480827606727049 - timing_per_token_ms/gen:0.17661892825659936 - perf/total_num_tokens:172338 - perf/time_per_step:37.97097833454609 - perf/throughput:567.3346051344905 (TaskRunner pid=2074004) Training Progress: 56%|█████▌ | 56/100 [48:00<32:51, 44.80s/it] (TaskRunner pid=2074004) step:57 - global_seqlen/min:19855 - global_seqlen/max:29475 - global_seqlen/minmax_diff:9620 - global_seqlen/balanced_min:23604 - global_seqlen/balanced_max:23634 - global_seqlen/mean:23625.125 - actor/entropy:0.09015602618455887 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.456105500459671 - training/rollout_probs_diff_mean:0.0024542445316910744 - training/rollout_probs_diff_std:0.011214017868041992 - training/rollout_actor_probs_pearson_corr:0.9974842667579651 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.68359375 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.68359375 - rollout_corr/rollout_is_mean:1.0000454187393188 - rollout_corr/rollout_is_ratio_fraction_high:5.9702000726247206e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010234629007754847 - rollout_corr/rollout_is_max:2.9357409477233887 - rollout_corr/rollout_is_min:0.22341881692409515 - rollout_corr/rollout_is_std:0.030461862683296204 - rollout_corr/rollout_is_eff_sample_size:0.9990729351369017 - rollout_corr/rollout_is_seq_mean:1.0001904964447021 - rollout_corr/rollout_is_seq_std:0.002443463308736682 - rollout_corr/rollout_is_seq_max:1.0193251371383667 - rollout_corr/rollout_is_seq_min:0.9929532408714294 - rollout_corr/rollout_is_seq_max_deviation:0.0193251371383667 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0897927153855562) - rollout_corr/training_log_ppl:np.float64(0.0843119082164776) - rollout_corr/kl:np.float64(0.00043658774763111197) - rollout_corr/k3_kl:np.float64(0.0005583141538294534) - rollout_corr/rollout_ppl:np.float64(1.0893031116575003) - rollout_corr/rollout_log_ppl:np.float64(0.0838753203752276) - rollout_corr/log_ppl_diff:np.float64(0.0004365878412500024) - rollout_corr/log_ppl_abs_diff:np.float64(0.001471972107538022) - rollout_corr/log_ppl_diff_max:np.float64(0.00870583951473236) - rollout_corr/log_ppl_diff_min:np.float64(-0.012766733765602112) - rollout_corr/ppl_ratio:np.float64(1.000438829883933) - rollout_corr/chi2_token:np.float64(0.0013955945614725351) - rollout_corr/chi2_seq:np.float64(1.2546640543732792) - actor/pg_loss:np.float64(0.0011045325081795454) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00035431018341114395) - actor/ppo_kl:np.float64(0.00010037352073410943) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.68359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(72716.30810955423) - self_distillation/token_reweight_w_std:np.float64(1347980.3629608657) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001339728478342) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02943878156656865) - self_distillation/empty_target_batch:np.float64(0.31640625) - actor/grad_norm:np.float64(0.24138232693076134) - perf/mfu/actor:np.float64(0.0430505553899597) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.70356369018555) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:2 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008093886077404022 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008093886077404022 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:458.00390625 - response_length/max:1631.0 - response_length/min:45.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:458.00390625 - response_length_non_aborted/max:1631.0 - response_length_non_aborted/min:45.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.28125 - prompt_length/max:382.0 - prompt_length/min:209.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013142824172973633 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5333198085427284) - timing_s/agent_loop/generate_sequences/max:np.float64(11.510037498548627) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.700965259689838) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.510037498548627) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:236 - timing_s/agent_loop/slowest/response_length:1631 - timing_s/gen:17.477134620770812 - timing_s/reward:0.0652174036949873 - timing_s/old_log_prob:2.3839294649660587 - timing_s/adv:0.5826396737247705 - timing_s/update_actor:15.731351995840669 - timing_s/step:36.32835609279573 - timing_s/stop_profile:0.00012176670134067535 - timing_per_token_ms/update_actor:0.08323422625192813 - timing_per_token_ms/adv:0.003082733285669232 - timing_per_token_ms/gen:0.1490599887484824 - perf/total_num_tokens:189001 - perf/time_per_step:36.32835609279573 - perf/throughput:650.3218846361477 (TaskRunner pid=2074004) Training Progress: 57%|█████▋ | 57/100 [48:36<30:17, 42.27s/it] (TaskRunner pid=2074004) step:58 - global_seqlen/min:14549 - global_seqlen/max:27391 - global_seqlen/minmax_diff:12842 - global_seqlen/balanced_min:22974 - global_seqlen/balanced_max:22989 - global_seqlen/mean:22981.5 - actor/entropy:0.10685306042432785 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6651362180709839 - training/rollout_probs_diff_mean:0.0026612752117216587 - training/rollout_probs_diff_std:0.011326903477311134 - training/rollout_actor_probs_pearson_corr:0.9977207779884338 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9998153448104858 - rollout_corr/rollout_is_ratio_fraction_high:8.753195288591087e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.00012254473404027522 - rollout_corr/rollout_is_max:2.095289468765259 - rollout_corr/rollout_is_min:0.04907945916056633 - rollout_corr/rollout_is_std:0.030399184674024582 - rollout_corr/rollout_is_eff_sample_size:0.9990763858113328 - rollout_corr/rollout_is_seq_mean:0.9997419118881226 - rollout_corr/rollout_is_seq_std:0.0024453294463455677 - rollout_corr/rollout_is_seq_max:1.0176780223846436 - rollout_corr/rollout_is_seq_min:0.9827814102172852 - rollout_corr/rollout_is_seq_max_deviation:0.017678022384643555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1037812139838934) - rollout_corr/training_log_ppl:np.float64(0.09657903140760027) - rollout_corr/kl:np.float64(0.0008447893188705535) - rollout_corr/k3_kl:np.float64(0.0005789172635033424) - rollout_corr/rollout_ppl:np.float64(1.1028364589437842) - rollout_corr/rollout_log_ppl:np.float64(0.09573424071641057) - rollout_corr/log_ppl_diff:np.float64(0.0008447906911897007) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017328100257145707) - rollout_corr/log_ppl_diff_max:np.float64(0.02989429235458374) - rollout_corr/log_ppl_diff_min:np.float64(-0.01144128292798996) - rollout_corr/ppl_ratio:np.float64(1.0008499294053763) - rollout_corr/chi2_token:np.float64(0.0006312790792435408) - rollout_corr/chi2_seq:np.float64(0.1915365003515035) - actor/pg_loss:np.float64(0.001689612865447998) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005289059029109922) - actor/ppo_kl:np.float64(3.6509948291296723e-06) - actor/pg_clipfrac_lower:np.float64(5.7276392908534035e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(124004.99291193858) - self_distillation/token_reweight_w_std:np.float64(2293706.94823437) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002958420664072) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03820525103947148) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.2612778563052416) - perf/mfu/actor:np.float64(0.04187616741345731) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.75995635986328) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:2 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.020541559904813766 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.020541559904813766 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:446.265625 - response_length/max:1848.0 - response_length/min:58.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:446.265625 - response_length_non_aborted/max:1848.0 - response_length_non_aborted/min:58.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.90625 - prompt_length/max:341.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.460374712944031e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.705725435167551) - timing_s/agent_loop/generate_sequences/max:np.float64(11.959222553297877) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.448366878685192) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.959222553297877) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:302 - timing_s/agent_loop/slowest/response_length:1848 - timing_s/gen:17.897763485088944 - timing_s/reward:0.06247272342443466 - timing_s/old_log_prob:2.423390854150057 - timing_s/adv:0.5865025147795677 - timing_s/update_actor:16.00029367208481 - timing_s/step:37.05911082029343 - timing_s/stop_profile:0.00011603906750679016 - timing_per_token_ms/update_actor:0.08702811866112312 - timing_per_token_ms/adv:0.0031900796008722653 - timing_per_token_ms/gen:0.1566626123480353 - perf/total_num_tokens:183852 - perf/time_per_step:37.05911082029343 - perf/throughput:620.130906848834 (TaskRunner pid=2074004) Training Progress: 58%|█████▊ | 58/100 [49:13<28:30, 40.72s/it] (TaskRunner pid=2074004) step:59 - global_seqlen/min:16270 - global_seqlen/max:25747 - global_seqlen/minmax_diff:9477 - global_seqlen/balanced_min:21707 - global_seqlen/balanced_max:21791 - global_seqlen/mean:21721.5 - actor/entropy:0.10231564939022064 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.46255624294281006 - training/rollout_probs_diff_mean:0.0026242148596793413 - training/rollout_probs_diff_std:0.011124146170914173 - training/rollout_actor_probs_pearson_corr:0.9976997971534729 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.67578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.0001174211502075 - rollout_corr/rollout_is_ratio_fraction_high:3.8038724596844986e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.8038724596844986e-05 - rollout_corr/rollout_is_max:2.163364887237549 - rollout_corr/rollout_is_min:2.3478437469748314e-06 - rollout_corr/rollout_is_std:0.03120991960167885 - rollout_corr/rollout_is_eff_sample_size:0.9990270077403499 - rollout_corr/rollout_is_seq_mean:1.000049114227295 - rollout_corr/rollout_is_seq_std:0.002008486306294799 - rollout_corr/rollout_is_seq_max:1.0093592405319214 - rollout_corr/rollout_is_seq_min:0.9932237863540649 - rollout_corr/rollout_is_seq_max_deviation:0.009359240531921387 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1050791395828128) - rollout_corr/training_log_ppl:np.float64(0.09638163917770726) - rollout_corr/kl:np.float64(0.0006338543649420103) - rollout_corr/k3_kl:np.float64(0.0009249671898068357) - rollout_corr/rollout_ppl:np.float64(1.1043821601197124) - rollout_corr/rollout_log_ppl:np.float64(0.09574778548085305) - rollout_corr/log_ppl_diff:np.float64(0.000633853696854203) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018177191577706253) - rollout_corr/log_ppl_diff_max:np.float64(0.02475271373987198) - rollout_corr/log_ppl_diff_min:np.float64(-0.016850709915161133) - rollout_corr/ppl_ratio:np.float64(1.0006383063737303) - rollout_corr/chi2_token:np.float64(0.002892733784392476) - rollout_corr/chi2_seq:np.float64(1.0235684728249907) - actor/pg_loss:np.float64(0.0013639027019962668) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011857084368784854) - actor/ppo_kl:np.float64(5.230466129590994e-05) - actor/pg_clipfrac_lower:np.float64(1.7208149074576795e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_w_mean:np.float64(187076.23620933085) - self_distillation/token_reweight_w_std:np.float64(2489852.4811536297) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.00058325775899) - self_distillation/token_reweight_w_clip_frac:np.float64(0.037732115903054364) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.295432485640049) - perf/mfu/actor:np.float64(0.03939332486847546) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.67478942871094) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:2 - critic/score/mean:0.67578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.67578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005545332562178373 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005545332562178373 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:410.765625 - response_length/max:1910.0 - response_length/min:68.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:410.765625 - response_length_non_aborted/max:1910.0 - response_length_non_aborted/min:68.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.03125 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.881092071533203e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5705593023449183) - timing_s/agent_loop/generate_sequences/max:np.float64(11.528048625215888) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9566591325128684) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.528048625215888) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:1910 - timing_s/gen:17.374146005138755 - timing_s/reward:0.0605713315308094 - timing_s/old_log_prob:2.3631235398352146 - timing_s/adv:0.5710447505116463 - timing_s/update_actor:15.808569202199578 - timing_s/step:36.26629160158336 - timing_s/stop_profile:0.00011047534644603729 - timing_per_token_ms/update_actor:0.0909730520578665 - timing_per_token_ms/adv:0.0032861724012593872 - timing_per_token_ms/gen:0.16522258363896264 - perf/total_num_tokens:173772 - perf/time_per_step:36.26629160158336 - perf/throughput:598.9446133238407 (TaskRunner pid=2074004) Training Progress: 59%|█████▉ | 59/100 [49:49<26:55, 39.40s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Solving for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substitute the known values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest answer is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50/actor (TaskRunner pid=2074004) step:60 - global_seqlen/min:14585 - global_seqlen/max:28896 - global_seqlen/minmax_diff:14311 - global_seqlen/balanced_min:22508 - global_seqlen/balanced_max:22534 - global_seqlen/mean:22528.125 - actor/entropy:0.10657462477684021 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5907468795776367 - training/rollout_probs_diff_mean:0.0025613645557314157 - training/rollout_probs_diff_std:0.011115039698779583 - training/rollout_actor_probs_pearson_corr:0.9978196620941162 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.66015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.0001739263534546 - rollout_corr/rollout_is_ratio_fraction_high:3.601494609029032e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.904109901981428e-05 - rollout_corr/rollout_is_max:3.41389799118042 - rollout_corr/rollout_is_min:0.04764045029878616 - rollout_corr/rollout_is_std:0.030336376279592514 - rollout_corr/rollout_is_eff_sample_size:0.9990810264110818 - rollout_corr/rollout_is_seq_mean:1.0001338720321655 - rollout_corr/rollout_is_seq_std:0.0019507677061483264 - rollout_corr/rollout_is_seq_max:1.0104360580444336 - rollout_corr/rollout_is_seq_min:0.9874913692474365 - rollout_corr/rollout_is_seq_max_deviation:0.012508630752563477 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.101980961393565) - rollout_corr/training_log_ppl:np.float64(0.09394121470904793) - rollout_corr/kl:np.float64(0.0005248432001501158) - rollout_corr/k3_kl:np.float64(0.0005477877917310359) - rollout_corr/rollout_ppl:np.float64(1.1014048550277948) - rollout_corr/rollout_log_ppl:np.float64(0.09341637074976461) - rollout_corr/log_ppl_diff:np.float64(0.0005248439592833165) - rollout_corr/log_ppl_abs_diff:np.float64(0.001438066206901567) - rollout_corr/log_ppl_diff_max:np.float64(0.013666749000549316) - rollout_corr/log_ppl_diff_min:np.float64(-0.009086042642593384) - rollout_corr/ppl_ratio:np.float64(1.0005271381232888) - rollout_corr/chi2_token:np.float64(0.0011212392710149288) - rollout_corr/chi2_seq:np.float64(0.3878027028404176) - actor/pg_loss:np.float64(0.0020158932311460376) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007685895379836438) - actor/ppo_kl:np.float64(0.0001438019309926153) - actor/pg_clipfrac_lower:np.float64(8.778089977568015e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66015625) - self_distillation/token_reweight_w_mean:np.float64(88144.51772208884) - self_distillation/token_reweight_w_std:np.float64(1184668.7561488296) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000381489051506) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04390312758914661) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.3155992925167084) - perf/mfu/actor:np.float64(0.04083755030532129) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.75739669799805) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.62265625) - val-aux/sciknoweval/reward/std@16:np.float64(0.1723676683272905) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.69395) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.14379069680642126) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5512875) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1475520499816467) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6222125000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1724222276112314) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7506250000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.10453211597440552) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4925750000000001) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11041911356297991) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6338750000000001) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13849568719279187) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7915000000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0681230254543447) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4484499999999999) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07370449353224175) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6356125) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10212696045932126) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.817) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03823086848173044) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.41914999999999997) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04379924098110083) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6377375000000001) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.0775045006079097) - val-aux/sciknoweval/score/mean@16:np.float64(0.62265625) - val-aux/sciknoweval/score/std@16:np.float64(0.1723676683272905) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.69395) - val-aux/sciknoweval/score/best@2/std:np.float64(0.14379069680642126) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5512875) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1475520499816467) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6222125000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1724222276112314) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7506250000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.10453211597440552) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4925750000000001) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11041911356297991) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6338750000000001) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13849568719279187) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7915000000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0681230254543447) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4484499999999999) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07370449353224175) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6356125) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10212696045932126) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.817) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03823086848173044) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.41914999999999997) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04379924098110083) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6377375000000001) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.0775045006079097) - val-core/sciknoweval/acc/mean@16:np.float64(0.62265625) - val-aux/sciknoweval/acc/std@16:np.float64(0.1723676683272905) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.69395) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.14379069680642126) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5512875) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1475520499816467) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6222125000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1724222276112314) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7506250000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.10453211597440552) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4925750000000001) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11041911356297991) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6338750000000001) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13849568719279187) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7915000000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0681230254543447) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4484499999999999) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07370449353224175) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6356125) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10212696045932126) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.817) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03823086848173044) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.41914999999999997) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04379924098110083) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6377375000000001) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.0775045006079097) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99765625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.007159754662762969) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9997875) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.002242736757610738) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9955625000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.009417988829529579) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9975125) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.007412161208789384) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.99175) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.01156057508733171) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99915) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.004201813298434218) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9865625000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.012175391485948633) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.999775) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0021268964117280294) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9809249999999998) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.010081889419878812) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999625) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.000683625445693766) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:2 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0012774050701409578 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0012774050701409578 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:433.84765625 - response_length/max:1465.0 - response_length/min:89.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:433.84765625 - response_length_non_aborted/max:1465.0 - response_length_non_aborted/min:89.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.15625 - prompt_length/max:340.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.011104702949524e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0925841629505157) - timing_s/agent_loop/generate_sequences/max:np.float64(10.3709436673671) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.332557140849531) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.3709436673671) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:1465 - timing_s/gen:16.32388431020081 - timing_s/reward:0.06206193566322327 - timing_s/old_log_prob:2.4646734315901995 - timing_s/adv:0.5586496517062187 - timing_s/update_actor:15.928700476884842 - timing_s/step:35.42581527121365 - timing_s/testing:84.386498471722 - timing_s/save_checkpoint:6.650143900886178 - timing_s/stop_profile:0.00012451224029064178 - timing_per_token_ms/update_actor:0.08838230254895182 - timing_per_token_ms/adv:0.0030997345080106464 - timing_per_token_ms/gen:0.14697595381263953 - perf/total_num_tokens:180225 - perf/time_per_step:35.42581527121365 - perf/throughput:635.9239675228005 (TaskRunner pid=2074004) Training Progress: 60%|██████ | 60/100 [51:56<43:41, 65.54s/it] (TaskRunner pid=2074004) step:61 - global_seqlen/min:15258 - global_seqlen/max:27550 - global_seqlen/minmax_diff:12292 - global_seqlen/balanced_min:21784 - global_seqlen/balanced_max:21817 - global_seqlen/mean:21809.5 - actor/entropy:0.11104559153318405 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.41980162262916565 - training/rollout_probs_diff_mean:0.0026704224292188883 - training/rollout_probs_diff_std:0.01082498300820589 - training/rollout_actor_probs_pearson_corr:0.9980793595314026 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.7265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:0.9999408721923828 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.5849687012378126e-05 - rollout_corr/rollout_is_max:1.967535376548767 - rollout_corr/rollout_is_min:0.2217474728822708 - rollout_corr/rollout_is_std:0.02934972383081913 - rollout_corr/rollout_is_eff_sample_size:0.9991390970476317 - rollout_corr/rollout_is_seq_mean:1.0000622272491455 - rollout_corr/rollout_is_seq_std:0.002196152927353978 - rollout_corr/rollout_is_seq_max:1.0167111158370972 - rollout_corr/rollout_is_seq_min:0.992173433303833 - rollout_corr/rollout_is_seq_max_deviation:0.016711115837097168 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1188437603414059) - rollout_corr/training_log_ppl:np.float64(0.10840739371997188) - rollout_corr/kl:np.float64(0.0004275032880443064) - rollout_corr/k3_kl:np.float64(0.0005405173702293098) - rollout_corr/rollout_ppl:np.float64(1.1183525188826025) - rollout_corr/rollout_log_ppl:np.float64(0.10797988958074711) - rollout_corr/log_ppl_diff:np.float64(0.0004275041392247658) - rollout_corr/log_ppl_abs_diff:np.float64(0.001455118432204472) - rollout_corr/log_ppl_diff_max:np.float64(0.01237882673740387) - rollout_corr/log_ppl_diff_min:np.float64(-0.009973615407943726) - rollout_corr/ppl_ratio:np.float64(1.0004298684652895) - rollout_corr/chi2_token:np.float64(0.001337600639089942) - rollout_corr/chi2_seq:np.float64(0.26282138866372406) - actor/pg_loss:np.float64(0.002470805891789496) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000613119240369997) - actor/ppo_kl:np.float64(-8.526104423367542e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_w_mean:np.float64(115092.67977330904) - self_distillation/token_reweight_w_std:np.float64(2121252.4452306055) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004522716626525) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04705037997337058) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.32100969552993774) - perf/mfu/actor:np.float64(0.03935632243399488) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.00535583496094) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:2 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0035281332675367594 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0035281332675367594 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:425.984375 - response_length/max:1622.0 - response_length/min:47.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:425.984375 - response_length_non_aborted/max:1622.0 - response_length_non_aborted/min:47.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.5625 - prompt_length/max:328.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013445131480693817 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5521563682705164) - timing_s/agent_loop/generate_sequences/max:np.float64(10.833473060280085) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.15038654532691) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.833473060280085) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:316 - timing_s/agent_loop/slowest/response_length:1622 - timing_s/gen:16.92672893963754 - timing_s/reward:0.05643085576593876 - timing_s/old_log_prob:2.6533840224146843 - timing_s/adv:0.6161332204937935 - timing_s/update_actor:15.688221232965589 - timing_s/step:36.045780032873154 - timing_s/stop_profile:0.00011107698082923889 - timing_per_token_ms/update_actor:0.08991621330707712 - timing_per_token_ms/adv:0.003531335086165395 - timing_per_token_ms/gen:0.15521704269190426 - perf/total_num_tokens:174476 - perf/time_per_step:36.045780032873154 - perf/throughput:605.0500219473707 (TaskRunner pid=2074004) Training Progress: 61%|██████ | 61/100 [52:32<36:51, 56.71s/it] (TaskRunner pid=2074004) step:62 - global_seqlen/min:20453 - global_seqlen/max:31926 - global_seqlen/minmax_diff:11473 - global_seqlen/balanced_min:24980 - global_seqlen/balanced_max:25000 - global_seqlen/mean:24994.125 - actor/entropy:0.11476144939661026 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8106536269187927 - training/rollout_probs_diff_mean:0.0027153317350894213 - training/rollout_probs_diff_std:0.011159801855683327 - training/rollout_actor_probs_pearson_corr:0.9979556202888489 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:0.9999613165855408 - rollout_corr/rollout_is_ratio_fraction_high:7.923365046735853e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.338692037388682e-05 - rollout_corr/rollout_is_max:2.658514976501465 - rollout_corr/rollout_is_min:0.1538795679807663 - rollout_corr/rollout_is_std:0.030247831717133522 - rollout_corr/rollout_is_eff_sample_size:0.9990857860453506 - rollout_corr/rollout_is_seq_mean:0.9999266862869263 - rollout_corr/rollout_is_seq_std:0.0018450476927682757 - rollout_corr/rollout_is_seq_max:1.011099934577942 - rollout_corr/rollout_is_seq_min:0.9912953972816467 - rollout_corr/rollout_is_seq_max_deviation:0.011099934577941895 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1163019719533622) - rollout_corr/training_log_ppl:np.float64(0.10734543754006154) - rollout_corr/kl:np.float64(0.0005882964252315048) - rollout_corr/k3_kl:np.float64(0.0005425430525072272) - rollout_corr/rollout_ppl:np.float64(1.1156547986902297) - rollout_corr/rollout_log_ppl:np.float64(0.1067571406965726) - rollout_corr/log_ppl_diff:np.float64(0.0005882968434889335) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013848803682776634) - rollout_corr/log_ppl_diff_max:np.float64(0.010647937655448914) - rollout_corr/log_ppl_diff_min:np.float64(-0.006409019231796265) - rollout_corr/ppl_ratio:np.float64(1.0005901597905904) - rollout_corr/chi2_token:np.float64(0.0009822722058743238) - rollout_corr/chi2_seq:np.float64(0.6805941013153642) - actor/pg_loss:np.float64(0.0017424660036340356) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005262740519356157) - actor/ppo_kl:np.float64(4.013105478151147e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(72289.93594874162) - self_distillation/token_reweight_w_std:np.float64(1211116.1148807127) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999488468747586) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04404561784758698) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.22749612480401993) - perf/mfu/actor:np.float64(0.04494797983979118) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.95013809204102) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:2 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005711755715310574 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005711755715310574 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:493.00390625 - response_length/max:1745.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:493.00390625 - response_length_non_aborted/max:1745.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:288.0625 - prompt_length/max:460.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.554384112358093e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6613222770392895) - timing_s/agent_loop/generate_sequences/max:np.float64(12.032618755474687) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.850435824046144) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.032618755474687) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:346 - timing_s/agent_loop/slowest/response_length:1745 - timing_s/gen:18.026200192049146 - timing_s/reward:0.06495929323136806 - timing_s/old_log_prob:2.460443489253521 - timing_s/adv:0.651430806145072 - timing_s/update_actor:16.151509700343013 - timing_s/step:37.442482167854905 - timing_s/stop_profile:0.00011717900633811951 - timing_per_token_ms/update_actor:0.0807765309864969 - timing_per_token_ms/adv:0.003257919641841193 - timing_per_token_ms/gen:0.14282816750032998 - perf/total_num_tokens:199953 - perf/time_per_step:37.442482167854905 - perf/throughput:667.533869361309 (TaskRunner pid=2074004) Training Progress: 62%|██████▏ | 62/100 [53:09<32:15, 50.94s/it] (TaskRunner pid=2074004) step:63 - global_seqlen/min:16031 - global_seqlen/max:28652 - global_seqlen/minmax_diff:12621 - global_seqlen/balanced_min:20376 - global_seqlen/balanced_max:20475 - global_seqlen/mean:20399.625 - actor/entropy:0.09949057549238205 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9106009602546692 - training/rollout_probs_diff_mean:0.002759087597951293 - training/rollout_probs_diff_std:0.011734765022993088 - training/rollout_actor_probs_pearson_corr:0.9973864555358887 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.6953125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.000015139579773 - rollout_corr/rollout_is_ratio_fraction_high:5.268204404274002e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.2145635234192014e-05 - rollout_corr/rollout_is_max:2.28180193901062 - rollout_corr/rollout_is_min:0.06190122291445732 - rollout_corr/rollout_is_std:0.03141549974679947 - rollout_corr/rollout_is_eff_sample_size:0.9990140393695902 - rollout_corr/rollout_is_seq_mean:0.9999146461486816 - rollout_corr/rollout_is_seq_std:0.001660522655583918 - rollout_corr/rollout_is_seq_max:1.0048866271972656 - rollout_corr/rollout_is_seq_min:0.9946311712265015 - rollout_corr/rollout_is_seq_max_deviation:0.005368828773498535 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0948191625066102) - rollout_corr/training_log_ppl:np.float64(0.08782734138367232) - rollout_corr/kl:np.float64(0.000854949058012977) - rollout_corr/k3_kl:np.float64(0.0006357853782912315) - rollout_corr/rollout_ppl:np.float64(1.0938773178495467) - rollout_corr/rollout_log_ppl:np.float64(0.08697239161483594) - rollout_corr/log_ppl_diff:np.float64(0.0008549497688363772) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017329079382761847) - rollout_corr/log_ppl_diff_max:np.float64(0.008672401309013367) - rollout_corr/log_ppl_diff_min:np.float64(-0.0064188651740550995) - rollout_corr/ppl_ratio:np.float64(1.0008579187560827) - rollout_corr/chi2_token:np.float64(0.0008239620365202427) - rollout_corr/chi2_seq:np.float64(0.4398700457531959) - actor/pg_loss:np.float64(0.00171933404635638) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005747673458245117) - actor/ppo_kl:np.float64(0.00029454395632910746) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6953125) - self_distillation/token_reweight_w_mean:np.float64(141111.829759978) - self_distillation/token_reweight_w_std:np.float64(1970238.3386651743) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000252681784332) - self_distillation/token_reweight_w_clip_frac:np.float64(0.031212968737236224) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.273105651140213) - perf/mfu/actor:np.float64(0.037443201028578815) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.92625427246094) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:2 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005000842735171318 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005000842735171318 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:370.73828125 - response_length/max:1427.0 - response_length/min:73.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:370.73828125 - response_length_non_aborted/max:1427.0 - response_length_non_aborted/min:73.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.75 - prompt_length/max:350.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010591745376586914 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8246997967362404) - timing_s/agent_loop/generate_sequences/max:np.float64(9.28955096192658) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.776446552736161) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.28955096192658) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:350 - timing_s/agent_loop/slowest/response_length:1427 - timing_s/gen:15.555157018825412 - timing_s/reward:0.05943015590310097 - timing_s/old_log_prob:2.3526105862110853 - timing_s/adv:0.5264585241675377 - timing_s/update_actor:15.563456859439611 - timing_s/step:34.14370860904455 - timing_s/stop_profile:0.00012073293328285217 - timing_per_token_ms/update_actor:0.09536607204445922 - timing_per_token_ms/adv:0.0032259080998274336 - timing_per_token_ms/gen:0.1638954895618478 - perf/total_num_tokens:163197 - perf/time_per_step:34.14370860904455 - perf/throughput:597.4636567334168 (TaskRunner pid=2074004) Training Progress: 63%|██████▎ | 63/100 [53:44<28:18, 45.92s/it] (TaskRunner pid=2074004) step:64 - global_seqlen/min:17389 - global_seqlen/max:31739 - global_seqlen/minmax_diff:14350 - global_seqlen/balanced_min:23753 - global_seqlen/balanced_max:30196 - global_seqlen/mean:24563.75 - actor/entropy:0.11078858375549316 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27813637256622314 - training/rollout_probs_diff_mean:0.0025074740406125784 - training/rollout_probs_diff_std:0.010136975906789303 - training/rollout_actor_probs_pearson_corr:0.9982666969299316 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9998918771743774 - rollout_corr/rollout_is_ratio_fraction_high:7.794596967869438e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.897298302035779e-05 - rollout_corr/rollout_is_max:2.2562098503112793 - rollout_corr/rollout_is_min:0.38679587841033936 - rollout_corr/rollout_is_std:0.02878168411552906 - rollout_corr/rollout_is_eff_sample_size:0.9991719432652497 - rollout_corr/rollout_is_seq_mean:0.9999525547027588 - rollout_corr/rollout_is_seq_std:0.0016079578781500459 - rollout_corr/rollout_is_seq_max:1.0066720247268677 - rollout_corr/rollout_is_seq_min:0.9929185509681702 - rollout_corr/rollout_is_seq_max_deviation:0.007081449031829834 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1173579916357994) - rollout_corr/training_log_ppl:np.float64(0.10709392841090448) - rollout_corr/kl:np.float64(0.0006130551650116445) - rollout_corr/k3_kl:np.float64(0.0006270227015860996) - rollout_corr/rollout_ppl:np.float64(1.1166537292301655) - rollout_corr/rollout_log_ppl:np.float64(0.1064808721675945) - rollout_corr/log_ppl_diff:np.float64(0.000613056243309984) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015000321327534039) - rollout_corr/log_ppl_diff_max:np.float64(0.00872485339641571) - rollout_corr/log_ppl_diff_min:np.float64(-0.005040250718593597) - rollout_corr/ppl_ratio:np.float64(1.0006150556728244) - rollout_corr/chi2_token:np.float64(0.0012433729134500027) - rollout_corr/chi2_seq:np.float64(1.6464725509285927) - actor/pg_loss:np.float64(0.0018512200331315398) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00047728462982377096) - actor/ppo_kl:np.float64(0.00010965376971583396) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(103190.21550233453) - self_distillation/token_reweight_w_std:np.float64(1658807.6000547835) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004303914029151) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04449158857460134) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.2668180838227272) - perf/mfu/actor:np.float64(0.0422181008652012) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.02878189086914) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:2 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009036860428750515 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009036860428750515 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:501.1484375 - response_length/max:8192.0 - response_length/min:107.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:501.1484375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:349.0 - prompt_length/min:192.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013284198939800262 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1968379076570272) - timing_s/agent_loop/generate_sequences/max:np.float64(51.116004686802626) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.855164358850743) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(51.116004686802626) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:307 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:57.01408413052559 - timing_s/reward:0.06438678875565529 - timing_s/old_log_prob:2.5394509602338076 - timing_s/adv:0.595608577132225 - timing_s/update_actor:16.805029960349202 - timing_s/step:77.1059439778328 - timing_s/stop_profile:0.00012509338557720184 - timing_per_token_ms/update_actor:0.08551742893669127 - timing_per_token_ms/adv:0.0030309326605883927 - timing_per_token_ms/gen:0.4444017968924937 - perf/total_num_tokens:196510 - perf/time_per_step:77.1059439778328 - perf/throughput:318.57141917699414 (TaskRunner pid=2074004) Training Progress: 64%|██████▍ | 64/100 [55:01<33:10, 55.29s/it] (TaskRunner pid=2074004) step:65 - global_seqlen/min:12881 - global_seqlen/max:28547 - global_seqlen/minmax_diff:15666 - global_seqlen/balanced_min:20761 - global_seqlen/balanced_max:20766 - global_seqlen/mean:20763.75 - actor/entropy:0.10208147764205933 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43991219997406006 - training/rollout_probs_diff_mean:0.00263621611520648 - training/rollout_probs_diff_std:0.01091404352337122 - training/rollout_actor_probs_pearson_corr:0.9977887272834778 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.93359375 - self_distillation/correct_sample_fraction:0.83984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.93359375 - rollout_corr/rollout_is_mean:0.9999848008155823 - rollout_corr/rollout_is_ratio_fraction_high:1.0132328498002607e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.052931399201043e-05 - rollout_corr/rollout_is_max:2.1628031730651855 - rollout_corr/rollout_is_min:0.17808698117733002 - rollout_corr/rollout_is_std:0.030277375131845474 - rollout_corr/rollout_is_eff_sample_size:0.9990840011771853 - rollout_corr/rollout_is_seq_mean:0.9999912977218628 - rollout_corr/rollout_is_seq_std:0.0019186957506462932 - rollout_corr/rollout_is_seq_max:1.0081483125686646 - rollout_corr/rollout_is_seq_min:0.9939520359039307 - rollout_corr/rollout_is_seq_max_deviation:0.00814831256866455 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0915365894325078) - rollout_corr/training_log_ppl:np.float64(0.08563249993312638) - rollout_corr/kl:np.float64(0.0005638776894053565) - rollout_corr/k3_kl:np.float64(0.0005349675133992093) - rollout_corr/rollout_ppl:np.float64(1.0908976537175477) - rollout_corr/rollout_log_ppl:np.float64(0.08506862222202471) - rollout_corr/log_ppl_diff:np.float64(0.000563877711101668) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015504594302910846) - rollout_corr/log_ppl_diff_max:np.float64(0.007803872227668762) - rollout_corr/log_ppl_diff_min:np.float64(-0.00601176917552948) - rollout_corr/ppl_ratio:np.float64(1.0005660941824317) - rollout_corr/chi2_token:np.float64(0.0010084726382046938) - rollout_corr/chi2_seq:np.float64(0.3784930878318846) - actor/pg_loss:np.float64(0.0010833193082362413) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00028873655492134276) - actor/ppo_kl:np.float64(7.600492999415565e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.93359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.93359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_w_mean:np.float64(153738.86696914816) - self_distillation/token_reweight_w_std:np.float64(1896279.4650243353) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996677823364735) - self_distillation/token_reweight_w_clip_frac:np.float64(0.026835397191462107) - self_distillation/empty_target_batch:np.float64(0.06640625) - actor/grad_norm:np.float64(0.22542214393615723) - perf/mfu/actor:np.float64(0.038259118996520076) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.00650787353516) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:2 - critic/score/mean:0.83984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.83984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0010626278817653656 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0010626278817653656 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:385.5234375 - response_length/max:1258.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:385.5234375 - response_length_non_aborted/max:1258.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.34375 - prompt_length/max:364.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013937614858150482 - timing_s/agent_loop/generate_sequences/min:np.float64(1.065683364868164) - timing_s/agent_loop/generate_sequences/max:np.float64(8.783862510696054) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.777955053730693) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.783862510696054) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:1258 - timing_s/gen:14.71335643529892 - timing_s/reward:0.06031355261802673 - timing_s/old_log_prob:2.3282906115055084 - timing_s/adv:0.5691004022955894 - timing_s/update_actor:15.625853784382343 - timing_s/step:33.38593070022762 - timing_s/stop_profile:0.00011672265827655792 - timing_per_token_ms/update_actor:0.09406931421577475 - timing_per_token_ms/adv:0.003426045405427665 - timing_per_token_ms/gen:0.1490805564198322 - perf/total_num_tokens:166110 - perf/time_per_step:33.38593070022762 - perf/throughput:621.9311417865741 (TaskRunner pid=2074004) Training Progress: 65%|██████▌ | 65/100 [55:34<28:25, 48.73s/it] (TaskRunner pid=2074004) step:66 - global_seqlen/min:18568 - global_seqlen/max:30571 - global_seqlen/minmax_diff:12003 - global_seqlen/balanced_min:23556 - global_seqlen/balanced_max:23573 - global_seqlen/mean:23566.625 - actor/entropy:0.10450827330350876 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3797508478164673 - training/rollout_probs_diff_mean:0.0026548842433840036 - training/rollout_probs_diff_std:0.010791671462357044 - training/rollout_actor_probs_pearson_corr:0.9979157447814941 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.6875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.0000296831130981 - rollout_corr/rollout_is_ratio_fraction_high:2.5662740881671198e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.698821718804538e-05 - rollout_corr/rollout_is_max:3.0360376834869385 - rollout_corr/rollout_is_min:0.10401836037635803 - rollout_corr/rollout_is_std:0.030111556872725487 - rollout_corr/rollout_is_eff_sample_size:0.9990941155144449 - rollout_corr/rollout_is_seq_mean:1.000089406967163 - rollout_corr/rollout_is_seq_std:0.0016775390831753612 - rollout_corr/rollout_is_seq_max:1.008592128753662 - rollout_corr/rollout_is_seq_min:0.9943743348121643 - rollout_corr/rollout_is_seq_max_deviation:0.00859212875366211 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1033647037111223) - rollout_corr/training_log_ppl:np.float64(0.09674622296006419) - rollout_corr/kl:np.float64(0.0004946426893255307) - rollout_corr/k3_kl:np.float64(0.00056559361085462) - rollout_corr/rollout_ppl:np.float64(1.1028108289465308) - rollout_corr/rollout_log_ppl:np.float64(0.09625157990376465) - rollout_corr/log_ppl_diff:np.float64(0.0004946430562995374) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014105098089203238) - rollout_corr/log_ppl_diff_max:np.float64(0.006463304162025452) - rollout_corr/log_ppl_diff_min:np.float64(-0.007465112954378128) - rollout_corr/ppl_ratio:np.float64(1.000496445223689) - rollout_corr/chi2_token:np.float64(0.001295700902119279) - rollout_corr/chi2_seq:np.float64(1.1554163373075426) - actor/pg_loss:np.float64(0.0020622570300474763) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004437317447809619) - actor/ppo_kl:np.float64(8.988743698523649e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_w_mean:np.float64(60744.646061601816) - self_distillation/token_reweight_w_std:np.float64(1164193.8388803264) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999702645232901) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0351556129171513) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.26315246149897575) - perf/mfu/actor:np.float64(0.04286121414241698) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.01708602905273) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:2 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0013922036159783602 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0013922036159783602 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:456.64453125 - response_length/max:1317.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:456.64453125 - response_length_non_aborted/max:1317.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.8125 - prompt_length/max:365.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.762759923934937e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1173687372356653) - timing_s/agent_loop/generate_sequences/max:np.float64(9.68836759403348) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.69582801679644) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.68836759403348) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:1317 - timing_s/gen:15.383818035945296 - timing_s/reward:0.06612947955727577 - timing_s/old_log_prob:2.3371682837605476 - timing_s/adv:0.568218607455492 - timing_s/update_actor:15.863027606159449 - timing_s/step:34.31379409506917 - timing_s/stop_profile:0.00011265650391578674 - timing_per_token_ms/update_actor:0.08413926265512907 - timing_per_token_ms/adv:0.0030138946892877745 - timing_per_token_ms/gen:0.13159697552583208 - perf/total_num_tokens:188533 - perf/time_per_step:34.31379409506917 - perf/throughput:686.7974125713624 (TaskRunner pid=2074004) Training Progress: 66%|██████▌ | 66/100 [56:09<25:10, 44.42s/it] (TaskRunner pid=2074004) step:67 - global_seqlen/min:13079 - global_seqlen/max:27690 - global_seqlen/minmax_diff:14611 - global_seqlen/balanced_min:20237 - global_seqlen/balanced_max:20247 - global_seqlen/mean:20243.125 - actor/entropy:0.11189419776201248 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6611925363540649 - training/rollout_probs_diff_mean:0.0028133972082287073 - training/rollout_probs_diff_std:0.011245041154325008 - training/rollout_actor_probs_pearson_corr:0.9979284405708313 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.78125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.000016450881958 - rollout_corr/rollout_is_ratio_fraction_high:2.1283614842104726e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.385084270732477e-05 - rollout_corr/rollout_is_max:3.327455997467041 - rollout_corr/rollout_is_min:0.31944146752357483 - rollout_corr/rollout_is_std:0.03032655082643032 - rollout_corr/rollout_is_eff_sample_size:0.9990811454013858 - rollout_corr/rollout_is_seq_mean:0.9999868869781494 - rollout_corr/rollout_is_seq_std:0.0020908608566969633 - rollout_corr/rollout_is_seq_max:1.0100783109664917 - rollout_corr/rollout_is_seq_min:0.9929949045181274 - rollout_corr/rollout_is_seq_max_deviation:0.0100783109664917 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1118552824482322) - rollout_corr/training_log_ppl:np.float64(0.10242664221368614) - rollout_corr/kl:np.float64(0.0007174603885786723) - rollout_corr/k3_kl:np.float64(0.0005228211956200823) - rollout_corr/rollout_ppl:np.float64(1.111049032304436) - rollout_corr/rollout_log_ppl:np.float64(0.10170918147196062) - rollout_corr/log_ppl_diff:np.float64(0.0007174607417255174) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015918946191959549) - rollout_corr/log_ppl_diff_max:np.float64(0.009511686861515045) - rollout_corr/log_ppl_diff_min:np.float64(-0.008098289370536804) - rollout_corr/ppl_ratio:np.float64(1.000719947507605) - rollout_corr/chi2_token:np.float64(0.0006735411006957293) - rollout_corr/chi2_seq:np.float64(0.3616406440269202) - actor/pg_loss:np.float64(0.0008698338642716408) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023361428748103208) - actor/ppo_kl:np.float64(0.00022379015435802785) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_w_mean:np.float64(29018.363461346366) - self_distillation/token_reweight_w_std:np.float64(446767.4180369621) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993730406276882) - self_distillation/token_reweight_w_clip_frac:np.float64(0.017900060265674256) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.10916494252160192) - perf/mfu/actor:np.float64(0.03730973843414816) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.93292999267578) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:3 - critic/score/mean:0.78125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006233438849449158 - critic/advantages/max:0.25 - critic/advantages/min:-0.875 - critic/returns/mean:0.006233438849449158 - critic/returns/max:0.25 - critic/returns/min:-0.875 - response_length/mean:367.06640625 - response_length/max:1082.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:367.06640625 - response_length_non_aborted/max:1082.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.53125 - prompt_length/max:361.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00020816735923290253 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7952778041362762) - timing_s/agent_loop/generate_sequences/max:np.float64(7.780997171998024) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.720951149596658) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.780997171998024) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:1082 - timing_s/gen:13.588720228523016 - timing_s/reward:0.05631382390856743 - timing_s/old_log_prob:2.3990508504211903 - timing_s/adv:0.5778900533914566 - timing_s/update_actor:15.670771421864629 - timing_s/step:32.40315775945783 - timing_s/stop_profile:0.00010994821786880493 - timing_per_token_ms/update_actor:0.09676600958266467 - timing_per_token_ms/adv:0.0035684340571889013 - timing_per_token_ms/gen:0.14460854354652083 - perf/total_num_tokens:161945 - perf/time_per_step:32.40315775945783 - perf/throughput:624.7269216868668 (TaskRunner pid=2074004) Training Progress: 67%|██████▋ | 67/100 [56:42<22:32, 40.98s/it] (TaskRunner pid=2074004) step:68 - global_seqlen/min:17605 - global_seqlen/max:31191 - global_seqlen/minmax_diff:13586 - global_seqlen/balanced_min:21836 - global_seqlen/balanced_max:21847 - global_seqlen/mean:21843.875 - actor/entropy:0.09757616370916367 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34953224658966064 - training/rollout_probs_diff_mean:0.002497250447049737 - training/rollout_probs_diff_std:0.010688133537769318 - training/rollout_actor_probs_pearson_corr:0.9978483319282532 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.74609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0000418424606323 - rollout_corr/rollout_is_ratio_fraction_high:2.808383942465298e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.872255961643532e-05 - rollout_corr/rollout_is_max:2.1123147010803223 - rollout_corr/rollout_is_min:0.3512527644634247 - rollout_corr/rollout_is_std:0.030320653691887856 - rollout_corr/rollout_is_eff_sample_size:0.999081502372468 - rollout_corr/rollout_is_seq_mean:1.0001800060272217 - rollout_corr/rollout_is_seq_std:0.001909724436700344 - rollout_corr/rollout_is_seq_max:1.0095264911651611 - rollout_corr/rollout_is_seq_min:0.9944645166397095 - rollout_corr/rollout_is_seq_max_deviation:0.009526491165161133 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0923993606120348) - rollout_corr/training_log_ppl:np.float64(0.08670492109376937) - rollout_corr/kl:np.float64(0.0004997735171485829) - rollout_corr/k3_kl:np.float64(0.0006032356435454744) - rollout_corr/rollout_ppl:np.float64(1.0918437647633255) - rollout_corr/rollout_log_ppl:np.float64(0.08620514698122861) - rollout_corr/log_ppl_diff:np.float64(0.0004997741125407629) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014279280803748406) - rollout_corr/log_ppl_diff_max:np.float64(0.0074855126440525055) - rollout_corr/log_ppl_diff_min:np.float64(-0.011258374899625778) - rollout_corr/ppl_ratio:np.float64(1.0005019064992666) - rollout_corr/chi2_token:np.float64(0.0018328602891415358) - rollout_corr/chi2_seq:np.float64(0.43353482661768794) - actor/pg_loss:np.float64(0.0012217597104609013) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019400951759962481) - actor/ppo_kl:np.float64(0.0001972818646009955) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_w_mean:np.float64(60185.39716754225) - self_distillation/token_reweight_w_std:np.float64(943506.6543101566) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000661604572088) - self_distillation/token_reweight_w_clip_frac:np.float64(0.01850332456524484) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.25972285121679306) - perf/mfu/actor:np.float64(0.03952048152380881) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.89424896240234) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:3 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004791805054992437 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.004791805054992437 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:417.27734375 - response_length/max:1557.0 - response_length/min:70.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:417.27734375 - response_length_non_aborted/max:1557.0 - response_length_non_aborted/min:70.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.34375 - prompt_length/max:345.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.857063949108124e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8643373139202595) - timing_s/agent_loop/generate_sequences/max:np.float64(10.241223026067019) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.15653961370117) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.241223026067019) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1557 - timing_s/gen:16.0516545958817 - timing_s/reward:0.05962240323424339 - timing_s/old_log_prob:2.291767431423068 - timing_s/adv:0.5920419842004776 - timing_s/update_actor:15.795362647622824 - timing_s/step:34.88568533025682 - timing_s/stop_profile:0.00011338107287883759 - timing_per_token_ms/update_actor:0.09038782409040763 - timing_per_token_ms/adv:0.0033879175752955784 - timing_per_token_ms/gen:0.15026403111578687 - perf/total_num_tokens:174751 - perf/time_per_step:34.88568533025682 - perf/throughput:626.1558227452828 (TaskRunner pid=2074004) Training Progress: 68%|██████▊ | 68/100 [57:16<20:53, 39.17s/it] (TaskRunner pid=2074004) step:69 - global_seqlen/min:18936 - global_seqlen/max:25688 - global_seqlen/minmax_diff:6752 - global_seqlen/balanced_min:22204 - global_seqlen/balanced_max:22257 - global_seqlen/mean:22220.75 - actor/entropy:0.10731451213359833 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4198373854160309 - training/rollout_probs_diff_mean:0.0025957662146538496 - training/rollout_probs_diff_std:0.010541104711592197 - training/rollout_actor_probs_pearson_corr:0.9981012940406799 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.73046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.000120997428894 - rollout_corr/rollout_is_ratio_fraction_high:1.8439637642586604e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.687927528517321e-05 - rollout_corr/rollout_is_max:24.342180252075195 - rollout_corr/rollout_is_min:0.34072938561439514 - rollout_corr/rollout_is_std:0.02951376512646675 - rollout_corr/rollout_is_eff_sample_size:0.9991294578066934 - rollout_corr/rollout_is_seq_mean:1.0000665187835693 - rollout_corr/rollout_is_seq_std:0.0023733978159725666 - rollout_corr/rollout_is_seq_max:1.0234099626541138 - rollout_corr/rollout_is_seq_min:0.9945483207702637 - rollout_corr/rollout_is_seq_max_deviation:0.02340996265411377 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1120971874333918) - rollout_corr/training_log_ppl:np.float64(0.1024366507299419) - rollout_corr/kl:np.float64(0.00044801794052590216) - rollout_corr/k3_kl:np.float64(0.0006100563592355002) - rollout_corr/rollout_ppl:np.float64(1.111582346726209) - rollout_corr/rollout_log_ppl:np.float64(0.10198863269033609) - rollout_corr/log_ppl_diff:np.float64(0.0004480180396058131) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014786184001422953) - rollout_corr/log_ppl_diff_max:np.float64(0.005983397364616394) - rollout_corr/log_ppl_diff_min:np.float64(-0.010212674736976624) - rollout_corr/ppl_ratio:np.float64(1.0004499880596995) - rollout_corr/chi2_token:np.float64(0.0031010701786726713) - rollout_corr/chi2_seq:np.float64(0.385674498276785) - actor/pg_loss:np.float64(0.001082204282283783) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022557121974386973) - actor/ppo_kl:np.float64(-3.220176354723492e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_w_mean:np.float64(74030.12377202115) - self_distillation/token_reweight_w_std:np.float64(1098970.7254105331) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995012604631484) - self_distillation/token_reweight_w_clip_frac:np.float64(0.028009024827042595) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.1886149998754263) - perf/mfu/actor:np.float64(0.04061615996018782) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.03355026245117) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:3 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00016365178453270346 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00016365178453270346 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:423.6796875 - response_length/max:1407.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:423.6796875 - response_length_non_aborted/max:1407.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:363.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001237001270055771 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7061853520572186) - timing_s/agent_loop/generate_sequences/max:np.float64(9.594152119010687) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.148857997002779) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.594152119010687) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:356 - timing_s/agent_loop/slowest/response_length:1407 - timing_s/gen:15.759770501405 - timing_s/reward:0.05864102765917778 - timing_s/old_log_prob:2.375713935121894 - timing_s/adv:0.5674699656665325 - timing_s/update_actor:15.860491406172514 - timing_s/step:34.719941690564156 - timing_s/stop_profile:0.00012359023094177246 - timing_per_token_ms/update_actor:0.08922117506256828 - timing_per_token_ms/adv:0.0031922300421145353 - timing_per_token_ms/gen:0.14530223028715128 - perf/total_num_tokens:177766 - perf/time_per_step:34.719941690564156 - perf/throughput:639.9996347355311 (TaskRunner pid=2074004) Training Progress: 69%|██████▉ | 69/100 [57:51<19:33, 37.85s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. We are to find the magnitude of the charge $ q $ using the formula: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Rearranging for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substituting the given values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest answer is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (WorkerDict pid=2074385) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /Qwen/Qwen3-4B/resolve/main/generation_config.json (Caused by SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1010)')))"), '(Request ID: 123c1826-04a8-41a1-bd46-2674c1e317a5)')' thrown while requesting HEAD https://huggingface.co/Qwen/Qwen3-4B/resolve/main/generation_config.json (WorkerDict pid=2074385) Retrying in 1s [Retry 1/5]. (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60/actor (TaskRunner pid=2074004) step:70 - global_seqlen/min:16730 - global_seqlen/max:27137 - global_seqlen/minmax_diff:10407 - global_seqlen/balanced_min:21643 - global_seqlen/balanced_max:21648 - global_seqlen/mean:21645.5 - actor/entropy:0.11388496309518814 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5830116271972656 - training/rollout_probs_diff_mean:0.0028158947825431824 - training/rollout_probs_diff_std:0.011242561973631382 - training/rollout_actor_probs_pearson_corr:0.9978851675987244 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:1.0001726150512695 - rollout_corr/rollout_is_ratio_fraction_high:1.9112419977318496e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.556209988659248e-05 - rollout_corr/rollout_is_max:2.3941094875335693 - rollout_corr/rollout_is_min:0.20454047620296478 - rollout_corr/rollout_is_std:0.03099912963807583 - rollout_corr/rollout_is_eff_sample_size:0.9990402144088839 - rollout_corr/rollout_is_seq_mean:1.0003561973571777 - rollout_corr/rollout_is_seq_std:0.0019966252148151398 - rollout_corr/rollout_is_seq_max:1.0088205337524414 - rollout_corr/rollout_is_seq_min:0.9942991137504578 - rollout_corr/rollout_is_seq_max_deviation:0.008820533752441406 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1065151896327734) - rollout_corr/training_log_ppl:np.float64(0.09889099766587606) - rollout_corr/kl:np.float64(-3.2485000064363945e-05) - rollout_corr/k3_kl:np.float64(0.0005556557533132889) - rollout_corr/rollout_ppl:np.float64(1.106540143955499) - rollout_corr/rollout_log_ppl:np.float64(0.09892348164066789) - rollout_corr/log_ppl_diff:np.float64(-3.2483974791830406e-05) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015090550186869223) - rollout_corr/log_ppl_diff_max:np.float64(0.014128297567367554) - rollout_corr/log_ppl_diff_min:np.float64(-0.008920878171920776) - rollout_corr/ppl_ratio:np.float64(0.999969924101606) - rollout_corr/chi2_token:np.float64(0.0022924926597625017) - rollout_corr/chi2_seq:np.float64(0.9184615898411721) - actor/pg_loss:np.float64(0.00168591586407274) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006731084324655967) - actor/ppo_kl:np.float64(-0.00014355223837636544) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(99180.82018345594) - self_distillation/token_reweight_w_std:np.float64(1940147.8677054113) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003361816052347) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04243566015793476) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.27422183752059937) - perf/mfu/actor:np.float64(0.03944531836232677) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.0858383178711) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.64375) - val-aux/sciknoweval/reward/std@16:np.float64(0.17544167945053135) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7129875) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1322752335653467) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5741999999999999) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1624310913125313) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6450374999999999) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.17432640534203808) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7630750000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.0897566649865629) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.510825) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.13892894608045508) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6609999999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1328026930026406) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7967500000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.05859835503578027) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.45375000000000004) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.1090491129202408) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.666575) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09131931477917679) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8197624999999998) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03386172398893218) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.40679999999999994) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07785458149403679) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6700250000000001) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06801004833603) - val-aux/sciknoweval/score/mean@16:np.float64(0.64375) - val-aux/sciknoweval/score/std@16:np.float64(0.17544167945053135) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7129875) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1322752335653467) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5741999999999999) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1624310913125313) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6450374999999999) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.17432640534203808) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7630750000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.0897566649865629) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.510825) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.13892894608045508) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6609999999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1328026930026406) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7967500000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.05859835503578027) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.45375000000000004) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.1090491129202408) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.666575) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09131931477917679) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8197624999999998) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03386172398893218) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.40679999999999994) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07785458149403679) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6700250000000001) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06801004833603) - val-core/sciknoweval/acc/mean@16:np.float64(0.64375) - val-aux/sciknoweval/acc/std@16:np.float64(0.17544167945053135) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7129875) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1322752335653467) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5741999999999999) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1624310913125313) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6450374999999999) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.17432640534203808) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7630750000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.0897566649865629) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.510825) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.13892894608045508) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6609999999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1328026930026406) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7967500000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.05859835503578027) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.45375000000000004) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.1090491129202408) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.666575) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09131931477917679) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8197624999999998) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03386172398893218) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.40679999999999994) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07785458149403679) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6700250000000001) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06801004833603) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99296875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.023033514121596016) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9992374999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.006965874281756439) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9869375) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.030132003555088865) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9933) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.022640924934477863) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999750000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0005584576975922169) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9765375000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.037335609975106154) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.997975) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.012752016030515461) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9597750000000002) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.04184278385125003) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9996875000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0044696795477923035) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9392125) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03812973334861979) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999874999999999) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0003950870157319778) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:3 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011775638908147812 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011775638908147812 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:408.765625 - response_length/max:1282.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:408.765625 - response_length_non_aborted/max:1282.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.65625 - prompt_length/max:375.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017072074115276337 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7840455882251263) - timing_s/agent_loop/generate_sequences/max:np.float64(8.811903426423669) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.002542052578065) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.811903426423669) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:350 - timing_s/agent_loop/slowest/response_length:1198 - timing_s/gen:14.664157785475254 - timing_s/reward:0.06123044900596142 - timing_s/old_log_prob:2.381863748654723 - timing_s/adv:0.5689894556999207 - timing_s/update_actor:15.963874887675047 - timing_s/step:33.727847304195166 - timing_s/testing:91.3584210369736 - timing_s/save_checkpoint:7.738546384498477 - timing_s/stop_profile:0.00014872848987579346 - timing_per_token_ms/update_actor:0.09218934009190737 - timing_per_token_ms/adv:0.003285841489570122 - timing_per_token_ms/gen:0.1401337657722875 - perf/total_num_tokens:173164 - perf/time_per_step:33.727847304195166 - perf/throughput:641.7693902838463 (TaskRunner pid=2074004) Training Progress: 70%|███████ | 70/100 [1:00:04<33:10, 66.36s/it] (TaskRunner pid=2074004) step:71 - global_seqlen/min:17341 - global_seqlen/max:23612 - global_seqlen/minmax_diff:6271 - global_seqlen/balanced_min:20701 - global_seqlen/balanced_max:20715 - global_seqlen/mean:20711.25 - actor/entropy:0.11080419272184372 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.531029462814331 - training/rollout_probs_diff_mean:0.002882769564166665 - training/rollout_probs_diff_std:0.011653199791908264 - training/rollout_actor_probs_pearson_corr:0.9976783990859985 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.74609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.0000351667404175 - rollout_corr/rollout_is_ratio_fraction_high:1.0566133823886048e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.452907059108838e-05 - rollout_corr/rollout_is_max:2.691452741622925 - rollout_corr/rollout_is_min:0.31489646434783936 - rollout_corr/rollout_is_std:0.031141091138124466 - rollout_corr/rollout_is_eff_sample_size:0.9990312909459407 - rollout_corr/rollout_is_seq_mean:1.0000897645950317 - rollout_corr/rollout_is_seq_std:0.0019137562485411763 - rollout_corr/rollout_is_seq_max:1.0093891620635986 - rollout_corr/rollout_is_seq_min:0.9933313727378845 - rollout_corr/rollout_is_seq_max_deviation:0.009389162063598633 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1135450787842274) - rollout_corr/training_log_ppl:np.float64(0.1041237109056965) - rollout_corr/kl:np.float64(0.0005952388999048708) - rollout_corr/k3_kl:np.float64(0.0005321949834606876) - rollout_corr/rollout_ppl:np.float64(1.1128493160940707) - rollout_corr/rollout_log_ppl:np.float64(0.10352847058675252) - rollout_corr/log_ppl_diff:np.float64(0.0005952403189439792) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014877674730087165) - rollout_corr/log_ppl_diff_max:np.float64(0.00793486088514328) - rollout_corr/log_ppl_diff_min:np.float64(-0.004785224795341492) - rollout_corr/ppl_ratio:np.float64(1.0005972273647785) - rollout_corr/chi2_token:np.float64(0.0009104576893150806) - rollout_corr/chi2_seq:np.float64(0.7369661091361195) - actor/pg_loss:np.float64(0.0014374544844031334) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003361999638400448) - actor/ppo_kl:np.float64(0.00016070706497472997) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_w_mean:np.float64(28669.062551695388) - self_distillation/token_reweight_w_std:np.float64(576396.9004883079) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999415366910398) - self_distillation/token_reweight_w_clip_frac:np.float64(0.028172490041470155) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.14137845113873482) - perf/mfu/actor:np.float64(0.037758217715448655) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.2748680114746) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:3 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003997960593551397 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003997960593551397 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:369.6953125 - response_length/max:1046.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:369.6953125 - response_length_non_aborted/max:1046.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:437.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014478527009487152 - timing_s/agent_loop/generate_sequences/min:np.float64(1.12398418225348) - timing_s/agent_loop/generate_sequences/max:np.float64(7.80044724419713) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.838224976650963) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.80044724419713) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:310 - timing_s/agent_loop/slowest/response_length:1046 - timing_s/gen:13.64636124484241 - timing_s/reward:0.05765148997306824 - timing_s/old_log_prob:2.378165924921632 - timing_s/adv:0.5682110711932182 - timing_s/update_actor:15.703643430024385 - timing_s/step:32.44597504287958 - timing_s/stop_profile:0.00010780245065689087 - timing_per_token_ms/update_actor:0.09477725529618194 - timing_per_token_ms/adv:0.0034293624913586712 - timing_per_token_ms/gen:0.14418927373515364 - perf/total_num_tokens:165690 - perf/time_per_step:32.44597504287958 - perf/throughput:638.3303313470673 (TaskRunner pid=2074004) Training Progress: 71%|███████ | 71/100 [1:00:37<27:09, 56.20s/it] (TaskRunner pid=2074004) step:72 - global_seqlen/min:18266 - global_seqlen/max:27074 - global_seqlen/minmax_diff:8808 - global_seqlen/balanced_min:22610 - global_seqlen/balanced_max:22942 - global_seqlen/mean:22655.75 - actor/entropy:0.09876604378223419 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5981020331382751 - training/rollout_probs_diff_mean:0.0024195632431656122 - training/rollout_probs_diff_std:0.010534233413636684 - training/rollout_actor_probs_pearson_corr:0.9980013966560364 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.75390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:1.0000324249267578 - rollout_corr/rollout_is_ratio_fraction_high:2.6222400265396573e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.370399983599782e-05 - rollout_corr/rollout_is_max:2.0842576026916504 - rollout_corr/rollout_is_min:0.0317724235355854 - rollout_corr/rollout_is_std:0.02847144566476345 - rollout_corr/rollout_is_eff_sample_size:0.9991900334087397 - rollout_corr/rollout_is_seq_mean:1.0000596046447754 - rollout_corr/rollout_is_seq_std:0.0016953593585640192 - rollout_corr/rollout_is_seq_max:1.010093092918396 - rollout_corr/rollout_is_seq_min:0.9925937652587891 - rollout_corr/rollout_is_seq_max_deviation:0.010093092918395996 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1005978477187455) - rollout_corr/training_log_ppl:np.float64(0.09388212007979746) - rollout_corr/kl:np.float64(0.000452447483150209) - rollout_corr/k3_kl:np.float64(0.0004727034649363304) - rollout_corr/rollout_ppl:np.float64(1.1000874377787113) - rollout_corr/rollout_log_ppl:np.float64(0.09342967234624666) - rollout_corr/log_ppl_diff:np.float64(0.00045244773355079815) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013087687766528688) - rollout_corr/log_ppl_diff_max:np.float64(0.008923083543777466) - rollout_corr/log_ppl_diff_min:np.float64(-0.006535425782203674) - rollout_corr/ppl_ratio:np.float64(1.0004541566595435) - rollout_corr/chi2_token:np.float64(0.0009954054839909077) - rollout_corr/chi2_seq:np.float64(0.5610992929432541) - actor/pg_loss:np.float64(0.0011489454191178083) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003143757853649731) - actor/ppo_kl:np.float64(5.7555564531242e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75390625) - self_distillation/token_reweight_w_mean:np.float64(69880.04204755742) - self_distillation/token_reweight_w_std:np.float64(1271668.7369876243) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998061931692064) - self_distillation/token_reweight_w_clip_frac:np.float64(0.025518682843539864) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.18177369236946106) - perf/mfu/actor:np.float64(0.04106344764567454) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.01401233673096) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:3 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008316871710121632 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008316871710121632 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:446.8984375 - response_length/max:1865.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:446.8984375 - response_length_non_aborted/max:1865.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.09375 - prompt_length/max:355.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010162405669689178 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4154884051531553) - timing_s/agent_loop/generate_sequences/max:np.float64(12.022635696455836) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.678455237481103) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.022635696455836) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:233 - timing_s/agent_loop/slowest/response_length:1865 - timing_s/gen:17.75904706120491 - timing_s/reward:0.0738532543182373 - timing_s/old_log_prob:2.3650817554444075 - timing_s/adv:0.630233459174633 - timing_s/update_actor:15.826980471611023 - timing_s/step:36.74509644508362 - timing_s/stop_profile:0.0001101028174161911 - timing_per_token_ms/update_actor:0.08732319870017007 - timing_per_token_ms/adv:0.0034772268583838153 - timing_per_token_ms/gen:0.15522828401661548 - perf/total_num_tokens:181246 - perf/time_per_step:36.74509644508362 - perf/throughput:616.5652615406666 (TaskRunner pid=2074004) Training Progress: 72%|███████▏ | 72/100 [1:01:13<23:30, 50.38s/it] (TaskRunner pid=2074004) step:73 - global_seqlen/min:16861 - global_seqlen/max:32908 - global_seqlen/minmax_diff:16047 - global_seqlen/balanced_min:26274 - global_seqlen/balanced_max:26303 - global_seqlen/mean:26293.375 - actor/entropy:0.11385305970907211 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5037192106246948 - training/rollout_probs_diff_mean:0.0026417120825499296 - training/rollout_probs_diff_std:0.011010304093360901 - training/rollout_actor_probs_pearson_corr:0.9979777932167053 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9999186396598816 - rollout_corr/rollout_is_ratio_fraction_high:2.1509231373784132e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.603692549513653e-05 - rollout_corr/rollout_is_max:2.2957048416137695 - rollout_corr/rollout_is_min:0.3969012498855591 - rollout_corr/rollout_is_std:0.03009967692196369 - rollout_corr/rollout_is_eff_sample_size:0.9990945914883025 - rollout_corr/rollout_is_seq_mean:0.9999231696128845 - rollout_corr/rollout_is_seq_std:0.0021570948883891106 - rollout_corr/rollout_is_seq_max:1.0173187255859375 - rollout_corr/rollout_is_seq_min:0.9907941818237305 - rollout_corr/rollout_is_seq_max_deviation:0.0173187255859375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.119977319613099) - rollout_corr/training_log_ppl:np.float64(0.11065010864695068) - rollout_corr/kl:np.float64(0.0006033849298385796) - rollout_corr/k3_kl:np.float64(0.0006128351814460586) - rollout_corr/rollout_ppl:np.float64(1.1192855625413358) - rollout_corr/rollout_log_ppl:np.float64(0.11004672372655477) - rollout_corr/log_ppl_diff:np.float64(0.0006033849203959107) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016537990886718035) - rollout_corr/log_ppl_diff_max:np.float64(0.010391712188720703) - rollout_corr/log_ppl_diff_min:np.float64(-0.010457545518875122) - rollout_corr/ppl_ratio:np.float64(1.0006065221969038) - rollout_corr/chi2_token:np.float64(0.0012946291826665401) - rollout_corr/chi2_seq:np.float64(1.4108337932266295) - actor/pg_loss:np.float64(0.0022633997723460197) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00048230918173430837) - actor/ppo_kl:np.float64(-3.4695178307231345e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(100532.0445677184) - self_distillation/token_reweight_w_std:np.float64(1938239.1147645125) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999280977062881) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04582360343192704) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.24655815586447716) - perf/mfu/actor:np.float64(0.04701854823971872) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.03591918945312) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:3 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002752285450696945 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002752285450696945 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:544.82421875 - response_length/max:1831.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:544.82421875 - response_length_non_aborted/max:1831.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.84375 - prompt_length/max:375.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011086836457252502 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8580625150352716) - timing_s/agent_loop/generate_sequences/max:np.float64(13.18264446593821) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.298752498922113) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.18264446593821) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:1831 - timing_s/gen:19.2261294554919 - timing_s/reward:0.06619958020746708 - timing_s/old_log_prob:2.412239581346512 - timing_s/adv:0.6206546351313591 - timing_s/update_actor:16.13218855857849 - timing_s/step:38.54634447954595 - timing_s/stop_profile:0.0001186523586511612 - timing_per_token_ms/update_actor:0.07669321910261849 - timing_per_token_ms/adv:0.002950622709767 - timing_per_token_ms/gen:0.13784642018635526 - perf/total_num_tokens:210347 - perf/time_per_step:38.54634447954595 - perf/throughput:682.1236969423181 (TaskRunner pid=2074004) Training Progress: 73%|███████▎ | 73/100 [1:01:52<21:04, 46.84s/it] (TaskRunner pid=2074004) step:74 - global_seqlen/min:15082 - global_seqlen/max:27595 - global_seqlen/minmax_diff:12513 - global_seqlen/balanced_min:20956 - global_seqlen/balanced_max:21007 - global_seqlen/mean:20963.375 - actor/entropy:0.1221182718873024 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4268117845058441 - training/rollout_probs_diff_mean:0.0029213139787316322 - training/rollout_probs_diff_std:0.011437395587563515 - training/rollout_actor_probs_pearson_corr:0.9979726672172546 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90625 - self_distillation/correct_sample_fraction:0.7890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90625 - rollout_corr/rollout_is_mean:0.9998773336410522 - rollout_corr/rollout_is_ratio_fraction_high:9.682135896582622e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.682135714683682e-05 - rollout_corr/rollout_is_max:2.0322699546813965 - rollout_corr/rollout_is_min:0.3633837103843689 - rollout_corr/rollout_is_std:0.03148183599114418 - rollout_corr/rollout_is_eff_sample_size:0.9990097563119252 - rollout_corr/rollout_is_seq_mean:1.0000077486038208 - rollout_corr/rollout_is_seq_std:0.0027337686624377966 - rollout_corr/rollout_is_seq_max:1.0147820711135864 - rollout_corr/rollout_is_seq_min:0.9881354570388794 - rollout_corr/rollout_is_seq_max_deviation:0.014782071113586426 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1157428310252726) - rollout_corr/training_log_ppl:np.float64(0.10688100929837674) - rollout_corr/kl:np.float64(0.0004768247579676199) - rollout_corr/k3_kl:np.float64(0.0006016286499246348) - rollout_corr/rollout_ppl:np.float64(1.1151970326900482) - rollout_corr/rollout_log_ppl:np.float64(0.10640418373441207) - rollout_corr/log_ppl_diff:np.float64(0.0004768255639646668) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018265418002556544) - rollout_corr/log_ppl_diff_max:np.float64(0.014855928719043732) - rollout_corr/log_ppl_diff_min:np.float64(-0.011131703853607178) - rollout_corr/ppl_ratio:np.float64(1.0004803847987205) - rollout_corr/chi2_token:np.float64(0.0014686237554997206) - rollout_corr/chi2_seq:np.float64(0.33443050901405513) - actor/pg_loss:np.float64(0.0020010630832985044) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00046587054316660215) - actor/ppo_kl:np.float64(-7.767568073546727e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7890625) - self_distillation/token_reweight_w_mean:np.float64(61973.32169602462) - self_distillation/token_reweight_w_std:np.float64(1112938.32659155) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995205644518137) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03652457233692985) - self_distillation/empty_target_batch:np.float64(0.09375) - actor/grad_norm:np.float64(0.23524155095219612) - perf/mfu/actor:np.float64(0.03752256342730821) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.06950759887695) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:3 - critic/score/mean:0.7890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0041500055231153965 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0041500055231153965 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:403.44921875 - response_length/max:1678.0 - response_length/min:48.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:403.44921875 - response_length_non_aborted/max:1678.0 - response_length_non_aborted/min:48.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.65625 - prompt_length/max:353.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013217143714427948 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5993990004062653) - timing_s/agent_loop/generate_sequences/max:np.float64(11.049694247543812) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.951693204406183) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.049694247543812) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:353 - timing_s/agent_loop/slowest/response_length:1678 - timing_s/gen:16.895056227222085 - timing_s/reward:0.058270735666155815 - timing_s/old_log_prob:2.356802634894848 - timing_s/adv:0.6194801684468985 - timing_s/update_actor:16.044949045404792 - timing_s/step:36.06812138669193 - timing_s/stop_profile:0.00010689161717891693 - timing_per_token_ms/update_actor:0.09567250648693729 - timing_per_token_ms/adv:0.003693824160272967 - timing_per_token_ms/gen:0.16358022353361235 - perf/total_num_tokens:167707 - perf/time_per_step:36.06812138669193 - perf/throughput:581.2161597009282 (TaskRunner pid=2074004) Training Progress: 74%|███████▍ | 74/100 [1:02:28<18:54, 43.62s/it] (TaskRunner pid=2074004) step:75 - global_seqlen/min:18314 - global_seqlen/max:24550 - global_seqlen/minmax_diff:6236 - global_seqlen/balanced_min:20293 - global_seqlen/balanced_max:27015 - global_seqlen/mean:21133.625 - actor/entropy:0.07867328822612762 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7417161464691162 - training/rollout_probs_diff_mean:0.002312364522367716 - training/rollout_probs_diff_std:0.010702241212129593 - training/rollout_actor_probs_pearson_corr:0.9973950982093811 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.93359375 - self_distillation/correct_sample_fraction:0.875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.93359375 - rollout_corr/rollout_is_mean:0.9999236464500427 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.896926136803813e-05 - rollout_corr/rollout_is_max:1.7065232992172241 - rollout_corr/rollout_is_min:0.09278460592031479 - rollout_corr/rollout_is_std:0.027797745540738106 - rollout_corr/rollout_is_eff_sample_size:0.9992276439614955 - rollout_corr/rollout_is_seq_mean:0.999890923500061 - rollout_corr/rollout_is_seq_std:0.00215591280721128 - rollout_corr/rollout_is_seq_max:1.0128012895584106 - rollout_corr/rollout_is_seq_min:0.9893057942390442 - rollout_corr/rollout_is_seq_max_deviation:0.012801289558410645 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.086495197378099) - rollout_corr/training_log_ppl:np.float64(0.0809084869997605) - rollout_corr/kl:np.float64(0.0006969666031730881) - rollout_corr/k3_kl:np.float64(0.000509651741253947) - rollout_corr/rollout_ppl:np.float64(1.085711132735014) - rollout_corr/rollout_log_ppl:np.float64(0.08021151977845875) - rollout_corr/log_ppl_diff:np.float64(0.0006969672213017475) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016032274397730362) - rollout_corr/log_ppl_diff_max:np.float64(0.012572050094604492) - rollout_corr/log_ppl_diff_min:np.float64(-0.01109679788351059) - rollout_corr/ppl_ratio:np.float64(1.0006995955482125) - rollout_corr/chi2_token:np.float64(0.0006079066079109907) - rollout_corr/chi2_seq:np.float64(0.20879352744668722) - actor/pg_loss:np.float64(0.0010695301461964846) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000338989874762774) - actor/ppo_kl:np.float64(0.00011593784117280848) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.93359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.93359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_w_mean:np.float64(98768.57021646714) - self_distillation/token_reweight_w_std:np.float64(1371672.2499703786) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993643839843571) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02514167891058605) - self_distillation/empty_target_batch:np.float64(0.06640625) - actor/grad_norm:np.float64(0.19157380517572165) - perf/mfu/actor:np.float64(0.0367450268799295) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.9773292541504) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:3 - critic/score/mean:0.875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.07149641960859299 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.07149641960859299 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:400.95703125 - response_length/max:8192.0 - response_length/min:76.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:400.95703125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:259.46875 - prompt_length/max:383.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013606809079647064 - timing_s/agent_loop/generate_sequences/min:np.float64(0.973297132179141) - timing_s/agent_loop/generate_sequences/max:np.float64(50.69700602442026) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.94060600863304) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.69700602442026) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:261 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.43104676157236 - timing_s/reward:0.05962674133479595 - timing_s/old_log_prob:2.578733177855611 - timing_s/adv:0.56367601826787 - timing_s/update_actor:16.684449028223753 - timing_s/step:76.40252554044127 - timing_s/stop_profile:0.00010974332690238953 - timing_per_token_ms/update_actor:0.09868425925642048 - timing_per_token_ms/adv:0.003333999835971526 - timing_per_token_ms/gen:0.5497690755669771 - perf/total_num_tokens:169069 - perf/time_per_step:76.40252554044127 - perf/throughput:276.6089844610383 (TaskRunner pid=2074004) Training Progress: 75%|███████▌ | 75/100 [1:03:45<22:17, 53.48s/it] (TaskRunner pid=2074004) step:76 - global_seqlen/min:19671 - global_seqlen/max:26890 - global_seqlen/minmax_diff:7219 - global_seqlen/balanced_min:23504 - global_seqlen/balanced_max:23513 - global_seqlen/mean:23508.125 - actor/entropy:0.11660213023424149 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5008273124694824 - training/rollout_probs_diff_mean:0.00285673257894814 - training/rollout_probs_diff_std:0.011434005573391914 - training/rollout_actor_probs_pearson_corr:0.9978346228599548 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:0.9998456835746765 - rollout_corr/rollout_is_ratio_fraction_high:1.7103226127801463e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010261935676680878 - rollout_corr/rollout_is_max:3.2767059803009033 - rollout_corr/rollout_is_min:0.304049551486969 - rollout_corr/rollout_is_std:0.03136803209781647 - rollout_corr/rollout_is_eff_sample_size:0.9990166568117982 - rollout_corr/rollout_is_seq_mean:0.9999971985816956 - rollout_corr/rollout_is_seq_std:0.002045826055109501 - rollout_corr/rollout_is_seq_max:1.0144394636154175 - rollout_corr/rollout_is_seq_min:0.9931577444076538 - rollout_corr/rollout_is_seq_max_deviation:0.01443946361541748 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.117191367316991) - rollout_corr/training_log_ppl:np.float64(0.108617360041535) - rollout_corr/kl:np.float64(0.0005786410445907109) - rollout_corr/k3_kl:np.float64(0.0006542783519307704) - rollout_corr/rollout_ppl:np.float64(1.1165225864388049) - rollout_corr/rollout_log_ppl:np.float64(0.10803871830285061) - rollout_corr/log_ppl_diff:np.float64(0.0005786417386843823) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014457525903708301) - rollout_corr/log_ppl_diff_max:np.float64(0.007953882217407227) - rollout_corr/log_ppl_diff_min:np.float64(-0.012358836829662323) - rollout_corr/ppl_ratio:np.float64(1.0005807224661112) - rollout_corr/chi2_token:np.float64(0.0018206238746643066) - rollout_corr/chi2_seq:np.float64(0.1669981717132032) - actor/pg_loss:np.float64(0.0019441124750301242) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005475720254253247) - actor/ppo_kl:np.float64(3.105731396224609e-05) - actor/pg_clipfrac_lower:np.float64(1.1507097951835021e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_w_mean:np.float64(69920.34874183638) - self_distillation/token_reweight_w_std:np.float64(1212741.120844526) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996711185667664) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04925982659915462) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.2786094695329666) - perf/mfu/actor:np.float64(0.04275767300860228) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.03911590576172) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:3 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007215423509478569 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007215423509478569 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:456.78515625 - response_length/max:1205.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:456.78515625 - response_length_non_aborted/max:1205.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.84375 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011574104428291321 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9071431942284107) - timing_s/agent_loop/generate_sequences/max:np.float64(9.096599321812391) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.676992036926094) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.096599321812391) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:1205 - timing_s/gen:15.07368259690702 - timing_s/reward:0.08101079240441322 - timing_s/old_log_prob:2.4621802624315023 - timing_s/adv:0.5551798567175865 - timing_s/update_actor:15.884726317599416 - timing_s/step:34.14583137631416 - timing_s/stop_profile:0.00012441538274288177 - timing_per_token_ms/update_actor:0.08446402210724704 - timing_per_token_ms/adv:0.0029520636839262303 - timing_per_token_ms/gen:0.1289043040005047 - perf/total_num_tokens:188065 - perf/time_per_step:34.14583137631416 - perf/throughput:688.4625165784311 (TaskRunner pid=2074004) Training Progress: 76%|███████▌ | 76/100 [1:04:19<19:04, 47.70s/it] (TaskRunner pid=2074004) step:77 - global_seqlen/min:15412 - global_seqlen/max:29304 - global_seqlen/minmax_diff:13892 - global_seqlen/balanced_min:21767 - global_seqlen/balanced_max:21798 - global_seqlen/mean:21789.125 - actor/entropy:0.0992889478802681 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9991886615753174 - training/rollout_probs_diff_mean:0.00266986689530313 - training/rollout_probs_diff_std:0.011961651965975761 - training/rollout_actor_probs_pearson_corr:0.9973249435424805 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.73828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:0.9998555779457092 - rollout_corr/rollout_is_ratio_fraction_high:2.7756959752878174e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.327088289661333e-05 - rollout_corr/rollout_is_max:4.194519519805908 - rollout_corr/rollout_is_min:6.96785764375818e-06 - rollout_corr/rollout_is_std:0.03096642531454563 - rollout_corr/rollout_is_eff_sample_size:0.999041761158671 - rollout_corr/rollout_is_seq_mean:0.9997901320457458 - rollout_corr/rollout_is_seq_std:0.002063008723780513 - rollout_corr/rollout_is_seq_max:1.0076104402542114 - rollout_corr/rollout_is_seq_min:0.991796612739563 - rollout_corr/rollout_is_seq_max_deviation:0.008203387260437012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.100688367150724) - rollout_corr/training_log_ppl:np.float64(0.09442752133691101) - rollout_corr/kl:np.float64(0.0008365866270294564) - rollout_corr/k3_kl:np.float64(0.0005724456864371064) - rollout_corr/rollout_ppl:np.float64(1.0997380977496505) - rollout_corr/rollout_log_ppl:np.float64(0.0935909333784366) - rollout_corr/log_ppl_diff:np.float64(0.0008365879584744107) - rollout_corr/log_ppl_abs_diff:np.float64(0.001732913853629725) - rollout_corr/log_ppl_diff_max:np.float64(0.011707089841365814) - rollout_corr/log_ppl_diff_min:np.float64(-0.0054268017411231995) - rollout_corr/ppl_ratio:np.float64(1.0008395314216614) - rollout_corr/chi2_token:np.float64(0.0006989024113863707) - rollout_corr/chi2_seq:np.float64(0.36190675757825375) - actor/pg_loss:np.float64(0.0014117311220616102) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00031758257955516456) - actor/ppo_kl:np.float64(3.489058610828266e-05) - actor/pg_clipfrac_lower:np.float64(1.2245297511981335e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_w_mean:np.float64(78938.03803747497) - self_distillation/token_reweight_w_std:np.float64(1405159.017775763) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000561457127333) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02887801337055862) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.1942897979170084) - perf/mfu/actor:np.float64(0.039820481115549144) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.1161994934082) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:3 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002417168579995632 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002417168579995632 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:422.19140625 - response_length/max:1559.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:422.19140625 - response_length_non_aborted/max:1559.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.71875 - prompt_length/max:355.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013943202793598175 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1664074026048183) - timing_s/agent_loop/generate_sequences/max:np.float64(10.330939007923007) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.180417315277737) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.330939007923007) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:344 - timing_s/agent_loop/slowest/response_length:1540 - timing_s/gen:16.243871303275228 - timing_s/reward:0.05987227149307728 - timing_s/old_log_prob:2.3679722510278225 - timing_s/adv:0.5635029003024101 - timing_s/update_actor:15.628741595894098 - timing_s/step:34.95607234351337 - timing_s/stop_profile:0.00011238269507884979 - timing_per_token_ms/update_actor:0.08965907072848324 - timing_per_token_ms/adv:0.0032327072582217626 - timing_per_token_ms/gen:0.15029349564932992 - perf/total_num_tokens:174313 - perf/time_per_step:34.95607234351337 - perf/throughput:623.3287534674445 (TaskRunner pid=2074004) Training Progress: 77%|███████▋ | 77/100 [1:04:54<16:49, 43.89s/it] (TaskRunner pid=2074004) step:78 - global_seqlen/min:14364 - global_seqlen/max:26342 - global_seqlen/minmax_diff:11978 - global_seqlen/balanced_min:20665 - global_seqlen/balanced_max:20678 - global_seqlen/mean:20672.125 - actor/entropy:0.10278760641813278 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30106163024902344 - training/rollout_probs_diff_mean:0.0025821924209594727 - training/rollout_probs_diff_std:0.010455216281116009 - training/rollout_actor_probs_pearson_corr:0.9980041980743408 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9375 - self_distillation/correct_sample_fraction:0.859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9375 - rollout_corr/rollout_is_mean:0.9999585151672363 - rollout_corr/rollout_is_ratio_fraction_high:1.0236567504762206e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.0946270019048825e-05 - rollout_corr/rollout_is_max:2.61385440826416 - rollout_corr/rollout_is_min:0.18725009262561798 - rollout_corr/rollout_is_std:0.03052050806581974 - rollout_corr/rollout_is_eff_sample_size:0.9990692465112823 - rollout_corr/rollout_is_seq_mean:0.9999092817306519 - rollout_corr/rollout_is_seq_std:0.0021261873189359903 - rollout_corr/rollout_is_seq_max:1.0074591636657715 - rollout_corr/rollout_is_seq_min:0.9891470670700073 - rollout_corr/rollout_is_seq_max_deviation:0.010852932929992676 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1040684799663723) - rollout_corr/training_log_ppl:np.float64(0.09624901220013271) - rollout_corr/kl:np.float64(0.0012303921190799372) - rollout_corr/k3_kl:np.float64(0.0009560748116967943) - rollout_corr/rollout_ppl:np.float64(1.1026978376321495) - rollout_corr/rollout_log_ppl:np.float64(0.09501861992612248) - rollout_corr/log_ppl_diff:np.float64(0.001230392274010228) - rollout_corr/log_ppl_abs_diff:np.float64(0.002099611494486453) - rollout_corr/log_ppl_diff_max:np.float64(0.024470895528793335) - rollout_corr/log_ppl_diff_min:np.float64(-0.0052408576011657715) - rollout_corr/ppl_ratio:np.float64(1.0012363716959953) - rollout_corr/chi2_token:np.float64(0.0011034025810658932) - rollout_corr/chi2_seq:np.float64(0.31427349406294525) - actor/pg_loss:np.float64(0.0014356522588059306) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005964356096228585) - actor/ppo_kl:np.float64(0.0006484976534100184) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_w_mean:np.float64(93486.41973545495) - self_distillation/token_reweight_w_std:np.float64(1284691.6428641423) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996571952942759) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02805576975515578) - self_distillation/empty_target_batch:np.float64(0.0625) - actor/grad_norm:np.float64(0.22916332073509693) - perf/mfu/actor:np.float64(0.03750177780047932) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.05967712402344) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:3 - critic/score/mean:0.859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009613416157662868 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009613416157662868 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:381.59765625 - response_length/max:1424.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:381.59765625 - response_length_non_aborted/max:1424.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.40625 - prompt_length/max:355.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.280653178691864e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.881490908563137) - timing_s/agent_loop/generate_sequences/max:np.float64(9.613876366987824) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7271649866961525) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.613876366987824) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:279 - timing_s/agent_loop/slowest/response_length:1424 - timing_s/gen:15.345683528110385 - timing_s/reward:0.0597789641469717 - timing_s/old_log_prob:2.3274049311876297 - timing_s/adv:0.6073618121445179 - timing_s/update_actor:15.772901671007276 - timing_s/step:34.20114932395518 - timing_s/stop_profile:0.00012829527258872986 - timing_per_token_ms/update_actor:0.09537542506519815 - timing_per_token_ms/adv:0.003672589369407583 - timing_per_token_ms/gen:0.15708711859175942 - perf/total_num_tokens:165377 - perf/time_per_step:34.20114932395518 - perf/throughput:604.4277870369937 (TaskRunner pid=2074004) Training Progress: 78%|███████▊ | 78/100 [1:05:28<15:01, 41.00s/it] (TaskRunner pid=2074004) step:79 - global_seqlen/min:17456 - global_seqlen/max:34104 - global_seqlen/minmax_diff:16648 - global_seqlen/balanced_min:23010 - global_seqlen/balanced_max:23387 - global_seqlen/mean:23066.5 - actor/entropy:0.10228408128023148 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4396432042121887 - training/rollout_probs_diff_mean:0.0026395104359835386 - training/rollout_probs_diff_std:0.011051760986447334 - training/rollout_actor_probs_pearson_corr:0.9977607131004333 - self_distillation/success_group_fraction:1.0 - self_distillation/success_sample_fraction:0.984375 - self_distillation/correct_sample_fraction:0.81640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.984375 - rollout_corr/rollout_is_mean:1.000108003616333 - rollout_corr/rollout_is_ratio_fraction_high:8.729964974918403e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.364982305560261e-05 - rollout_corr/rollout_is_max:2.167071580886841 - rollout_corr/rollout_is_min:0.29309991002082825 - rollout_corr/rollout_is_std:0.030481424182653427 - rollout_corr/rollout_is_eff_sample_size:0.9990721022190584 - rollout_corr/rollout_is_seq_mean:1.0001380443572998 - rollout_corr/rollout_is_seq_std:0.0018053654348477721 - rollout_corr/rollout_is_seq_max:1.0080220699310303 - rollout_corr/rollout_is_seq_min:0.9950286149978638 - rollout_corr/rollout_is_seq_max_deviation:0.008022069931030273 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1028944421559572) - rollout_corr/training_log_ppl:np.float64(0.09561884048889624) - rollout_corr/kl:np.float64(0.000491056393927991) - rollout_corr/k3_kl:np.float64(0.0006740913643881186) - rollout_corr/rollout_ppl:np.float64(1.1023467793129385) - rollout_corr/rollout_log_ppl:np.float64(0.0951277833446511) - rollout_corr/log_ppl_diff:np.float64(0.0004910571442451328) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015971105312928557) - rollout_corr/log_ppl_diff_max:np.float64(0.012345612049102783) - rollout_corr/log_ppl_diff_min:np.float64(-0.006457008421421051) - rollout_corr/ppl_ratio:np.float64(1.00049376511015) - rollout_corr/chi2_token:np.float64(0.0030803862027823925) - rollout_corr/chi2_seq:np.float64(0.8471683100797236) - actor/pg_loss:np.float64(0.0012736148200929165) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005856991260770883) - actor/ppo_kl:np.float64(0.00015690973241966333) - actor/pg_clipfrac_lower:np.float64(1.272394092666218e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.81640625) - self_distillation/token_reweight_w_mean:np.float64(105670.61742275464) - self_distillation/token_reweight_w_std:np.float64(1558575.511290349) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007708300836384) - self_distillation/token_reweight_w_clip_frac:np.float64(0.034760000635287724) - self_distillation/empty_target_batch:np.float64(0.015625) - actor/grad_norm:np.float64(0.2251501902937889) - perf/mfu/actor:np.float64(0.04172759803828769) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.39704513549805) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:3 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00025862519396468997 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00025862519396468997 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:447.453125 - response_length/max:1822.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:447.453125 - response_length_non_aborted/max:1822.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.375 - prompt_length/max:365.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.048916399478912e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.363132268190384) - timing_s/agent_loop/generate_sequences/max:np.float64(12.375673668459058) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.632285126703209) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.375673668459058) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:1822 - timing_s/gen:18.45864589139819 - timing_s/reward:0.39899917505681515 - timing_s/old_log_prob:2.5533297322690487 - timing_s/adv:0.6657597441226244 - timing_s/update_actor:15.879793219268322 - timing_s/step:37.99234522506595 - timing_s/stop_profile:0.00012224353849887848 - timing_per_token_ms/update_actor:0.08605441451492599 - timing_per_token_ms/adv:0.0036078281497118354 - timing_per_token_ms/gen:0.16114332761286265 - perf/total_num_tokens:184532 - perf/time_per_step:37.99234522506595 - perf/throughput:607.1354601395224 (TaskRunner pid=2074004) Training Progress: 79%|███████▉ | 79/100 [1:06:06<14:02, 40.11s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Solving for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substituting the known values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest answer is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70/actor (TaskRunner pid=2074004) step:80 - global_seqlen/min:15487 - global_seqlen/max:29827 - global_seqlen/minmax_diff:14340 - global_seqlen/balanced_min:22537 - global_seqlen/balanced_max:22562 - global_seqlen/mean:22553.375 - actor/entropy:0.09155089408159256 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27730071544647217 - training/rollout_probs_diff_mean:0.0024328790605068207 - training/rollout_probs_diff_std:0.010556120425462723 - training/rollout_actor_probs_pearson_corr:0.9977543950080872 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.0000025033950806 - rollout_corr/rollout_is_ratio_fraction_high:3.5952147300122306e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.4940185034647584e-05 - rollout_corr/rollout_is_max:4.466158390045166 - rollout_corr/rollout_is_min:0.25997382402420044 - rollout_corr/rollout_is_std:0.029982615262269974 - rollout_corr/rollout_is_eff_sample_size:0.9991017311505869 - rollout_corr/rollout_is_seq_mean:0.9999626278877258 - rollout_corr/rollout_is_seq_std:0.0018385304138064384 - rollout_corr/rollout_is_seq_max:1.005773901939392 - rollout_corr/rollout_is_seq_min:0.9919787049293518 - rollout_corr/rollout_is_seq_max_deviation:0.008021295070648193 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0906855990178883) - rollout_corr/training_log_ppl:np.float64(0.08496327018656302) - rollout_corr/kl:np.float64(0.0005949745295730224) - rollout_corr/k3_kl:np.float64(0.0005148513625989892) - rollout_corr/rollout_ppl:np.float64(1.0900255064480007) - rollout_corr/rollout_log_ppl:np.float64(0.08436829397760448) - rollout_corr/log_ppl_diff:np.float64(0.000594976208958542) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013135308654455002) - rollout_corr/log_ppl_diff_max:np.float64(0.009091086685657501) - rollout_corr/log_ppl_diff_min:np.float64(-0.007834956049919128) - rollout_corr/ppl_ratio:np.float64(1.00059686251916) - rollout_corr/chi2_token:np.float64(0.0010652847122401) - rollout_corr/chi2_seq:np.float64(0.4150397318881005) - actor/pg_loss:np.float64(0.0015692964661866426) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005134487364557572) - actor/ppo_kl:np.float64(7.53170088216848e-05) - actor/pg_clipfrac_lower:np.float64(5.053363565821201e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(107108.59001243045) - self_distillation/token_reweight_w_std:np.float64(1823206.4579460123) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007828855887055) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0354066689760657) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.2679561898112297) - perf/mfu/actor:np.float64(0.04098502127101053) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.34692001342773) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6734375) - val-aux/sciknoweval/reward/std@16:np.float64(0.14740819888829457) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.73165) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1155182901181198) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6184875) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.13261326513354657) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.67505) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.14789792369181032) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7746125) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.08533694637512469) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5687374999999999) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11152861009447176) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6805749999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1142760356947001) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8090125) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06002478260877771) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.52285) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09450955191647543) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6825875) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07668878228681326) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8320874999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03566959645834318) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.48012499999999997) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07047066140853427) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.67935) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.05739226575463654) - val-aux/sciknoweval/score/mean@16:np.float64(0.6734375) - val-aux/sciknoweval/score/std@16:np.float64(0.14740819888829457) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.73165) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1155182901181198) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6184875) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.13261326513354657) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.67505) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.14789792369181032) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7746125) - val-aux/sciknoweval/score/best@4/std:np.float64(0.08533694637512469) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5687374999999999) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11152861009447176) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6805749999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1142760356947001) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8090125) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06002478260877771) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.52285) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09450955191647543) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6825875) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07668878228681326) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8320874999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03566959645834318) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.48012499999999997) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07047066140853427) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.67935) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.05739226575463654) - val-core/sciknoweval/acc/mean@16:np.float64(0.6734375) - val-aux/sciknoweval/acc/std@16:np.float64(0.14740819888829457) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.73165) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1155182901181198) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6184875) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.13261326513354657) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.67505) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.14789792369181032) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7746125) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.08533694637512469) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5687374999999999) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11152861009447176) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6805749999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1142760356947001) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8090125) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06002478260877771) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.52285) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09450955191647543) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6825875) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07668878228681326) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8320874999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03566959645834318) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.48012499999999997) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07047066140853427) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.67935) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.05739226575463654) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99921875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0030257682392245445) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999375) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0008816709987291178) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9984999999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.00406201920231798) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9993124999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0028497532787945) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9971375) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.005252365538497868) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9997875) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.001615887913810856) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.995125) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.006096874199128599) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9923249999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.006085382075104241) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:3 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00442548468708992 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00442548468708992 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:434.60546875 - response_length/max:1332.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:434.60546875 - response_length_non_aborted/max:1332.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.1875 - prompt_length/max:363.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.599832653999329e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1593389865010977) - timing_s/agent_loop/generate_sequences/max:np.float64(9.577507866546512) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.48384735804575) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.577507866546512) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:1332 - timing_s/gen:15.530941212549806 - timing_s/reward:0.05138901062309742 - timing_s/old_log_prob:2.2361939437687397 - timing_s/adv:0.5929265525192022 - timing_s/update_actor:15.585156086832285 - timing_s/step:34.08298168331385 - timing_s/testing:84.83454155549407 - timing_s/save_checkpoint:6.7697754595428705 - timing_s/stop_profile:0.00011528097093105316 - timing_per_token_ms/update_actor:0.08637928961204412 - timing_per_token_ms/adv:0.0032862407096454645 - timing_per_token_ms/gen:0.1395926730650986 - perf/total_num_tokens:180427 - perf/time_per_step:34.08298168331385 - perf/throughput:661.7195411351453 (TaskRunner pid=2074004) Training Progress: 80%|████████ | 80/100 [1:08:12<21:56, 65.81s/it] (TaskRunner pid=2074004) step:81 - global_seqlen/min:17592 - global_seqlen/max:32104 - global_seqlen/minmax_diff:14512 - global_seqlen/balanced_min:22879 - global_seqlen/balanced_max:22902 - global_seqlen/mean:22894.0 - actor/entropy:0.11737882345914841 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4744115471839905 - training/rollout_probs_diff_mean:0.0028525586239993572 - training/rollout_probs_diff_std:0.01126584131270647 - training/rollout_actor_probs_pearson_corr:0.9979852437973022 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.95703125 - self_distillation/correct_sample_fraction:0.76953125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.95703125 - rollout_corr/rollout_is_mean:0.9998797178268433 - rollout_corr/rollout_is_ratio_fraction_high:8.564577001379803e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.2822885006899014e-05 - rollout_corr/rollout_is_max:2.0156476497650146 - rollout_corr/rollout_is_min:0.12281379103660583 - rollout_corr/rollout_is_std:0.03066662885248661 - rollout_corr/rollout_is_eff_sample_size:0.9990602035443582 - rollout_corr/rollout_is_seq_mean:1.000148892402649 - rollout_corr/rollout_is_seq_std:0.002365858294069767 - rollout_corr/rollout_is_seq_max:1.0138297080993652 - rollout_corr/rollout_is_seq_min:0.9926231503486633 - rollout_corr/rollout_is_seq_max_deviation:0.013829708099365234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1165403798222542) - rollout_corr/training_log_ppl:np.float64(0.10833677103437367) - rollout_corr/kl:np.float64(0.0006071828474922825) - rollout_corr/k3_kl:np.float64(0.0006022309770301604) - rollout_corr/rollout_ppl:np.float64(1.1158664338290691) - rollout_corr/rollout_log_ppl:np.float64(0.10772958788948017) - rollout_corr/log_ppl_diff:np.float64(0.0006071831448934972) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018550536478869617) - rollout_corr/log_ppl_diff_max:np.float64(0.009145308285951614) - rollout_corr/log_ppl_diff_min:np.float64(-0.012295559048652649) - rollout_corr/ppl_ratio:np.float64(1.0006107091903687) - rollout_corr/chi2_token:np.float64(0.001226232387125492) - rollout_corr/chi2_seq:np.float64(0.12261000624857843) - actor/pg_loss:np.float64(0.0018157117301598191) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004039403577280609) - actor/ppo_kl:np.float64(0.00021660448263460808) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.95703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.95703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_w_mean:np.float64(76784.49322666228) - self_distillation/token_reweight_w_std:np.float64(1652886.481738999) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002255805302411) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04571778689569328) - self_distillation/empty_target_batch:np.float64(0.04296875) - actor/grad_norm:np.float64(0.2025649379938841) - perf/mfu/actor:np.float64(0.04102161940188653) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.92991256713867) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:3 - critic/score/mean:0.76953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009963815100491047 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009963815100491047 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:456.09375 - response_length/max:1588.0 - response_length/min:61.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:456.09375 - response_length_non_aborted/max:1588.0 - response_length_non_aborted/min:61.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.34375 - prompt_length/max:460.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.460662603378296e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6343090515583754) - timing_s/agent_loop/generate_sequences/max:np.float64(10.963617943227291) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.317745556931186) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.963617943227291) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:1588 - timing_s/gen:16.757680144160986 - timing_s/reward:0.056961143389344215 - timing_s/old_log_prob:2.566490786150098 - timing_s/adv:0.6639319956302643 - timing_s/update_actor:15.944705080240965 - timing_s/step:36.09860094822943 - timing_s/stop_profile:0.00011157244443893433 - timing_per_token_ms/update_actor:0.08705722613043246 - timing_per_token_ms/adv:0.0036250327358165035 - timing_per_token_ms/gen:0.14352244042618179 - perf/total_num_tokens:183152 - perf/time_per_step:36.09860094822943 - perf/throughput:634.2074041271926 (TaskRunner pid=2074004) Training Progress: 81%|████████ | 81/100 [1:08:48<18:01, 56.91s/it] (TaskRunner pid=2074004) step:82 - global_seqlen/min:18726 - global_seqlen/max:28987 - global_seqlen/minmax_diff:10261 - global_seqlen/balanced_min:23632 - global_seqlen/balanced_max:23847 - global_seqlen/mean:23672.375 - actor/entropy:0.10426922142505646 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.506376326084137 - training/rollout_probs_diff_mean:0.0025403054896742105 - training/rollout_probs_diff_std:0.010555604472756386 - training/rollout_actor_probs_pearson_corr:0.9980065822601318 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:1.0000488758087158 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.208931204630062e-05 - rollout_corr/rollout_is_max:1.914077877998352 - rollout_corr/rollout_is_min:0.227353036403656 - rollout_corr/rollout_is_std:0.02918272092938423 - rollout_corr/rollout_is_eff_sample_size:0.999149212500466 - rollout_corr/rollout_is_seq_mean:0.9999838471412659 - rollout_corr/rollout_is_seq_std:0.0015772389015182853 - rollout_corr/rollout_is_seq_max:1.0057252645492554 - rollout_corr/rollout_is_seq_min:0.9941803216934204 - rollout_corr/rollout_is_seq_max_deviation:0.00581967830657959 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.109215888660401) - rollout_corr/training_log_ppl:np.float64(0.10109948862009333) - rollout_corr/kl:np.float64(0.0005538763568267713) - rollout_corr/k3_kl:np.float64(0.0005117594077042042) - rollout_corr/rollout_ppl:np.float64(1.1085950387641788) - rollout_corr/rollout_log_ppl:np.float64(0.10054561170181842) - rollout_corr/log_ppl_diff:np.float64(0.0005538769182749093) - rollout_corr/log_ppl_abs_diff:np.float64(0.001272055131266825) - rollout_corr/log_ppl_diff_max:np.float64(0.006336919963359833) - rollout_corr/log_ppl_diff_min:np.float64(-0.004927821457386017) - rollout_corr/ppl_ratio:np.float64(1.000555366044864) - rollout_corr/chi2_token:np.float64(0.0009450593497604132) - rollout_corr/chi2_seq:np.float64(1.0984282961580902) - actor/pg_loss:np.float64(0.0019886784721165895) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003411252544083254) - actor/ppo_kl:np.float64(9.175821236606652e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(50298.95451163477) - self_distillation/token_reweight_w_std:np.float64(977247.3528398918) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999259575270116) - self_distillation/token_reweight_w_clip_frac:np.float64(0.038464386205305345) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.25309666618704796) - perf/mfu/actor:np.float64(0.04338852253393083) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.42945861816406) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:3 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00773917231708765 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00773917231708765 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:464.04296875 - response_length/max:1880.0 - response_length/min:155.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:464.04296875 - response_length_non_aborted/max:1880.0 - response_length_non_aborted/min:155.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:275.71875 - prompt_length/max:382.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001201629638671875 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6977196279913187) - timing_s/agent_loop/generate_sequences/max:np.float64(12.517238644883037) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.792556140542729) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.517238644883037) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:349 - timing_s/agent_loop/slowest/response_length:1880 - timing_s/gen:18.328095687553287 - timing_s/reward:0.06305506825447083 - timing_s/old_log_prob:2.362060610204935 - timing_s/adv:0.6524072252213955 - timing_s/update_actor:15.651612892746925 - timing_s/step:37.145222717896104 - timing_s/stop_profile:4.989840090274811e-05 - timing_per_token_ms/update_actor:0.08264703527184601 - timing_per_token_ms/adv:0.0034449818893403992 - timing_per_token_ms/gen:0.1542833931356815 - perf/total_num_tokens:189379 - perf/time_per_step:37.145222717896104 - perf/throughput:637.2925848306987 (TaskRunner pid=2074004) Training Progress: 82%|████████▏ | 82/100 [1:09:25<15:17, 50.99s/it] (TaskRunner pid=2074004) step:83 - global_seqlen/min:16528 - global_seqlen/max:26927 - global_seqlen/minmax_diff:10399 - global_seqlen/balanced_min:21714 - global_seqlen/balanced_max:21743 - global_seqlen/mean:21736.25 - actor/entropy:0.1269112080335617 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6248966455459595 - training/rollout_probs_diff_mean:0.0030177931766957045 - training/rollout_probs_diff_std:0.011725279502570629 - training/rollout_actor_probs_pearson_corr:0.9978418350219727 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.7421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:0.9999353289604187 - rollout_corr/rollout_is_ratio_fraction_high:3.8849284464959055e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.769856892991811e-05 - rollout_corr/rollout_is_max:2.3085124492645264 - rollout_corr/rollout_is_min:0.010780704207718372 - rollout_corr/rollout_is_std:0.03224320337176323 - rollout_corr/rollout_is_eff_sample_size:0.9989612175582933 - rollout_corr/rollout_is_seq_mean:0.9999208450317383 - rollout_corr/rollout_is_seq_std:0.001950851990841329 - rollout_corr/rollout_is_seq_max:1.0089315176010132 - rollout_corr/rollout_is_seq_min:0.9943766593933105 - rollout_corr/rollout_is_seq_max_deviation:0.008931517601013184 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1371404435485601) - rollout_corr/training_log_ppl:np.float64(0.12448573324945755) - rollout_corr/kl:np.float64(0.0008972727727523111) - rollout_corr/k3_kl:np.float64(0.0008475244219425804) - rollout_corr/rollout_ppl:np.float64(1.1360785402357578) - rollout_corr/rollout_log_ppl:np.float64(0.12358846047573024) - rollout_corr/log_ppl_diff:np.float64(0.000897272773727309) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018847366955014877) - rollout_corr/log_ppl_diff_max:np.float64(0.01430445909500122) - rollout_corr/log_ppl_diff_min:np.float64(-0.007731936872005463) - rollout_corr/ppl_ratio:np.float64(1.0009009479545057) - rollout_corr/chi2_token:np.float64(0.001457967096939683) - rollout_corr/chi2_seq:np.float64(0.5747340724337846) - actor/pg_loss:np.float64(0.00172864377964288) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005559734236157965) - actor/ppo_kl:np.float64(0.0002423281556467316) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_w_mean:np.float64(34890.77068122127) - self_distillation/token_reweight_w_std:np.float64(689731.6690365797) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998807616066188) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03181662493443582) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.24900896847248077) - perf/mfu/actor:np.float64(0.039740719700471376) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.4448585510254) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:3 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0005997358239255846 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0005997358239255846 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:402.1953125 - response_length/max:1439.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:402.1953125 - response_length_non_aborted/max:1439.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.0625 - prompt_length/max:375.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.76057893037796e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6644078604876995) - timing_s/agent_loop/generate_sequences/max:np.float64(10.04941943101585) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.103168618654308) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.04941943101585) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:346 - timing_s/agent_loop/slowest/response_length:1439 - timing_s/gen:15.86747464723885 - timing_s/reward:0.06120395101606846 - timing_s/old_log_prob:2.3169026523828506 - timing_s/adv:0.5539372637867928 - timing_s/update_actor:15.275738924741745 - timing_s/step:34.1625665910542 - timing_s/stop_profile:3.450922667980194e-05 - timing_per_token_ms/update_actor:0.08784713856312465 - timing_per_token_ms/adv:0.0031855613536534173 - timing_per_token_ms/gen:0.1541100080344093 - perf/total_num_tokens:173890 - perf/time_per_step:34.1625665910542 - perf/throughput:636.2592793508626 (TaskRunner pid=2074004) Training Progress: 83%|████████▎ | 83/100 [1:09:59<13:01, 45.94s/it] (TaskRunner pid=2074004) step:84 - global_seqlen/min:18342 - global_seqlen/max:28767 - global_seqlen/minmax_diff:10425 - global_seqlen/balanced_min:22295 - global_seqlen/balanced_max:22560 - global_seqlen/mean:22373.75 - actor/entropy:0.11953727900981903 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016381561756134 - training/rollout_probs_diff_mean:0.0028403468895703554 - training/rollout_probs_diff_std:0.010894829407334328 - training/rollout_actor_probs_pearson_corr:0.9980471134185791 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.0000654458999634 - rollout_corr/rollout_is_ratio_fraction_high:8.987794899439905e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.6963383788825013e-05 - rollout_corr/rollout_is_max:2.1378886699676514 - rollout_corr/rollout_is_min:0.1871485859155655 - rollout_corr/rollout_is_std:0.030487289652228355 - rollout_corr/rollout_is_eff_sample_size:0.9990715072785922 - rollout_corr/rollout_is_seq_mean:1.0000371932983398 - rollout_corr/rollout_is_seq_std:0.0019689607433974743 - rollout_corr/rollout_is_seq_max:1.0071165561676025 - rollout_corr/rollout_is_seq_min:0.9907252788543701 - rollout_corr/rollout_is_seq_max_deviation:0.009274721145629883 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1271462766453624) - rollout_corr/training_log_ppl:np.float64(0.11585301743616583) - rollout_corr/kl:np.float64(0.0005694015161132882) - rollout_corr/k3_kl:np.float64(0.0006418242667365348) - rollout_corr/rollout_ppl:np.float64(1.1264640409499407) - rollout_corr/rollout_log_ppl:np.float64(0.11528361437740386) - rollout_corr/log_ppl_diff:np.float64(0.0005694030587619636) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015966345054039266) - rollout_corr/log_ppl_diff_max:np.float64(0.010501980781555176) - rollout_corr/log_ppl_diff_min:np.float64(-0.006051577627658844) - rollout_corr/ppl_ratio:np.float64(1.0005718814209104) - rollout_corr/chi2_token:np.float64(0.0016377929132431746) - rollout_corr/chi2_seq:np.float64(0.6639628079719841) - actor/pg_loss:np.float64(0.002379689714871347) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000606918740686524) - actor/ppo_kl:np.float64(9.888500281718393e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(92119.91488653235) - self_distillation/token_reweight_w_std:np.float64(1798317.0580325122) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002044991124421) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05570600633654976) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.29781750962138176) - perf/mfu/actor:np.float64(0.040360571878840845) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.56901168823242) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:3 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0005010695313103497 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0005010695313103497 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:434.6171875 - response_length/max:2171.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:434.6171875 - response_length_non_aborted/max:2171.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.5625 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.546158015727997e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7891156543046236) - timing_s/agent_loop/generate_sequences/max:np.float64(13.379426375031471) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.27586570179119) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.379426375031471) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:2171 - timing_s/gen:19.16517908498645 - timing_s/reward:0.06223759055137634 - timing_s/old_log_prob:2.38674658536911 - timing_s/adv:0.6694579813629389 - timing_s/update_actor:15.630993586033583 - timing_s/step:37.9998825173825 - timing_s/stop_profile:3.123842179775238e-05 - timing_per_token_ms/update_actor:0.08732886522170838 - timing_per_token_ms/adv:0.003740197672288613 - timing_per_token_ms/gen:0.17225269260831594 - perf/total_num_tokens:178990 - perf/time_per_step:37.9998825173825 - perf/throughput:588.7847150518281 (TaskRunner pid=2074004) Training Progress: 84%|████████▍ | 84/100 [1:10:37<11:37, 43.57s/it] (TaskRunner pid=2074004) step:85 - global_seqlen/min:13444 - global_seqlen/max:29698 - global_seqlen/minmax_diff:16254 - global_seqlen/balanced_min:21579 - global_seqlen/balanced_max:21625 - global_seqlen/mean:21607.0 - actor/entropy:0.1099928542971611 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7565039396286011 - training/rollout_probs_diff_mean:0.002620653947815299 - training/rollout_probs_diff_std:0.011461569927632809 - training/rollout_actor_probs_pearson_corr:0.9977118968963623 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.72265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9998971819877625 - rollout_corr/rollout_is_ratio_fraction_high:9.451795449422207e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.00010396975267212838 - rollout_corr/rollout_is_max:2.1670234203338623 - rollout_corr/rollout_is_min:0.16362474858760834 - rollout_corr/rollout_is_std:0.030536126345396042 - rollout_corr/rollout_is_eff_sample_size:0.999068294612318 - rollout_corr/rollout_is_seq_mean:1.0000232458114624 - rollout_corr/rollout_is_seq_std:0.0025593643076717854 - rollout_corr/rollout_is_seq_max:1.008395791053772 - rollout_corr/rollout_is_seq_min:0.9886937141418457 - rollout_corr/rollout_is_seq_max_deviation:0.011306285858154297 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1031886860728264) - rollout_corr/training_log_ppl:np.float64(0.09535550630971557) - rollout_corr/kl:np.float64(0.000747708574247774) - rollout_corr/k3_kl:np.float64(0.0006307248504953122) - rollout_corr/rollout_ppl:np.float64(1.1023486168123782) - rollout_corr/rollout_log_ppl:np.float64(0.09460779764049221) - rollout_corr/log_ppl_diff:np.float64(0.000747708669223357) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018803431230480783) - rollout_corr/log_ppl_diff_max:np.float64(0.013932567089796066) - rollout_corr/log_ppl_diff_min:np.float64(-0.007006004452705383) - rollout_corr/ppl_ratio:np.float64(1.0007511703297496) - rollout_corr/chi2_token:np.float64(0.000999337062239647) - rollout_corr/chi2_seq:np.float64(0.17764197965152562) - actor/pg_loss:np.float64(0.001810003537684679) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005623773683964828) - actor/ppo_kl:np.float64(0.000197975718934984) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.72265625) - self_distillation/token_reweight_w_mean:np.float64(103010.1768341891) - self_distillation/token_reweight_w_std:np.float64(1687668.886640767) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000071118120104) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03546456596814096) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.2238437756896019) - perf/mfu/actor:np.float64(0.03919303431293418) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.48588943481445) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:3 - critic/score/mean:0.72265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.72265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0027670131530612707 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0027670131530612707 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:413.28125 - response_length/max:1808.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:413.28125 - response_length_non_aborted/max:1808.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.9375 - prompt_length/max:365.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.203617572784424e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7867520544677973) - timing_s/agent_loop/generate_sequences/max:np.float64(11.670810116454959) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.985955903917784) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.670810116454959) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:1808 - timing_s/gen:17.77034630998969 - timing_s/reward:0.08232808858156204 - timing_s/old_log_prob:2.3802652321755886 - timing_s/adv:0.6049736253917217 - timing_s/update_actor:15.622556304559112 - timing_s/step:36.56155405379832 - timing_s/stop_profile:5.652382969856262e-05 - timing_per_token_ms/update_actor:0.09037902244966395 - timing_per_token_ms/adv:0.003499870559261592 - timing_per_token_ms/gen:0.167961685349619 - perf/total_num_tokens:172856 - perf/time_per_step:36.56155405379832 - perf/throughput:590.9759735104938 (TaskRunner pid=2074004) Training Progress: 85%|████████▌ | 85/100 [1:11:14<10:22, 41.47s/it] (TaskRunner pid=2074004) step:86 - global_seqlen/min:17265 - global_seqlen/max:27278 - global_seqlen/minmax_diff:10013 - global_seqlen/balanced_min:22005 - global_seqlen/balanced_max:22347 - global_seqlen/mean:22060.875 - actor/entropy:0.1164645105600357 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5546000003814697 - training/rollout_probs_diff_mean:0.0027262147050350904 - training/rollout_probs_diff_std:0.011048424988985062 - training/rollout_actor_probs_pearson_corr:0.9979538917541504 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:0.9999389052391052 - rollout_corr/rollout_is_ratio_fraction_high:2.80670246866066e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.6778372052358463e-05 - rollout_corr/rollout_is_max:2.691387176513672 - rollout_corr/rollout_is_min:0.28650444746017456 - rollout_corr/rollout_is_std:0.030255712568759918 - rollout_corr/rollout_is_eff_sample_size:0.9990854290712073 - rollout_corr/rollout_is_seq_mean:1.0000141859054565 - rollout_corr/rollout_is_seq_std:0.001993668731302023 - rollout_corr/rollout_is_seq_max:1.0104111433029175 - rollout_corr/rollout_is_seq_min:0.9925797581672668 - rollout_corr/rollout_is_seq_max_deviation:0.01041114330291748 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.118015536107123) - rollout_corr/training_log_ppl:np.float64(0.10848677079411573) - rollout_corr/kl:np.float64(0.0006072961230856322) - rollout_corr/k3_kl:np.float64(0.0005712031893807534) - rollout_corr/rollout_ppl:np.float64(1.1173007288016379) - rollout_corr/rollout_log_ppl:np.float64(0.10787947439894197) - rollout_corr/log_ppl_diff:np.float64(0.000607296395173762) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015548389492323622) - rollout_corr/log_ppl_diff_max:np.float64(0.008465766906738281) - rollout_corr/log_ppl_diff_min:np.float64(-0.008714862167835236) - rollout_corr/ppl_ratio:np.float64(1.0006095981225371) - rollout_corr/chi2_token:np.float64(0.0010650106705725193) - rollout_corr/chi2_seq:np.float64(0.39449123083613813) - actor/pg_loss:np.float64(0.000594716053456068) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002205379737461044) - actor/ppo_kl:np.float64(0.00013570546891372715) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(10468.989274371881) - self_distillation/token_reweight_w_std:np.float64(246581.90501564916) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998316585551947) - self_distillation/token_reweight_w_clip_frac:np.float64(0.015069760556798428) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.09993577841669321) - perf/mfu/actor:np.float64(0.04011440240529307) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.5170135498047) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:3 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0022991569712758064 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0022991569712758064 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:417.52734375 - response_length/max:1781.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:417.52734375 - response_length_non_aborted/max:1781.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.875 - prompt_length/max:365.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.181055843830109e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.044587777927518) - timing_s/agent_loop/generate_sequences/max:np.float64(11.397420486435294) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.3413470640807645) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.397420486435294) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:1781 - timing_s/gen:17.311374351382256 - timing_s/reward:0.062451690435409546 - timing_s/old_log_prob:2.356656324118376 - timing_s/adv:0.579136572778225 - timing_s/update_actor:15.45087962038815 - timing_s/step:35.84615935757756 - timing_s/stop_profile:0.00011509284377098083 - timing_per_token_ms/update_actor:0.08754684265916556 - timing_per_token_ms/adv:0.0032814687358175103 - timing_per_token_ms/gen:0.16195958677278113 - perf/total_num_tokens:176487 - perf/time_per_step:35.84615935757756 - perf/throughput:615.4320405691252 (TaskRunner pid=2074004) Training Progress: 86%|████████▌ | 86/100 [1:11:50<09:17, 39.80s/it] (TaskRunner pid=2074004) step:87 - global_seqlen/min:18840 - global_seqlen/max:25215 - global_seqlen/minmax_diff:6375 - global_seqlen/balanced_min:22425 - global_seqlen/balanced_max:22454 - global_seqlen/mean:22447.0 - actor/entropy:0.12408103793859482 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.33947664499282837 - training/rollout_probs_diff_mean:0.0029533642809838057 - training/rollout_probs_diff_std:0.011205787770450115 - training/rollout_actor_probs_pearson_corr:0.9980510473251343 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.0000004768371582 - rollout_corr/rollout_is_ratio_fraction_high:9.21557057154132e-06 - rollout_corr/rollout_is_ratio_fraction_low:7.372456457233056e-05 - rollout_corr/rollout_is_max:2.10186767578125 - rollout_corr/rollout_is_min:0.05793754756450653 - rollout_corr/rollout_is_std:0.03159523010253906 - rollout_corr/rollout_is_eff_sample_size:0.9990027369357228 - rollout_corr/rollout_is_seq_mean:1.000128984451294 - rollout_corr/rollout_is_seq_std:0.002129511907696724 - rollout_corr/rollout_is_seq_max:1.0119233131408691 - rollout_corr/rollout_is_seq_min:0.9937775135040283 - rollout_corr/rollout_is_seq_max_deviation:0.01192331314086914 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1291866833344102) - rollout_corr/training_log_ppl:np.float64(0.11755550177258556) - rollout_corr/kl:np.float64(0.0004603647517873455) - rollout_corr/k3_kl:np.float64(0.0006310233444892788) - rollout_corr/rollout_ppl:np.float64(1.1286979308351874) - rollout_corr/rollout_log_ppl:np.float64(0.11709513665482518) - rollout_corr/log_ppl_diff:np.float64(0.00046036511776037514) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016326033364748582) - rollout_corr/log_ppl_diff_max:np.float64(0.007639244198799133) - rollout_corr/log_ppl_diff_min:np.float64(-0.010535374283790588) - rollout_corr/ppl_ratio:np.float64(1.0004629627801478) - rollout_corr/chi2_token:np.float64(0.0016186770517379045) - rollout_corr/chi2_seq:np.float64(0.7282668207772076) - actor/pg_loss:np.float64(0.0011052467161789536) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023867743630034965) - actor/ppo_kl:np.float64(1.1898970175394652e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_w_mean:np.float64(55895.11731084273) - self_distillation/token_reweight_w_std:np.float64(747223.8999097699) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997360575944185) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02877963568607811) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.2662094905972481) - perf/mfu/actor:np.float64(0.04090594702599697) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.4480094909668) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:3 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0027278088964521885 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0027278088964521885 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:423.875 - response_length/max:1575.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:423.875 - response_length_non_aborted/max:1575.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.59375 - prompt_length/max:364.0 - prompt_length/min:219.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011235661804676056 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7568011898547411) - timing_s/agent_loop/generate_sequences/max:np.float64(10.530228452757001) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.055000598782499) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.530228452757001) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:239 - timing_s/agent_loop/slowest/response_length:1575 - timing_s/gen:16.49220446869731 - timing_s/reward:0.06013626419007778 - timing_s/old_log_prob:2.299874674528837 - timing_s/adv:0.572662265971303 - timing_s/update_actor:15.511149242520332 - timing_s/step:34.9925875030458 - timing_s/stop_profile:0.00011404044926166534 - timing_per_token_ms/update_actor:0.0863765160295381 - timing_per_token_ms/adv:0.0031889688264094477 - timing_per_token_ms/gen:0.1519850750948956 - perf/total_num_tokens:179576 - perf/time_per_step:34.9925875030458 - perf/throughput:641.4787131144899 (TaskRunner pid=2074004) Training Progress: 87%|████████▋ | 87/100 [1:12:25<08:18, 38.37s/it] (TaskRunner pid=2074004) step:88 - global_seqlen/min:17924 - global_seqlen/max:32180 - global_seqlen/minmax_diff:14256 - global_seqlen/balanced_min:22814 - global_seqlen/balanced_max:22832 - global_seqlen/mean:22825.125 - actor/entropy:0.10808072239160538 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2607862949371338 - training/rollout_probs_diff_mean:0.0027171641122549772 - training/rollout_probs_diff_std:0.010765996761620045 - training/rollout_actor_probs_pearson_corr:0.9980359077453613 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9296875 - self_distillation/correct_sample_fraction:0.75390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9296875 - rollout_corr/rollout_is_mean:0.9999313950538635 - rollout_corr/rollout_is_ratio_fraction_high:1.7341692000627518e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.3354230001568794e-05 - rollout_corr/rollout_is_max:3.181804656982422 - rollout_corr/rollout_is_min:0.01616358757019043 - rollout_corr/rollout_is_std:0.030833328142762184 - rollout_corr/rollout_is_eff_sample_size:0.9990502088766346 - rollout_corr/rollout_is_seq_mean:0.9999111890792847 - rollout_corr/rollout_is_seq_std:0.001777790836058557 - rollout_corr/rollout_is_seq_max:1.0082212686538696 - rollout_corr/rollout_is_seq_min:0.9945080876350403 - rollout_corr/rollout_is_seq_max_deviation:0.008221268653869629 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.112278978805989) - rollout_corr/training_log_ppl:np.float64(0.10384648678154917) - rollout_corr/kl:np.float64(0.0007002655332808416) - rollout_corr/k3_kl:np.float64(0.0007456282953626214) - rollout_corr/rollout_ppl:np.float64(1.111463341396302) - rollout_corr/rollout_log_ppl:np.float64(0.10314622065925505) - rollout_corr/log_ppl_diff:np.float64(0.0007002661222941242) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017377006079186685) - rollout_corr/log_ppl_diff_max:np.float64(0.015303656458854675) - rollout_corr/log_ppl_diff_min:np.float64(-0.006968945264816284) - rollout_corr/ppl_ratio:np.float64(1.0007034207228571) - rollout_corr/chi2_token:np.float64(0.0014822578523308039) - rollout_corr/chi2_seq:np.float64(1.0283904024399817) - actor/pg_loss:np.float64(0.0028022772166877985) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011713389812939567) - actor/ppo_kl:np.float64(6.31784731428553e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75390625) - self_distillation/token_reweight_w_mean:np.float64(102159.92961579212) - self_distillation/token_reweight_w_std:np.float64(1634106.4369794251) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001399826724082) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04912651109043509) - self_distillation/empty_target_batch:np.float64(0.0703125) - actor/grad_norm:np.float64(0.34347517415881157) - perf/mfu/actor:np.float64(0.041204584153654134) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.4816131591797) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:3 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009243122301995754 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009243122301995754 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:450.50390625 - response_length/max:1537.0 - response_length/min:72.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:450.50390625 - response_length_non_aborted/max:1537.0 - response_length_non_aborted/min:72.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.78125 - prompt_length/max:375.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010040774941444397 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8878467436879873) - timing_s/agent_loop/generate_sequences/max:np.float64(10.596595136448741) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.400526633304253) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.596595136448741) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:1537 - timing_s/gen:16.373642224818468 - timing_s/reward:0.06224523112177849 - timing_s/old_log_prob:2.349311839789152 - timing_s/adv:0.6097414214164019 - timing_s/update_actor:15.592103796079755 - timing_s/step:35.07431404106319 - timing_s/stop_profile:0.00012129172682762146 - timing_per_token_ms/update_actor:0.08538892884529524 - timing_per_token_ms/adv:0.003339200888365353 - timing_per_token_ms/gen:0.1419733304270259 - perf/total_num_tokens:182601 - perf/time_per_step:35.07431404106319 - perf/throughput:650.764687037857 (TaskRunner pid=2074004) Training Progress: 88%|████████▊ | 88/100 [1:13:00<07:28, 37.40s/it] (TaskRunner pid=2074004) step:89 - global_seqlen/min:13752 - global_seqlen/max:27817 - global_seqlen/minmax_diff:14065 - global_seqlen/balanced_min:22301 - global_seqlen/balanced_max:22323 - global_seqlen/mean:22317.25 - actor/entropy:0.10688827931880951 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.601607620716095 - training/rollout_probs_diff_mean:0.002714034402742982 - training/rollout_probs_diff_std:0.011373176239430904 - training/rollout_actor_probs_pearson_corr:0.9978145360946655 - self_distillation/success_group_fraction:1.0 - self_distillation/success_sample_fraction:1.0 - self_distillation/correct_sample_fraction:0.8125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9999041557312012 - rollout_corr/rollout_is_ratio_fraction_high:3.661863593151793e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.239193266490474e-05 - rollout_corr/rollout_is_max:2.066577672958374 - rollout_corr/rollout_is_min:0.30741891264915466 - rollout_corr/rollout_is_std:0.030931759625673294 - rollout_corr/rollout_is_eff_sample_size:0.9990437838386964 - rollout_corr/rollout_is_seq_mean:0.9999012351036072 - rollout_corr/rollout_is_seq_std:0.00172822130843997 - rollout_corr/rollout_is_seq_max:1.0071568489074707 - rollout_corr/rollout_is_seq_min:0.9942929744720459 - rollout_corr/rollout_is_seq_max_deviation:0.007156848907470703 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1084096948616207) - rollout_corr/training_log_ppl:np.float64(0.09965023837503395) - rollout_corr/kl:np.float64(0.0006493180134174992) - rollout_corr/k3_kl:np.float64(0.0006724119927685734) - rollout_corr/rollout_ppl:np.float64(1.107654474209994) - rollout_corr/rollout_log_ppl:np.float64(0.09900091987219639) - rollout_corr/log_ppl_diff:np.float64(0.0006493185028375592) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017076216572604608) - rollout_corr/log_ppl_diff_max:np.float64(0.01190245896577835) - rollout_corr/log_ppl_diff_min:np.float64(-0.013669520616531372) - rollout_corr/ppl_ratio:np.float64(1.0006527956575155) - rollout_corr/chi2_token:np.float64(0.0013369775842875242) - rollout_corr/chi2_seq:np.float64(5.031164270127192) - actor/pg_loss:np.float64(0.002956106443889439) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008950727274168457) - actor/ppo_kl:np.float64(4.916048081948787e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_w_mean:np.float64(71424.80499303155) - self_distillation/token_reweight_w_std:np.float64(1224926.4624935077) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000664854887873) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04491751454042969) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.3388006314635277) - perf/mfu/actor:np.float64(0.04076402686351015) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.86504364013672) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:4 - critic/score/mean:0.8125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008070976473391056 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.008070976473391056 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:426.6953125 - response_length/max:1575.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:426.6953125 - response_length_non_aborted/max:1575.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003277640789747238 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9218454416841269) - timing_s/agent_loop/generate_sequences/max:np.float64(10.495202466845512) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.246527711606177) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.495202466845512) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1575 - timing_s/gen:16.446799524128437 - timing_s/reward:0.056499673053622246 - timing_s/old_log_prob:2.5402090717107058 - timing_s/adv:0.6734065376222134 - timing_s/update_actor:15.645092817023396 - timing_s/step:35.48852607421577 - timing_s/stop_profile:0.00012188777327537537 - timing_per_token_ms/update_actor:0.08762892390988695 - timing_per_token_ms/adv:0.003771782688403664 - timing_per_token_ms/gen:0.15056483809188015 - perf/total_num_tokens:178538 - perf/time_per_step:35.48852607421577 - perf/throughput:628.8581823130328 (TaskRunner pid=2074004) Training Progress: 89%|████████▉ | 89/100 [1:13:36<06:46, 36.99s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. We are to find the charge $ q $ using the formula: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Rearranging for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substituting the given values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest answer is **B: 56 pC**. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80/actor (TaskRunner pid=2074004) step:90 - global_seqlen/min:17503 - global_seqlen/max:30542 - global_seqlen/minmax_diff:13039 - global_seqlen/balanced_min:22163 - global_seqlen/balanced_max:22749 - global_seqlen/mean:22249.5 - actor/entropy:0.10518425703048706 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40461307764053345 - training/rollout_probs_diff_mean:0.0026247852947562933 - training/rollout_probs_diff_std:0.010737664066255093 - training/rollout_actor_probs_pearson_corr:0.9979662895202637 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:1.0000884532928467 - rollout_corr/rollout_is_ratio_fraction_high:1.823819002311211e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.20718560134992e-05 - rollout_corr/rollout_is_max:2.0903313159942627 - rollout_corr/rollout_is_min:0.19070646166801453 - rollout_corr/rollout_is_std:0.029787156730890274 - rollout_corr/rollout_is_eff_sample_size:0.9991136308144901 - rollout_corr/rollout_is_seq_mean:1.0000239610671997 - rollout_corr/rollout_is_seq_std:0.00162831402849406 - rollout_corr/rollout_is_seq_max:1.0044366121292114 - rollout_corr/rollout_is_seq_min:0.9923098683357239 - rollout_corr/rollout_is_seq_max_deviation:0.007690131664276123 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.102782507892698) - rollout_corr/training_log_ppl:np.float64(0.09468803418349125) - rollout_corr/kl:np.float64(0.0005543236077372171) - rollout_corr/k3_kl:np.float64(0.0005838268394882107) - rollout_corr/rollout_ppl:np.float64(1.1021574116311967) - rollout_corr/rollout_log_ppl:np.float64(0.0941337100121018) - rollout_corr/log_ppl_diff:np.float64(0.0005543241713894531) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015017515033832751) - rollout_corr/log_ppl_diff_max:np.float64(0.010810032486915588) - rollout_corr/log_ppl_diff_min:np.float64(-0.006302401423454285) - rollout_corr/ppl_ratio:np.float64(1.0005565818864852) - rollout_corr/chi2_token:np.float64(0.0012237117625772953) - rollout_corr/chi2_seq:np.float64(0.7812420942354947) - actor/pg_loss:np.float64(0.001676898100413382) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036616270745071233) - actor/ppo_kl:np.float64(0.00012324695833854094) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.796875) - self_distillation/token_reweight_w_mean:np.float64(32326.37215174688) - self_distillation/token_reweight_w_std:np.float64(704860.9180972856) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991597740445286) - self_distillation/token_reweight_w_clip_frac:np.float64(0.029383839399088174) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.14125302247703075) - perf/mfu/actor:np.float64(0.0403704412148885) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.92715072631836) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6921875) - val-aux/sciknoweval/reward/std@16:np.float64(0.1468032056932851) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7523500000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.10240961009878324) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6337875) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1445766161320927) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6938375000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1469724185844502) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.790025) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.06162330599483268) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5755125000000001) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1281247414410848) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7121125000000001) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.11456407227764583) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8124499999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.030947104305230778) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5231125000000001) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10100008417206525) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7222500000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.08294761183043528) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.821725) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.011989192674950796) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4808) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.0733318252008867) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.729125) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.060664540093849084) - val-aux/sciknoweval/score/mean@16:np.float64(0.6921875) - val-aux/sciknoweval/score/std@16:np.float64(0.1468032056932851) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7523500000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.10240961009878324) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6337875) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1445766161320927) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6938375000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1469724185844502) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.790025) - val-aux/sciknoweval/score/best@4/std:np.float64(0.06162330599483268) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5755125000000001) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1281247414410848) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7121125000000001) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.11456407227764583) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8124499999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.030947104305230778) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5231125000000001) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10100008417206525) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7222500000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.08294761183043528) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.821725) - val-aux/sciknoweval/score/best@16/std:np.float64(0.011989192674950796) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4808) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.0733318252008867) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.729125) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.060664540093849084) - val-core/sciknoweval/acc/mean@16:np.float64(0.6921875) - val-aux/sciknoweval/acc/std@16:np.float64(0.1468032056932851) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7523500000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.10240961009878324) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6337875) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1445766161320927) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6938375000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1469724185844502) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.790025) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.06162330599483268) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5755125000000001) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1281247414410848) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7121125000000001) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.11456407227764583) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8124499999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.030947104305230778) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5231125000000001) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10100008417206525) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7222500000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.08294761183043528) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.821725) - val-core/sciknoweval/acc/best@16/std:np.float64(0.011989192674950796) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4808) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.0733318252008867) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.729125) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.060664540093849084) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99765625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.009077304717673634) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9998374999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.002405465425136726) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9957625) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.011852098487061466) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9977625) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.008832748449106717) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9916125000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.015594544271371125) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9995750000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.003952916592698636) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9849375) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.018366765656865063) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.999925) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0013474443894951921) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9760875) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0180217269071149) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:4 - critic/score/mean:0.796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.018307723104953766 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.018307723104953766 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:428.359375 - response_length/max:2385.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:428.359375 - response_length_non_aborted/max:2385.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.9375 - prompt_length/max:437.0 - prompt_length/min:210.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017513521015644073 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2588027901947498) - timing_s/agent_loop/generate_sequences/max:np.float64(14.54117732308805) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.3901306394909625) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.54117732308805) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:265 - timing_s/agent_loop/slowest/response_length:2385 - timing_s/gen:21.131900370121002 - timing_s/reward:0.05953768454492092 - timing_s/old_log_prob:2.3052819333970547 - timing_s/adv:0.5906773693859577 - timing_s/update_actor:15.686287306249142 - timing_s/step:39.856287997215986 - timing_s/testing:87.98717088066041 - timing_s/save_checkpoint:6.872611632570624 - timing_s/stop_profile:4.615262150764465e-05 - timing_per_token_ms/update_actor:0.08812718997196084 - timing_per_token_ms/adv:0.0033184867602977465 - timing_per_token_ms/gen:0.19270381515703996 - perf/total_num_tokens:177996 - perf/time_per_step:39.856287997215986 - perf/throughput:558.24315604991 (TaskRunner pid=2074004) Training Progress: 90%|█████████ | 90/100 [1:15:51<11:03, 66.32s/it] (TaskRunner pid=2074004) step:91 - global_seqlen/min:17728 - global_seqlen/max:25869 - global_seqlen/minmax_diff:8141 - global_seqlen/balanced_min:21525 - global_seqlen/balanced_max:21553 - global_seqlen/mean:21545.25 - actor/entropy:0.08843715488910675 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.41412943601608276 - training/rollout_probs_diff_mean:0.002419212367385626 - training/rollout_probs_diff_std:0.010600846260786057 - training/rollout_actor_probs_pearson_corr:0.9976838231086731 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.92578125 - self_distillation/correct_sample_fraction:0.83984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.92578125 - rollout_corr/rollout_is_mean:0.9999130368232727 - rollout_corr/rollout_is_ratio_fraction_high:9.281086931878235e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.712434772751294e-05 - rollout_corr/rollout_is_max:2.0842576026916504 - rollout_corr/rollout_is_min:0.18285298347473145 - rollout_corr/rollout_is_std:0.027998575940728188 - rollout_corr/rollout_is_eff_sample_size:0.999216455716407 - rollout_corr/rollout_is_seq_mean:0.9998674392700195 - rollout_corr/rollout_is_seq_std:0.0018235978204756975 - rollout_corr/rollout_is_seq_max:1.007836103439331 - rollout_corr/rollout_is_seq_min:0.9920446276664734 - rollout_corr/rollout_is_seq_max_deviation:0.007955372333526611 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0833140267059207) - rollout_corr/training_log_ppl:np.float64(0.07899068885672023) - rollout_corr/kl:np.float64(0.0006408023299613319) - rollout_corr/k3_kl:np.float64(0.00047720418808694376) - rollout_corr/rollout_ppl:np.float64(1.0826310594566166) - rollout_corr/rollout_log_ppl:np.float64(0.07834988591639558) - rollout_corr/log_ppl_diff:np.float64(0.0006408029403246474) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014788467706239317) - rollout_corr/log_ppl_diff_max:np.float64(0.01340678334236145) - rollout_corr/log_ppl_diff_min:np.float64(-0.006584286689758301) - rollout_corr/ppl_ratio:np.float64(1.0006434579845518) - rollout_corr/chi2_token:np.float64(0.0005903118290007114) - rollout_corr/chi2_seq:np.float64(0.44575339392758906) - actor/pg_loss:np.float64(0.00038603588473051786) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00025614264700379863) - actor/ppo_kl:np.float64(7.664802197382414e-05) - actor/pg_clipfrac_lower:np.float64(5.7444854064669926e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.92578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.92578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_w_mean:np.float64(74129.82275516656) - self_distillation/token_reweight_w_std:np.float64(1313364.9157028943) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003251135349274) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0171360018721316) - self_distillation/empty_target_batch:np.float64(0.07421875) - actor/grad_norm:np.float64(0.11729132384061813) - perf/mfu/actor:np.float64(0.03910900715029075) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.49448776245117) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:4 - critic/score/mean:0.83984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.83984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0034270414616912603 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0034270414616912603 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:420.8828125 - response_length/max:1701.0 - response_length/min:54.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:420.8828125 - response_length_non_aborted/max:1701.0 - response_length_non_aborted/min:54.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:252.40625 - prompt_length/max:340.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.895302355289459e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.649962367489934) - timing_s/agent_loop/generate_sequences/max:np.float64(11.452977204695344) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.1154736819225946) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.452977204695344) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:1701 - timing_s/gen:17.09042201936245 - timing_s/reward:0.056309279054403305 - timing_s/old_log_prob:2.4856445379555225 - timing_s/adv:0.686548063531518 - timing_s/update_actor:15.663952223956585 - timing_s/step:36.09207380563021 - timing_s/stop_profile:0.00012318789958953857 - timing_per_token_ms/update_actor:0.0908782227170524 - timing_per_token_ms/adv:0.003983175314347234 - timing_per_token_ms/gen:0.1586176936439631 - perf/total_num_tokens:172362 - perf/time_per_step:36.09207380563021 - perf/throughput:596.95239780428 (TaskRunner pid=2074004) Training Progress: 91%|█████████ | 91/100 [1:16:27<08:35, 57.27s/it] (TaskRunner pid=2074004) step:92 - global_seqlen/min:18485 - global_seqlen/max:32646 - global_seqlen/minmax_diff:14161 - global_seqlen/balanced_min:24274 - global_seqlen/balanced_max:24304 - global_seqlen/mean:24296.5 - actor/entropy:0.1161329373717308 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.46171876788139343 - training/rollout_probs_diff_mean:0.0028441078029572964 - training/rollout_probs_diff_std:0.011383011005818844 - training/rollout_actor_probs_pearson_corr:0.997846245765686 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.73046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:1.000044584274292 - rollout_corr/rollout_is_ratio_fraction_high:3.2485462725162506e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.060682840645313e-05 - rollout_corr/rollout_is_max:2.6116247177124023 - rollout_corr/rollout_is_min:0.3187040090560913 - rollout_corr/rollout_is_std:0.03276945650577545 - rollout_corr/rollout_is_eff_sample_size:0.998927552516703 - rollout_corr/rollout_is_seq_mean:0.9999802708625793 - rollout_corr/rollout_is_seq_std:0.0019448020029813051 - rollout_corr/rollout_is_seq_max:1.0066347122192383 - rollout_corr/rollout_is_seq_min:0.9897977113723755 - rollout_corr/rollout_is_seq_max_deviation:0.010202288627624512 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.125696862116456) - rollout_corr/training_log_ppl:np.float64(0.11609659137320705) - rollout_corr/kl:np.float64(0.0009234349944136078) - rollout_corr/k3_kl:np.float64(0.0007315961058225184) - rollout_corr/rollout_ppl:np.float64(1.1246338286437094) - rollout_corr/rollout_log_ppl:np.float64(0.11517315577657428) - rollout_corr/log_ppl_diff:np.float64(0.0009234355966327712) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018325499549973756) - rollout_corr/log_ppl_diff_max:np.float64(0.012700438499450684) - rollout_corr/log_ppl_diff_min:np.float64(-0.006820157170295715) - rollout_corr/ppl_ratio:np.float64(1.0009266710840166) - rollout_corr/chi2_token:np.float64(0.0010245046578347683) - rollout_corr/chi2_seq:np.float64(0.6241324651055038) - actor/pg_loss:np.float64(0.0013751451624557376) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006302250340013416) - actor/ppo_kl:np.float64(0.0003064914879740144) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_w_mean:np.float64(70968.99266718118) - self_distillation/token_reweight_w_std:np.float64(1136299.2140333871) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998132719192654) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03681001826771535) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.22948458045721054) - perf/mfu/actor:np.float64(0.04381601917617662) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.27560424804688) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:4 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007187408627942204 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0007187408627942204 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:480.984375 - response_length/max:1689.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:480.984375 - response_length_non_aborted/max:1689.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.28125 - prompt_length/max:460.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012553296983242035 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0251992959529161) - timing_s/agent_loop/generate_sequences/max:np.float64(11.716115767136216) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.778788086819986) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.716115767136216) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:1689 - timing_s/gen:17.857277484610677 - timing_s/reward:0.061120714992284775 - timing_s/old_log_prob:2.40597465634346 - timing_s/adv:0.6833560299128294 - timing_s/update_actor:15.71237531118095 - timing_s/step:36.81252425536513 - timing_s/stop_profile:0.00011310726404190063 - timing_per_token_ms/update_actor:0.08083661901498647 - timing_per_token_ms/adv:0.003515712293503331 - timing_per_token_ms/gen:0.14502548065986645 - perf/total_num_tokens:194372 - perf/time_per_step:36.81252425536513 - perf/throughput:660.0063563003011 (TaskRunner pid=2074004) Training Progress: 92%|█████████▏| 92/100 [1:17:04<06:49, 51.14s/it] (TaskRunner pid=2074004) step:93 - global_seqlen/min:14252 - global_seqlen/max:27080 - global_seqlen/minmax_diff:12828 - global_seqlen/balanced_min:19571 - global_seqlen/balanced_max:25808 - global_seqlen/mean:20389.375 - actor/entropy:0.11118520796298981 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2569490671157837 - training/rollout_probs_diff_mean:0.0027298631612211466 - training/rollout_probs_diff_std:0.01081868913024664 - training/rollout_actor_probs_pearson_corr:0.9979827404022217 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.75390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999743103981018 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9497259855270386 - rollout_corr/rollout_is_min:0.5504913330078125 - rollout_corr/rollout_is_std:0.030463820323348045 - rollout_corr/rollout_is_eff_sample_size:0.9990726971602333 - rollout_corr/rollout_is_seq_mean:1.000046968460083 - rollout_corr/rollout_is_seq_std:0.0025716039817780256 - rollout_corr/rollout_is_seq_max:1.0111936330795288 - rollout_corr/rollout_is_seq_min:0.9910350441932678 - rollout_corr/rollout_is_seq_max_deviation:0.011193633079528809 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1135128010064363) - rollout_corr/training_log_ppl:np.float64(0.1049198287037143) - rollout_corr/kl:np.float64(0.0006423072564984977) - rollout_corr/k3_kl:np.float64(0.0005943681892190966) - rollout_corr/rollout_ppl:np.float64(1.1127843372523785) - rollout_corr/rollout_log_ppl:np.float64(0.10427752112082089) - rollout_corr/log_ppl_diff:np.float64(0.0006423075828934088) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018248588894493878) - rollout_corr/log_ppl_diff_max:np.float64(0.009763795882463455) - rollout_corr/log_ppl_diff_min:np.float64(-0.008692525327205658) - rollout_corr/ppl_ratio:np.float64(1.0006455411203206) - rollout_corr/chi2_token:np.float64(0.0011548018082976341) - rollout_corr/chi2_seq:np.float64(0.28039377136155963) - actor/pg_loss:np.float64(0.0013577784411609173) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004785834780705045) - actor/ppo_kl:np.float64(0.0001363070303765923) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75390625) - self_distillation/token_reweight_w_mean:np.float64(101776.64841872128) - self_distillation/token_reweight_w_std:np.float64(1415721.968034971) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006784941069782) - self_distillation/token_reweight_w_clip_frac:np.float64(0.037072227569296956) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.1903113927692175) - perf/mfu/actor:np.float64(0.0352703899917267) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.2206916809082) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:4 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.06920255720615387 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.06920255720615387 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:379.94921875 - response_length/max:8192.0 - response_length/min:72.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:379.94921875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:72.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:257.21875 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.636208415031433e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.785060852766037) - timing_s/agent_loop/generate_sequences/max:np.float64(50.456341341137886) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6300632040947676) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.456341341137886) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:280 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.36329558491707 - timing_s/reward:0.06057967245578766 - timing_s/old_log_prob:2.5846155025064945 - timing_s/adv:0.5508739780634642 - timing_s/update_actor:16.457574239000678 - timing_s/step:76.1049917601049 - timing_s/stop_profile:0.00011239387094974518 - timing_per_token_ms/update_actor:0.10089552916041246 - timing_per_token_ms/adv:0.0033772122616771245 - timing_per_token_ms/gen:0.5794698673231113 - perf/total_num_tokens:163115 - perf/time_per_step:76.1049917601049 - perf/throughput:267.91113865790265 (TaskRunner pid=2074004) Training Progress: 93%|█████████▎| 93/100 [1:18:20<06:50, 58.65s/it] (TaskRunner pid=2074004) step:94 - global_seqlen/min:16134 - global_seqlen/max:25356 - global_seqlen/minmax_diff:9222 - global_seqlen/balanced_min:19639 - global_seqlen/balanced_max:19643 - global_seqlen/mean:19641.0 - actor/entropy:0.09782024472951889 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7454107999801636 - training/rollout_probs_diff_mean:0.002670757472515106 - training/rollout_probs_diff_std:0.01144047174602747 - training/rollout_actor_probs_pearson_corr:0.9975129961967468 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.7265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.999981164932251 - rollout_corr/rollout_is_ratio_fraction_high:2.235536157968454e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010059912165161222 - rollout_corr/rollout_is_max:2.278707504272461 - rollout_corr/rollout_is_min:0.010440262034535408 - rollout_corr/rollout_is_std:0.03061799891293049 - rollout_corr/rollout_is_eff_sample_size:0.9990632971725152 - rollout_corr/rollout_is_seq_mean:0.9999101161956787 - rollout_corr/rollout_is_seq_std:0.0022188930306583643 - rollout_corr/rollout_is_seq_max:1.0089306831359863 - rollout_corr/rollout_is_seq_min:0.9909623265266418 - rollout_corr/rollout_is_seq_max_deviation:0.009037673473358154 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1009153043851256) - rollout_corr/training_log_ppl:np.float64(0.09386854151671287) - rollout_corr/kl:np.float64(0.0008569538749156891) - rollout_corr/k3_kl:np.float64(0.0006892196433412323) - rollout_corr/rollout_ppl:np.float64(1.0999419218860567) - rollout_corr/rollout_log_ppl:np.float64(0.093011587527144) - rollout_corr/log_ppl_diff:np.float64(0.0008569539895688649) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018225670246465597) - rollout_corr/log_ppl_diff_max:np.float64(0.012111961841583252) - rollout_corr/log_ppl_diff_min:np.float64(-0.009987182915210724) - rollout_corr/ppl_ratio:np.float64(1.0008604750037193) - rollout_corr/chi2_token:np.float64(0.0010376402642577887) - rollout_corr/chi2_seq:np.float64(0.17592160333879292) - actor/pg_loss:np.float64(0.0020128018222749233) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006171084928610071) - actor/ppo_kl:np.float64(0.00018625303451358377) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_w_mean:np.float64(47890.5027854254) - self_distillation/token_reweight_w_std:np.float64(780379.3705423808) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003406463656574) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03524100726644974) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.30077048763632774) - perf/mfu/actor:np.float64(0.03571617935583021) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.26049041748047) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:4 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005389038939028978 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005389038939028978 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:349.46875 - response_length/max:992.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:349.46875 - response_length_non_aborted/max:992.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.3125 - prompt_length/max:341.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011242926120758057 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7382768783718348) - timing_s/agent_loop/generate_sequences/max:np.float64(7.45197612978518) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5364193320201593) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.45197612978518) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:258 - timing_s/agent_loop/slowest/response_length:992 - timing_s/gen:13.292611926794052 - timing_s/reward:0.058578019961714745 - timing_s/old_log_prob:2.304050751030445 - timing_s/adv:0.5737872198224068 - timing_s/update_actor:15.573226625099778 - timing_s/step:31.890709141269326 - timing_s/stop_profile:0.00011250190436840057 - timing_per_token_ms/update_actor:0.09911172181342459 - timing_per_token_ms/adv:0.0036517184704343386 - timing_per_token_ms/gen:0.14858056790210647 - perf/total_num_tokens:157128 - perf/time_per_step:31.890709141269326 - perf/throughput:615.8847052599044 (TaskRunner pid=2074004) Training Progress: 94%|█████████▍| 94/100 [1:18:52<05:03, 50.64s/it] (TaskRunner pid=2074004) step:95 - global_seqlen/min:19898 - global_seqlen/max:27127 - global_seqlen/minmax_diff:7229 - global_seqlen/balanced_min:22383 - global_seqlen/balanced_max:22390 - global_seqlen/mean:22386.125 - actor/entropy:0.11839713156223297 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4174291491508484 - training/rollout_probs_diff_mean:0.002880886197090149 - training/rollout_probs_diff_std:0.01125163584947586 - training/rollout_actor_probs_pearson_corr:0.9979742765426636 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.000062108039856 - rollout_corr/rollout_is_ratio_fraction_high:1.846091436163988e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.7691370632965118e-05 - rollout_corr/rollout_is_max:2.8053035736083984 - rollout_corr/rollout_is_min:0.35777518153190613 - rollout_corr/rollout_is_std:0.031417399644851685 - rollout_corr/rollout_is_eff_sample_size:0.9990140393695902 - rollout_corr/rollout_is_seq_mean:1.0001864433288574 - rollout_corr/rollout_is_seq_std:0.001943354494869709 - rollout_corr/rollout_is_seq_max:1.009150743484497 - rollout_corr/rollout_is_seq_min:0.9956559538841248 - rollout_corr/rollout_is_seq_max_deviation:0.00915074348449707 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1184041863307357) - rollout_corr/training_log_ppl:np.float64(0.10852470160170924) - rollout_corr/kl:np.float64(0.0004109893218089411) - rollout_corr/k3_kl:np.float64(0.0005756196864368235) - rollout_corr/rollout_ppl:np.float64(1.1179265826940536) - rollout_corr/rollout_log_ppl:np.float64(0.10811371153613436) - rollout_corr/log_ppl_diff:np.float64(0.0004109900655748788) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015215025887300726) - rollout_corr/log_ppl_diff_max:np.float64(0.010581329464912415) - rollout_corr/log_ppl_diff_min:np.float64(-0.007771208882331848) - rollout_corr/ppl_ratio:np.float64(1.0004132050089538) - rollout_corr/chi2_token:np.float64(0.0015168595127761364) - rollout_corr/chi2_seq:np.float64(0.37046705558896065) - actor/pg_loss:np.float64(0.0014443554682657123) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039592202506355534) - actor/ppo_kl:np.float64(9.238178380144291e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(35129.66564378468) - self_distillation/token_reweight_w_std:np.float64(673946.7570342987) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999094212660566) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03862924790882971) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.22043711598962545) - perf/mfu/actor:np.float64(0.04063184804557128) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.2927703857422) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:4 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0033506557811051607 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0033506557811051607 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:423.19140625 - response_length/max:1177.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:423.19140625 - response_length_non_aborted/max:1177.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.375 - prompt_length/max:375.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.422190487384796e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9937835987657309) - timing_s/agent_loop/generate_sequences/max:np.float64(8.682577695697546) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.320787707940326) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.682577695697546) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:1171 - timing_s/gen:14.465107165277004 - timing_s/reward:0.061005301773548126 - timing_s/old_log_prob:2.500914089381695 - timing_s/adv:0.5702372565865517 - timing_s/update_actor:15.484657960012555 - timing_s/step:33.17570861801505 - timing_s/stop_profile:0.00012160651385784149 - timing_per_token_ms/update_actor:0.08646347882903224 - timing_per_token_ms/adv:0.0031840998419029177 - timing_per_token_ms/gen:0.13351954701788865 - perf/total_num_tokens:179089 - perf/time_per_step:33.17570861801505 - perf/throughput:674.7745845538293 (TaskRunner pid=2074004) Training Progress: 95%|█████████▌| 95/100 [1:19:25<03:47, 45.41s/it] (TaskRunner pid=2074004) step:96 - global_seqlen/min:14700 - global_seqlen/max:31388 - global_seqlen/minmax_diff:16688 - global_seqlen/balanced_min:22938 - global_seqlen/balanced_max:22942 - global_seqlen/mean:22940.5 - actor/entropy:0.10098060220479965 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6580971479415894 - training/rollout_probs_diff_mean:0.002621095860376954 - training/rollout_probs_diff_std:0.011137053370475769 - training/rollout_actor_probs_pearson_corr:0.9977489113807678 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.6796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.0001559257507324 - rollout_corr/rollout_is_ratio_fraction_high:1.786160828487482e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.251563172554597e-05 - rollout_corr/rollout_is_max:2.2557666301727295 - rollout_corr/rollout_is_min:0.252839058637619 - rollout_corr/rollout_is_std:0.03050878830254078 - rollout_corr/rollout_is_eff_sample_size:0.9990705553753199 - rollout_corr/rollout_is_seq_mean:1.0001717805862427 - rollout_corr/rollout_is_seq_std:0.001738636172376573 - rollout_corr/rollout_is_seq_max:1.007121205329895 - rollout_corr/rollout_is_seq_min:0.9931297898292542 - rollout_corr/rollout_is_seq_max_deviation:0.0071212053298950195 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.103144368622452) - rollout_corr/training_log_ppl:np.float64(0.09644664141524117) - rollout_corr/kl:np.float64(0.0005017887852607483) - rollout_corr/k3_kl:np.float64(0.0005338758359130225) - rollout_corr/rollout_ppl:np.float64(1.1025838060304523) - rollout_corr/rollout_log_ppl:np.float64(0.09594485214620363) - rollout_corr/log_ppl_diff:np.float64(0.0005017892690375447) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013042025966569781) - rollout_corr/log_ppl_diff_max:np.float64(0.006274871528148651) - rollout_corr/log_ppl_diff_min:np.float64(-0.005490683019161224) - rollout_corr/ppl_ratio:np.float64(1.0005033176857978) - rollout_corr/chi2_token:np.float64(0.0011128755286335945) - rollout_corr/chi2_seq:np.float64(0.5927000071387738) - actor/pg_loss:np.float64(0.0014072328340262175) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000444404973222845) - actor/ppo_kl:np.float64(0.000197858096928627) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_w_mean:np.float64(106493.64426750923) - self_distillation/token_reweight_w_std:np.float64(2085160.6336320788) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998901744838804) - self_distillation/token_reweight_w_clip_frac:np.float64(0.036775027387193404) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.2108887266367674) - perf/mfu/actor:np.float64(0.0416768245283575) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.38690567016602) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:4 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002023943467065692 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002023943467065692 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:437.390625 - response_length/max:1046.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:437.390625 - response_length_non_aborted/max:1046.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.5 - prompt_length/max:355.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.732507169246674e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1755850836634636) - timing_s/agent_loop/generate_sequences/max:np.float64(8.23095478862524) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.5998671856941655) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.23095478862524) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:296 - timing_s/agent_loop/slowest/response_length:1046 - timing_s/gen:14.00526099652052 - timing_s/reward:0.06283171847462654 - timing_s/old_log_prob:2.3491326589137316 - timing_s/adv:0.5709024202078581 - timing_s/update_actor:15.67631054110825 - timing_s/step:32.75171426124871 - timing_s/stop_profile:0.0001319441944360733 - timing_per_token_ms/update_actor:0.08541831336015045 - timing_per_token_ms/adv:0.0031107779920220686 - timing_per_token_ms/gen:0.1250782427439049 - perf/total_num_tokens:183524 - perf/time_per_step:32.75171426124871 - perf/throughput:700.4366188899866 (TaskRunner pid=2074004) Training Progress: 96%|█████████▌| 96/100 [1:19:58<02:46, 41.63s/it] (TaskRunner pid=2074004) step:97 - global_seqlen/min:17529 - global_seqlen/max:26956 - global_seqlen/minmax_diff:9427 - global_seqlen/balanced_min:22453 - global_seqlen/balanced_max:22536 - global_seqlen/mean:22478.25 - actor/entropy:0.11855632066726685 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3018026649951935 - training/rollout_probs_diff_mean:0.0028152333106845617 - training/rollout_probs_diff_std:0.010734545066952705 - training/rollout_actor_probs_pearson_corr:0.9980775713920593 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.9609375 - self_distillation/correct_sample_fraction:0.79296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9609375 - rollout_corr/rollout_is_mean:0.9999098777770996 - rollout_corr/rollout_is_ratio_fraction_high:1.8008608094532974e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.4025826102588326e-05 - rollout_corr/rollout_is_max:4.380692005157471 - rollout_corr/rollout_is_min:0.2371114045381546 - rollout_corr/rollout_is_std:0.030879687517881393 - rollout_corr/rollout_is_eff_sample_size:0.9990471153295336 - rollout_corr/rollout_is_seq_mean:0.9999960660934448 - rollout_corr/rollout_is_seq_std:0.0019316772231832147 - rollout_corr/rollout_is_seq_max:1.0092357397079468 - rollout_corr/rollout_is_seq_min:0.9920583367347717 - rollout_corr/rollout_is_seq_max_deviation:0.009235739707946777 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1194870863109827) - rollout_corr/training_log_ppl:np.float64(0.10984483735228423) - rollout_corr/kl:np.float64(0.0007211861242026885) - rollout_corr/k3_kl:np.float64(0.0006180598990965791) - rollout_corr/rollout_ppl:np.float64(1.118659039027989) - rollout_corr/rollout_log_ppl:np.float64(0.10912365088734077) - rollout_corr/log_ppl_diff:np.float64(0.0007211864649434574) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015570802715956233) - rollout_corr/log_ppl_diff_max:np.float64(0.011463932693004608) - rollout_corr/log_ppl_diff_min:np.float64(-0.005851447582244873) - rollout_corr/ppl_ratio:np.float64(1.000723579665646) - rollout_corr/chi2_token:np.float64(0.0009887183550745249) - rollout_corr/chi2_seq:np.float64(0.33689292683266103) - actor/pg_loss:np.float64(0.0028610387817025185) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005207407527905161) - actor/ppo_kl:np.float64(0.00018532980526941856) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.79296875) - self_distillation/token_reweight_w_mean:np.float64(61391.32670080848) - self_distillation/token_reweight_w_std:np.float64(1146314.7785266174) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994598587509245) - self_distillation/token_reweight_w_clip_frac:np.float64(0.060132228885777295) - self_distillation/empty_target_batch:np.float64(0.0390625) - actor/grad_norm:np.float64(0.27978259697556496) - perf/mfu/actor:np.float64(0.04065052381475308) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.33649063110352) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:4 - critic/score/mean:0.79296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.79296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006340155377984047 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006340155377984047 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:433.8203125 - response_length/max:1625.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:433.8203125 - response_length_non_aborted/max:1625.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.625 - prompt_length/max:353.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014923326671123505 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1448223646730185) - timing_s/agent_loop/generate_sequences/max:np.float64(10.64167152158916) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.490232729578565) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.64167152158916) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:1625 - timing_s/gen:16.820046979933977 - timing_s/reward:0.062204478308558464 - timing_s/old_log_prob:2.3776866644620895 - timing_s/adv:0.6371833365410566 - timing_s/update_actor:15.58851253055036 - timing_s/step:35.57156837172806 - timing_s/stop_profile:0.00011635571718215942 - timing_per_token_ms/update_actor:0.08668664448161201 - timing_per_token_ms/adv:0.003543332646786653 - timing_per_token_ms/gen:0.15145281726605897 - perf/total_num_tokens:179826 - perf/time_per_step:35.57156837172806 - perf/throughput:631.9161911867089 (TaskRunner pid=2074004) Training Progress: 97%|█████████▋| 97/100 [1:20:34<01:59, 39.83s/it] (TaskRunner pid=2074004) step:98 - global_seqlen/min:17481 - global_seqlen/max:26054 - global_seqlen/minmax_diff:8573 - global_seqlen/balanced_min:22060 - global_seqlen/balanced_max:22069 - global_seqlen/mean:22065.375 - actor/entropy:0.09252151846885681 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6313449740409851 - training/rollout_probs_diff_mean:0.0024829101748764515 - training/rollout_probs_diff_std:0.01076439581811428 - training/rollout_actor_probs_pearson_corr:0.9977108836174011 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.70703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:0.9999709129333496 - rollout_corr/rollout_is_ratio_fraction_high:2.7911390134249814e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.582278026849963e-05 - rollout_corr/rollout_is_max:2.398716926574707 - rollout_corr/rollout_is_min:0.06967287510633469 - rollout_corr/rollout_is_std:0.02976113185286522 - rollout_corr/rollout_is_eff_sample_size:0.9991149397948247 - rollout_corr/rollout_is_seq_mean:0.9999794363975525 - rollout_corr/rollout_is_seq_std:0.0017344678053632379 - rollout_corr/rollout_is_seq_max:1.0062588453292847 - rollout_corr/rollout_is_seq_min:0.9945258498191833 - rollout_corr/rollout_is_seq_max_deviation:0.006258845329284668 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.093379558995366) - rollout_corr/training_log_ppl:np.float64(0.08759709404694149) - rollout_corr/kl:np.float64(0.000628665536446249) - rollout_corr/k3_kl:np.float64(0.0005191412595024758) - rollout_corr/rollout_ppl:np.float64(1.0926776281557977) - rollout_corr/rollout_log_ppl:np.float64(0.08696842801145976) - rollout_corr/log_ppl_diff:np.float64(0.0006286660354817286) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013417766604106873) - rollout_corr/log_ppl_diff_max:np.float64(0.00907956063747406) - rollout_corr/log_ppl_diff_min:np.float64(-0.005724966526031494) - rollout_corr/ppl_ratio:np.float64(1.0006304366979748) - rollout_corr/chi2_token:np.float64(0.000801675021648407) - rollout_corr/chi2_seq:np.float64(0.10571352113038301) - actor/pg_loss:np.float64(0.0008303822251036763) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003051055082323728) - actor/ppo_kl:np.float64(0.00013278382333581362) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_w_mean:np.float64(32778.418274484575) - self_distillation/token_reweight_w_std:np.float64(637282.1950311251) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001593332272023) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02661549543699948) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.16087789833545685) - perf/mfu/actor:np.float64(0.03977215253160561) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.22827196121216) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:4 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006000949069857597 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006000949069857597 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:419.85546875 - response_length/max:1305.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:419.85546875 - response_length_non_aborted/max:1305.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.6875 - prompt_length/max:350.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011768564581871033 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7680980022996664) - timing_s/agent_loop/generate_sequences/max:np.float64(9.386873960494995) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.251178185098979) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.386873960494995) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:279 - timing_s/agent_loop/slowest/response_length:1305 - timing_s/gen:15.361656874418259 - timing_s/reward:0.06246807053685188 - timing_s/old_log_prob:2.382600510492921 - timing_s/adv:0.5709490403532982 - timing_s/update_actor:15.436897011473775 - timing_s/step:33.90267143212259 - timing_s/stop_profile:0.00010995566844940186 - timing_per_token_ms/update_actor:0.0874497771478718 - timing_per_token_ms/adv:0.003234417273405155 - timing_per_token_ms/gen:0.14292173529226257 - perf/total_num_tokens:176523 - perf/time_per_step:33.90267143212259 - perf/throughput:650.8447289818342 (TaskRunner pid=2074004) Training Progress: 98%|█████████▊| 98/100 [1:21:08<01:16, 38.07s/it] (TaskRunner pid=2074004) step:99 - global_seqlen/min:17883 - global_seqlen/max:26736 - global_seqlen/minmax_diff:8853 - global_seqlen/balanced_min:21621 - global_seqlen/balanced_max:21644 - global_seqlen/mean:21636.5 - actor/entropy:0.10580387711524963 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4862961173057556 - training/rollout_probs_diff_mean:0.002484919736161828 - training/rollout_probs_diff_std:0.0104207843542099 - training/rollout_actor_probs_pearson_corr:0.9981136918067932 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.96875 - self_distillation/correct_sample_fraction:0.90234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.96875 - rollout_corr/rollout_is_mean:0.9999649524688721 - rollout_corr/rollout_is_ratio_fraction_high:9.457158739678562e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.674295607605018e-05 - rollout_corr/rollout_is_max:2.7389602661132812 - rollout_corr/rollout_is_min:0.13493947684764862 - rollout_corr/rollout_is_std:0.028364475816488266 - rollout_corr/rollout_is_eff_sample_size:0.9991959842570144 - rollout_corr/rollout_is_seq_mean:0.9998395442962646 - rollout_corr/rollout_is_seq_std:0.001808670349419117 - rollout_corr/rollout_is_seq_max:1.007258415222168 - rollout_corr/rollout_is_seq_min:0.9866185188293457 - rollout_corr/rollout_is_seq_max_deviation:0.013381481170654297 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0939670382067561) - rollout_corr/training_log_ppl:np.float64(0.08732558643532684) - rollout_corr/kl:np.float64(0.000521703354597669) - rollout_corr/k3_kl:np.float64(0.00046708716944898754) - rollout_corr/rollout_ppl:np.float64(1.0933743044734001) - rollout_corr/rollout_log_ppl:np.float64(0.0868038823173265) - rollout_corr/log_ppl_diff:np.float64(0.0005217041180003434) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012858145564678125) - rollout_corr/log_ppl_diff_max:np.float64(0.015529707074165344) - rollout_corr/log_ppl_diff_min:np.float64(-0.006662759929895401) - rollout_corr/ppl_ratio:np.float64(1.000523641007021) - rollout_corr/chi2_token:np.float64(0.0009014918468892574) - rollout_corr/chi2_seq:np.float64(0.2561511362437159) - actor/pg_loss:np.float64(0.0010093091987073421) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018811171594279585) - actor/ppo_kl:np.float64(-4.058463273981516e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.96875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.96875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_w_mean:np.float64(28957.405509632314) - self_distillation/token_reweight_w_std:np.float64(466637.31905894005) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997467994689941) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0230792346119415) - self_distillation/empty_target_batch:np.float64(0.03125) - actor/grad_norm:np.float64(0.24430519342422485) - perf/mfu/actor:np.float64(0.039178741434394745) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.19931030273438) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:4 - critic/score/mean:0.90234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.90234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005199073348194361 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005199073348194361 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:413.046875 - response_length/max:1520.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:413.046875 - response_length_non_aborted/max:1520.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.09375 - prompt_length/max:365.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013734027743339539 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8036548234522343) - timing_s/agent_loop/generate_sequences/max:np.float64(10.340177113190293) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.11781918026827) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.340177113190293) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:313 - timing_s/agent_loop/slowest/response_length:1520 - timing_s/gen:16.068710835650563 - timing_s/reward:0.06060960330069065 - timing_s/old_log_prob:2.314846996217966 - timing_s/adv:0.6209873277693987 - timing_s/update_actor:15.458110202103853 - timing_s/step:34.610270539298654 - timing_s/stop_profile:0.0001095551997423172 - timing_per_token_ms/update_actor:0.08930574608938514 - timing_per_token_ms/adv:0.0035876142616030706 - timing_per_token_ms/gen:0.15196435441318862 - perf/total_num_tokens:173092 - perf/time_per_step:34.610270539298654 - perf/throughput:625.1468036180928 (TaskRunner pid=2074004) Training Progress: 99%|█████████▉| 99/100 [1:21:42<00:37, 37.05s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Solving for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substituting the given values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest answer is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90/actor (TaskRunner pid=2074004) step:100 - global_seqlen/min:14131 - global_seqlen/max:24831 - global_seqlen/minmax_diff:10700 - global_seqlen/balanced_min:20764 - global_seqlen/balanced_max:20778 - global_seqlen/mean:20773.375 - actor/entropy:0.10674719512462616 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6354759931564331 - training/rollout_probs_diff_mean:0.0026373525615781546 - training/rollout_probs_diff_std:0.011061684228479862 - training/rollout_actor_probs_pearson_corr:0.9978691935539246 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.890625 - self_distillation/correct_sample_fraction:0.7265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.890625 - rollout_corr/rollout_is_mean:1.0000841617584229 - rollout_corr/rollout_is_ratio_fraction_high:3.084801210206933e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.169602420413867e-05 - rollout_corr/rollout_is_max:3.1517021656036377 - rollout_corr/rollout_is_min:0.13044416904449463 - rollout_corr/rollout_is_std:0.03166119009256363 - rollout_corr/rollout_is_eff_sample_size:0.998998810886964 - rollout_corr/rollout_is_seq_mean:1.0001347064971924 - rollout_corr/rollout_is_seq_std:0.0022613920737057924 - rollout_corr/rollout_is_seq_max:1.0161210298538208 - rollout_corr/rollout_is_seq_min:0.9922299981117249 - rollout_corr/rollout_is_seq_max_deviation:0.0161210298538208 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1050673802383244) - rollout_corr/training_log_ppl:np.float64(0.09719672136634472) - rollout_corr/kl:np.float64(0.00043732599254298066) - rollout_corr/k3_kl:np.float64(0.0005812929390707211) - rollout_corr/rollout_ppl:np.float64(1.1045606597326696) - rollout_corr/rollout_log_ppl:np.float64(0.09675939454609761) - rollout_corr/log_ppl_diff:np.float64(0.0004373268202471081) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015722126299806405) - rollout_corr/log_ppl_diff_max:np.float64(0.009256497025489807) - rollout_corr/log_ppl_diff_min:np.float64(-0.006794825196266174) - rollout_corr/ppl_ratio:np.float64(1.0004396380390972) - rollout_corr/chi2_token:np.float64(0.0014464769046753645) - rollout_corr/chi2_seq:np.float64(0.31380061944946647) - actor/pg_loss:np.float64(0.0012798310490325093) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005447886960610049) - actor/ppo_kl:np.float64(1.8937110884076347e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.890625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.890625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_w_mean:np.float64(181507.5251128471) - self_distillation/token_reweight_w_std:np.float64(3188081.7809230387) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007254967931658) - self_distillation/token_reweight_w_clip_frac:np.float64(0.029081717497319914) - self_distillation/empty_target_batch:np.float64(0.109375) - actor/grad_norm:np.float64(0.18284212984144688) - perf/mfu/actor:np.float64(0.038138719584986394) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.18253707885742) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7109375) - val-aux/sciknoweval/reward/std@16:np.float64(0.16604232971456567) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.78) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1094670184264562) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.645725) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.16903151272091094) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7134375000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.16525033629837935) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8191750000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.0603992963721259) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.57575) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.15001400456735414) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.733725) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1297479179197353) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8390250000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.02670945685411765) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5126625) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.12034852442580153) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7460875) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.0934240016452198) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8469249999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.009961583262996359) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4606) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.08389710095714561) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7487375000000001) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07219685215265889) - val-aux/sciknoweval/score/mean@16:np.float64(0.7109375) - val-aux/sciknoweval/score/std@16:np.float64(0.16604232971456567) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.78) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1094670184264562) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.645725) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.16903151272091094) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7134375000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.16525033629837935) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8191750000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.0603992963721259) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.57575) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.15001400456735414) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.733725) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1297479179197353) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8390250000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.02670945685411765) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5126625) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.12034852442580153) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7460875) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.0934240016452198) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8469249999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.009961583262996359) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4606) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.08389710095714561) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7487375000000001) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07219685215265889) - val-core/sciknoweval/acc/mean@16:np.float64(0.7109375) - val-aux/sciknoweval/acc/std@16:np.float64(0.16604232971456567) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.78) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1094670184264562) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.645725) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.16903151272091094) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7134375000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.16525033629837935) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8191750000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.0603992963721259) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.57575) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.15001400456735414) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.733725) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1297479179197353) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8390250000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.02670945685411765) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5126625) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.12034852442580153) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7460875) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.0934240016452198) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8469249999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.009961583262996359) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4606) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.08389710095714561) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7487375000000001) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07219685215265889) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9984375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.006051536478449089) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999125) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.001440128696321335) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9969625000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.008153644123697198) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9985125) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.005913422059734724) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9942624999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.010511931298848583) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9997125) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.002649136032174741) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9901125000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.012221532639586832) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9999874999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.00039508701573197775) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9841000000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.012028687586990721) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:4 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0031310732010751963 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:-0.0031310732010751963 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:379.88671875 - response_length/max:1422.0 - response_length/min:75.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:379.88671875 - response_length_non_aborted/max:1422.0 - response_length_non_aborted/min:75.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.28125 - prompt_length/max:383.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011692196130752563 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5598204713314772) - timing_s/agent_loop/generate_sequences/max:np.float64(9.012866754084826) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3422235102189006) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.012866754084826) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1422 - timing_s/gen:14.940290039405227 - timing_s/reward:0.05983664467930794 - timing_s/old_log_prob:2.2800443712621927 - timing_s/adv:0.5855069477111101 - timing_s/update_actor:15.32030195184052 - timing_s/step:33.273349760100245 - timing_s/testing:83.49365962296724 - timing_s/save_checkpoint:6.64394349232316 - timing_s/stop_profile:0.0001235567033290863 - timing_per_token_ms/update_actor:0.09218712626042061 - timing_per_token_ms/adv:0.0035231814023426026 - timing_per_token_ms/gen:0.1536260813709394 - perf/total_num_tokens:166187 - perf/time_per_step:33.273349760100245 - perf/throughput:624.3247268392076 (TaskRunner pid=2074004) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2074004) "np.float64(0.7109375), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2074004) "np.float64(0.16604232971456567), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2074004) "np.float64(0.78), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2074004) "np.float64(0.1094670184264562), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2074004) "np.float64(0.645725), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2074004) "np.float64(0.16903151272091094), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2074004) "np.float64(0.7134375000000001), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2074004) "np.float64(0.16525033629837935), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2074004) "np.float64(0.8191750000000001), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2074004) "np.float64(0.0603992963721259), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2074004) "np.float64(0.57575), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2074004) "np.float64(0.15001400456735414), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2074004) "np.float64(0.733725), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2074004) "np.float64(0.1297479179197353), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2074004) "np.float64(0.8390250000000001), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2074004) "np.float64(0.02670945685411765), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2074004) "np.float64(0.5126625), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2074004) "np.float64(0.12034852442580153), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2074004) "np.float64(0.7460875), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2074004) "np.float64(0.0934240016452198), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2074004) "np.float64(0.8469249999999999), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2074004) 'np.float64(0.009961583262996359), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/reward/worst@16/mean': np.float64(0.4606), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/reward/worst@16/std': np.float64(0.08389710095714561), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/reward/maj@16/mean': np.float64(0.7487375000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/reward/maj@16/std': np.float64(0.07219685215265889), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/mean@16': np.float64(0.7109375), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/std@16': np.float64(0.16604232971456567), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@2/mean': np.float64(0.78), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@2/std': np.float64(0.1094670184264562), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@2/mean': np.float64(0.645725), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@2/std': np.float64(0.16903151272091094), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@2/mean': np.float64(0.7134375000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@2/std': np.float64(0.16525033629837935), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@4/mean': np.float64(0.8191750000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@4/std': np.float64(0.0603992963721259), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@4/mean': np.float64(0.57575), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@4/std': np.float64(0.15001400456735414), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@4/mean': np.float64(0.733725), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@4/std': np.float64(0.1297479179197353), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@8/mean': np.float64(0.8390250000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@8/std': np.float64(0.02670945685411765), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@8/mean': np.float64(0.5126625), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@8/std': np.float64(0.12034852442580153), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@8/mean': np.float64(0.7460875), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@8/std': np.float64(0.0934240016452198), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@16/mean': np.float64(0.8469249999999999), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@16/std': np.float64(0.009961583262996359), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@16/mean': np.float64(0.4606), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@16/std': np.float64(0.08389710095714561), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@16/mean': np.float64(0.7487375000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@16/std': np.float64(0.07219685215265889), " (TaskRunner pid=2074004) "'val-core/sciknoweval/acc/mean@16': np.float64(0.7109375), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/std@16': np.float64(0.16604232971456567), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/best@2/mean': np.float64(0.78), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/best@2/std': np.float64(0.1094670184264562), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@2/mean': np.float64(0.645725), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@2/std': np.float64(0.16903151272091094), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/maj@2/mean': np.float64(0.7134375000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/maj@2/std': np.float64(0.16525033629837935), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/best@4/mean': np.float64(0.8191750000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/best@4/std': np.float64(0.0603992963721259), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@4/mean': np.float64(0.57575), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@4/std': np.float64(0.15001400456735414), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/maj@4/mean': np.float64(0.733725), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/maj@4/std': np.float64(0.1297479179197353), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/best@8/mean': np.float64(0.8390250000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/best@8/std': np.float64(0.02670945685411765), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@8/mean': np.float64(0.5126625), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@8/std': np.float64(0.12034852442580153), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/maj@8/mean': np.float64(0.7460875), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/maj@8/std': np.float64(0.0934240016452198), " (TaskRunner pid=2074004) "'val-core/sciknoweval/acc/best@16/mean': np.float64(0.8469249999999999), " (TaskRunner pid=2074004) "'val-core/sciknoweval/acc/best@16/std': np.float64(0.009961583262996359), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@16/mean': np.float64(0.4606), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@16/std': np.float64(0.08389710095714561), " (TaskRunner pid=2074004) "'val-core/sciknoweval/acc/maj@16/mean': np.float64(0.7487375000000001), " (TaskRunner pid=2074004) "'val-core/sciknoweval/acc/maj@16/std': np.float64(0.07219685215265889), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/mean@16': np.float64(0.9984375), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2074004) 'np.float64(0.006051536478449089), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@2/mean': np.float64(0.9999125), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2074004) 'np.float64(0.001440128696321335), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': " (TaskRunner pid=2074004) 'np.float64(0.9969625000000001), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2074004) 'np.float64(0.008153644123697198), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(0.9985125), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2074004) 'np.float64(0.005913422059734724), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@4/mean': np.float64(1.0), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@4/std': np.float64(0.0), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': " (TaskRunner pid=2074004) 'np.float64(0.9942624999999999), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2074004) 'np.float64(0.010511931298848583), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': np.float64(0.9997125), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2074004) 'np.float64(0.002649136032174741), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': " (TaskRunner pid=2074004) 'np.float64(0.9901125000000001), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2074004) 'np.float64(0.012221532639586832), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': " (TaskRunner pid=2074004) 'np.float64(0.9999874999999999), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@8/std': " (TaskRunner pid=2074004) 'np.float64(0.00039508701573197775), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': " (TaskRunner pid=2074004) 'np.float64(0.9841000000000001), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2074004) 'np.float64(0.012028687586990721), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': np.float64(1.0), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@16/std': np.float64(0.0), " (TaskRunner pid=2074004) "'val-aux/num_turns/min': np.int32(2), 'val-aux/num_turns/max': np.int32(2), " (TaskRunner pid=2074004) "'val-aux/num_turns/mean': np.float64(2.0)}") (TaskRunner pid=2074004) Training Progress: 100%|██████████| 100/100 [1:23:46<00:00, 62.98s/it] Training Progress: 100%|██████████| 100/100 [1:23:46<00:00, 50.26s/it] (TaskRunner pid=2074004) wandb: updating run metadata (TaskRunner pid=2074004) wandb: updating run metadata; uploading output.log; uploading wandb-summary.json (TaskRunner pid=2074004) wandb: uploading output.log; uploading config.yaml (TaskRunner pid=2074004) wandb: uploading history steps 98-99, summary, console lines 791-912 (TaskRunner pid=2074004) wandb: (TaskRunner pid=2074004) wandb: Run history: (TaskRunner pid=2074004) wandb: actor/entropy ▇▆█▇▇█▅▆▄▃▂▆▃▂▄▁▄▃▂▁▂▁▃▄▄▅▃▅▃▄▆▄▄▆▄▄▅▆▃▄ (TaskRunner pid=2074004) wandb: actor/grad_norm ▇▃█▂▃▃▅▄▅▄▃▂▄▃▃▃▄▄▂▃▃▃▂▃▃▂▂▃▁▂▃▂▂▂▂▃▂▄▁▁ (TaskRunner pid=2074004) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2074004) wandb: actor/lr ▁▂▃█████████████████████████████████████ (TaskRunner pid=2074004) wandb: actor/pg_clipfrac ▁▄▄▅▄▄▄▄▅▇▅▆▄▄▄▆▄▆▃▃▅▅▅▆▅▄▆▅▄▃▂▂▅▃▄▂█▆▄▄ (TaskRunner pid=2074004) wandb: actor/pg_clipfrac_lower ▁▅▁▁▁▁▁█▅▁▁▁▃▁▆▁▁▁▁▃▃▁▁▁▅▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2074004) wandb: actor/pg_loss ▄▇█▄▇▅▇▇▅▅▄▆▆▅▄▄▆▄▄▆▄▃▂▃▄▃▂▂▃▃▄▄▄▃▅▁▃▃▅▂ (TaskRunner pid=2074004) wandb: actor/ppo_kl ▁▄▂▂▃▇▂▂▄▂▄▄▃▃▃▅▃▄▃▄▇▃▃▄▃▄▃▄▃█▃▃▅▄▄▃▄▃▄▃ (TaskRunner pid=2074004) wandb: critic/advantages/max ▇███▇████████▇██▅█▅▇██▇███▁██▇▇████▇███▇ (TaskRunner pid=2074004) wandb: critic/advantages/mean ▇▃▃▅▄▃▅▆▇▆▅█▃▆▃▃▃▄▃▃▅▆▇▄▁▇▆▇▃▅▅▅▇▇▆▅▅▄▅▅ (TaskRunner pid=2074004) wandb: +220 ... (TaskRunner pid=2074004) wandb: (TaskRunner pid=2074004) wandb: Run summary: (TaskRunner pid=2074004) wandb: actor/entropy 0.10675 (TaskRunner pid=2074004) wandb: actor/grad_norm 0.18284 (TaskRunner pid=2074004) wandb: actor/kl_loss 0 (TaskRunner pid=2074004) wandb: actor/lr 0.0 (TaskRunner pid=2074004) wandb: actor/pg_clipfrac 0.00054 (TaskRunner pid=2074004) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2074004) wandb: actor/pg_loss 0.00128 (TaskRunner pid=2074004) wandb: actor/ppo_kl 2e-05 (TaskRunner pid=2074004) wandb: critic/advantages/max 0.875 (TaskRunner pid=2074004) wandb: critic/advantages/mean -0.00313 (TaskRunner pid=2074004) wandb: +220 ... (TaskRunner pid=2074004) wandb: (TaskRunner pid=2074004) wandb: 🚀 View run qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/eajdt9hv (TaskRunner pid=2074004) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2074004) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2074004) wandb: Find logs at: ./wandb/run-20260702_085822-eajdt9hv/logs (TaskRunner pid=2074004) Exception ignored in atexit callback: .teardown_atexit at 0x7f1bd03c7880> (TaskRunner pid=2074004) Traceback (most recent call last): (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 65, in teardown_atexit (TaskRunner pid=2074004) conn.teardown(hooks.exit_code) (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 299, in teardown (TaskRunner pid=2074004) self._asyncer.run(publish_teardown_and_close) (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run (TaskRunner pid=2074004) return future.result() (TaskRunner pid=2074004) ^^^^^^^^^^^^^^^ (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result (TaskRunner pid=2074004) return self.__get_result() (TaskRunner pid=2074004) ^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (TaskRunner pid=2074004) raise self._exception (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap (TaskRunner pid=2074004) return await fn() (TaskRunner pid=2074004) ^^^^^^^^^^ (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 297, in publish_teardown_and_close (TaskRunner pid=2074004) await self._client.close() (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 69, in close (TaskRunner pid=2074004) await self._writer.wait_closed() (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 364, in wait_closed (TaskRunner pid=2074004) await self._protocol._get_close_waiter(self) (TaskRunner pid=2074004) BrokenPipeError: [Errno 32] Broken pipe main_ppo exit code: 0 [2026-07-02 10:22:16] Finished physics rlsd_tr [2026-07-02 10:22:16] Starting biology grpo Experiment: qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Dataset: biology Method: grpo Config: baseline_grpo Model: Qwen/Qwen3-4B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet Ray tmp: /tmp/ray_q3g_biology_grpo_Qwen3_4B 2026-07-02 10:22:17,959 VINFO scripts.py:1286 -- Killed `ray::IDLE "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""` (via SIGKILL) 2026-07-02 10:22:17,960 VINFO scripts.py:1286 -- Killed `ray::IDLE "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""` (via SIGKILL) 2026-07-02 10:22:17,960 VINFO scripts.py:1286 -- Killed `ray::IDLE "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""` (via SIGKILL) 2026-07-02 10:22:17,960 VINFO scripts.py:1286 -- Killed `ray::IDLE "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""` (via SIGKILL) 2026-07-02 10:22:17,961 VINFO scripts.py:1286 -- Killed `ray::IDLE "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""` (via SIGKILL) 2026-07-02 10:22:17,961 VINFO scripts.py:1286 -- Killed `ray::IDLE "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""` (via SIGKILL) 2026-07-02 10:22:17,962 VINFO scripts.py:1286 -- Killed `ray::IDLE "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""` (via SIGKILL) 2026-07-02 10:22:17,970 VINFO scripts.py:1286 -- Killed `/mnt/mole/SDPO/L2T/.venv/bin/python -u /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/runtime_env/agent/main.py --node-ip-address=198.19.44.212 --runtime-env-agent-port=56317 --gcs-address=198.19.44.212:57594 --cluster-id-hex=1a1f12b856019ab7823cf8c9c76e73dc7eaa5439177dd16f1d3c0b76 --runtime-env-dir=/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B/session_2026-07-02_08-56-26_561774_2071629/runtime_resources --logging-rotate-bytes=536870912 --logging-rotate-backup-count=5 --log-dir=/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B/session_2026-07-02_08-56-26_561774_2071629/logs --temp-dir=/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B --stdout-filepath=/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B/session_2026-07-02_08-56-26_561774_2071629/logs/runtime_env_agent.out --stderr-filepath=/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B/session_2026-07-02_08-56-26_561774_2071629/logs/runtime_env_agent.err` (via SIGKILL) 2026-07-02 10:22:25,626 WARN scripts.py:1369 -- Stopped only 0 out of 8 Ray processes within the grace period 16 seconds. Set `-v` to see more details. Remaining processes [psutil.Process(pid=2074225, name='ray::IDLE', status='zombie'), psutil.Process(pid=2072135, name='ray::IDLE', status='zombie'), psutil.Process(pid=2072138, name='ray::IDLE', status='zombie'), psutil.Process(pid=2072091, name='python', status='zombie'), psutil.Process(pid=2072147, name='ray::IDLE', status='zombie'), psutil.Process(pid=2072134, name='ray::IDLE', status='zombie'), psutil.Process(pid=2072141, name='ray::IDLE', status='zombie'), psutil.Process(pid=2072137, name='ray::IDLE', status='zombie')] will be forcefully terminated. 2026-07-02 10:22:25,626 WARN scripts.py:1376 -- You can also use `--force` to forcefully terminate processes or set higher `--grace-period` to wait longer time for proper termination. Experiment: qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Config: baseline_grpo Task: datasets/sciknoweval/biology Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-GRPO-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_biology_grpo_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/biology +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.actor.policy_loss.loss_mode=vanilla actor_rollout_ref.actor.kl_loss_coef=0.0 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_biology_grpo_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/biology', 'EXPERIMENT': 'qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 10:22:36,329 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2099163) TaskRunner hostname: mole-workspace-rw, PID: 2099163 (TaskRunner pid=2099163) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2099163) 'calculate_entropy': False, (TaskRunner pid=2099163) 'calculate_sum_pi_squared': False, (TaskRunner pid=2099163) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2099163) 'async_save': False, (TaskRunner pid=2099163) 'load_contents': ['model', (TaskRunner pid=2099163) 'optimizer', (TaskRunner pid=2099163) 'extra'], (TaskRunner pid=2099163) 'save_contents': ['model', (TaskRunner pid=2099163) 'optimizer', (TaskRunner pid=2099163) 'extra']}, (TaskRunner pid=2099163) 'clip_ratio': 0.2, (TaskRunner pid=2099163) 'clip_ratio_c': 3.0, (TaskRunner pid=2099163) 'clip_ratio_high': 0.28, (TaskRunner pid=2099163) 'clip_ratio_low': 0.2, (TaskRunner pid=2099163) 'data_loader_seed': 42, (TaskRunner pid=2099163) 'entropy_checkpointing': False, (TaskRunner pid=2099163) 'entropy_coeff': 0, (TaskRunner pid=2099163) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2099163) 'freeze_vision_tower': False, (TaskRunner pid=2099163) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2099163) 'dtype': 'bfloat16', (TaskRunner pid=2099163) 'entropy_checkpointing': False, (TaskRunner pid=2099163) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2099163) 'forward_only': False, (TaskRunner pid=2099163) 'forward_prefetch': False, (TaskRunner pid=2099163) 'fsdp_size': -1, (TaskRunner pid=2099163) 'full_determinism': False, (TaskRunner pid=2099163) 'model_dtype': 'fp32', (TaskRunner pid=2099163) 'offload_policy': False, (TaskRunner pid=2099163) 'optimizer_offload': False, (TaskRunner pid=2099163) 'param_offload': False, (TaskRunner pid=2099163) 'reshard_after_forward': True, (TaskRunner pid=2099163) 'seed': 42, (TaskRunner pid=2099163) 'strategy': 'fsdp', (TaskRunner pid=2099163) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2099163) 'use_orig_params': False, (TaskRunner pid=2099163) 'use_torch_compile': True, (TaskRunner pid=2099163) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2099163) 'grad_clip': 1.0, (TaskRunner pid=2099163) 'kl_loss_coef': 0.0, (TaskRunner pid=2099163) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2099163) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2099163) 'loss_scale_factor': None, (TaskRunner pid=2099163) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2099163) 'betas': [0.9, 0.999], (TaskRunner pid=2099163) 'clip_grad': 1.0, (TaskRunner pid=2099163) 'lr': 1e-06, (TaskRunner pid=2099163) 'lr_scheduler_type': 'constant', (TaskRunner pid=2099163) 'lr_warmup_steps': 10, (TaskRunner pid=2099163) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2099163) 'min_lr_ratio': 0.0, (TaskRunner pid=2099163) 'num_cycles': 0.5, (TaskRunner pid=2099163) 'optimizer': 'AdamW', (TaskRunner pid=2099163) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2099163) 'override_optimizer_config': None, (TaskRunner pid=2099163) 'total_training_steps': -1, (TaskRunner pid=2099163) 'warmup_style': None, (TaskRunner pid=2099163) 'weight_decay': 0.01}, (TaskRunner pid=2099163) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2099163) 'clip_cov_lb': 1.0, (TaskRunner pid=2099163) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2099163) 'clip_cov_ub': 5.0, (TaskRunner pid=2099163) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2099163) 'loss_mode': 'vanilla', (TaskRunner pid=2099163) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2099163) 'ppo_epochs': 1, (TaskRunner pid=2099163) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2099163) 'ppo_micro_batch_size': None, (TaskRunner pid=2099163) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2099163) 'ppo_mini_batch_size': 8, (TaskRunner pid=2099163) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2099163) 'all_ranks': False, (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'ranks': [], (TaskRunner pid=2099163) 'save_path': 'outputs/profile', (TaskRunner pid=2099163) 'tool': None, (TaskRunner pid=2099163) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2099163) 'analysis': True, (TaskRunner pid=2099163) 'contents': [], (TaskRunner pid=2099163) 'discrete': False, (TaskRunner pid=2099163) 'level': 'level0'}, (TaskRunner pid=2099163) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2099163) 'discrete': False}, (TaskRunner pid=2099163) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2099163) 'step_end': None, (TaskRunner pid=2099163) 'step_start': 0}, (TaskRunner pid=2099163) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2099163) 'stack_depth': 32, (TaskRunner pid=2099163) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2099163) 'rollout_n': 8, (TaskRunner pid=2099163) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2099163) 'mode': 'disabled', (TaskRunner pid=2099163) 'record_file': None, (TaskRunner pid=2099163) 'replay_file': None}, (TaskRunner pid=2099163) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2099163) 'alpha': 0.5, (TaskRunner pid=2099163) 'distillation_add_tail': True, (TaskRunner pid=2099163) 'distillation_topk': 100, (TaskRunner pid=2099163) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2099163) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2099163) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'loss_weight': 0.0, (TaskRunner pid=2099163) 'mask': 'all'}, (TaskRunner pid=2099163) 'feedback_template': '\n' (TaskRunner pid=2099163) 'The ' (TaskRunner pid=2099163) 'following ' (TaskRunner pid=2099163) 'is ' (TaskRunner pid=2099163) 'feedback ' (TaskRunner pid=2099163) 'from ' (TaskRunner pid=2099163) 'your ' (TaskRunner pid=2099163) 'unsuccessful ' (TaskRunner pid=2099163) 'earlier ' (TaskRunner pid=2099163) 'attempt:\n' (TaskRunner pid=2099163) '\n' (TaskRunner pid=2099163) '{feedback_raw}', (TaskRunner pid=2099163) 'full_logit_distillation': True, (TaskRunner pid=2099163) 'include_environment_feedback': True, (TaskRunner pid=2099163) 'is_clip': 2, (TaskRunner pid=2099163) 'max_reprompt_len': 22528, (TaskRunner pid=2099163) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2099163) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2099163) '\n' (TaskRunner pid=2099163) 'Correctly ' (TaskRunner pid=2099163) 'solve ' (TaskRunner pid=2099163) 'the ' (TaskRunner pid=2099163) 'original ' (TaskRunner pid=2099163) 'question.', (TaskRunner pid=2099163) 'reprompt_truncation': 'right', (TaskRunner pid=2099163) 'solution_template': '\n' (TaskRunner pid=2099163) 'Correct ' (TaskRunner pid=2099163) 'solution:\n' (TaskRunner pid=2099163) '\n' (TaskRunner pid=2099163) '{successful_previous_attempt}', (TaskRunner pid=2099163) 'srpo_dynamic_weighting': False, (TaskRunner pid=2099163) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2099163) 'success_reward_threshold': 0.5, (TaskRunner pid=2099163) 'teacher_regularization': 'ema', (TaskRunner pid=2099163) 'teacher_update_rate': 0.0, (TaskRunner pid=2099163) 'token_reweight_decay_steps': None, (TaskRunner pid=2099163) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2099163) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2099163) 'shuffle': False, (TaskRunner pid=2099163) 'strategy': 'fsdp', (TaskRunner pid=2099163) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2099163) 'tau_neg': 1.05, (TaskRunner pid=2099163) 'tau_pos': 1.0, (TaskRunner pid=2099163) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2099163) 'use_dynamic_bsz': False, (TaskRunner pid=2099163) 'use_fused_kernels': False, (TaskRunner pid=2099163) 'use_kl_loss': False, (TaskRunner pid=2099163) 'use_prefix_grouper': False, (TaskRunner pid=2099163) 'use_remove_padding': True, (TaskRunner pid=2099163) 'use_torch_compile': True}, (TaskRunner pid=2099163) 'hybrid_engine': True, (TaskRunner pid=2099163) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2099163) 'custom_chat_template': None, (TaskRunner pid=2099163) 'enable_activation_offload': False, (TaskRunner pid=2099163) 'enable_gradient_checkpointing': True, (TaskRunner pid=2099163) 'exclude_modules': None, (TaskRunner pid=2099163) 'external_lib': None, (TaskRunner pid=2099163) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2099163) 'hf_config_path': None, (TaskRunner pid=2099163) 'lora_adapter_path': None, (TaskRunner pid=2099163) 'lora_alpha': 16, (TaskRunner pid=2099163) 'lora_rank': 0, (TaskRunner pid=2099163) 'override_config': {}, (TaskRunner pid=2099163) 'path': 'Qwen/Qwen3-4B', (TaskRunner pid=2099163) 'target_modules': 'all-linear', (TaskRunner pid=2099163) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2099163) 'num_shards': 4}, (TaskRunner pid=2099163) 'tokenizer_path': None, (TaskRunner pid=2099163) 'trust_remote_code': True, (TaskRunner pid=2099163) 'use_fused_kernels': False, (TaskRunner pid=2099163) 'use_liger': False, (TaskRunner pid=2099163) 'use_remove_padding': True, (TaskRunner pid=2099163) 'use_shm': False}, (TaskRunner pid=2099163) 'nccl_timeout': 600, (TaskRunner pid=2099163) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2099163) 'entropy_checkpointing': False, (TaskRunner pid=2099163) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2099163) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2099163) 'dtype': 'bfloat16', (TaskRunner pid=2099163) 'entropy_checkpointing': False, (TaskRunner pid=2099163) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2099163) 'forward_only': True, (TaskRunner pid=2099163) 'forward_prefetch': False, (TaskRunner pid=2099163) 'fsdp_size': -1, (TaskRunner pid=2099163) 'full_determinism': False, (TaskRunner pid=2099163) 'model_dtype': 'fp32', (TaskRunner pid=2099163) 'offload_policy': False, (TaskRunner pid=2099163) 'optimizer_offload': False, (TaskRunner pid=2099163) 'param_offload': False, (TaskRunner pid=2099163) 'reshard_after_forward': True, (TaskRunner pid=2099163) 'seed': 42, (TaskRunner pid=2099163) 'strategy': 'fsdp', (TaskRunner pid=2099163) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2099163) 'use_orig_params': False, (TaskRunner pid=2099163) 'use_torch_compile': True, (TaskRunner pid=2099163) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2099163) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2099163) 'log_prob_micro_batch_size': None, (TaskRunner pid=2099163) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2099163) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2099163) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2099163) 'all_ranks': False, (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'ranks': [], (TaskRunner pid=2099163) 'save_path': 'outputs/profile', (TaskRunner pid=2099163) 'tool': None, (TaskRunner pid=2099163) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2099163) 'analysis': True, (TaskRunner pid=2099163) 'contents': [], (TaskRunner pid=2099163) 'discrete': False, (TaskRunner pid=2099163) 'level': 'level0'}, (TaskRunner pid=2099163) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2099163) 'discrete': False}, (TaskRunner pid=2099163) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2099163) 'step_end': None, (TaskRunner pid=2099163) 'step_start': 0}, (TaskRunner pid=2099163) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2099163) 'stack_depth': 32, (TaskRunner pid=2099163) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2099163) 'rollout_n': 8, (TaskRunner pid=2099163) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2099163) 'mode': 'disabled', (TaskRunner pid=2099163) 'record_file': None, (TaskRunner pid=2099163) 'replay_file': None}, (TaskRunner pid=2099163) 'strategy': 'fsdp', (TaskRunner pid=2099163) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2099163) 'use_torch_compile': True}, (TaskRunner pid=2099163) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2099163) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2099163) 'agent_loop_config_path': None, (TaskRunner pid=2099163) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2099163) 'name': None, (TaskRunner pid=2099163) 'path': None}, (TaskRunner pid=2099163) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2099163) 'num_workers': 8}, (TaskRunner pid=2099163) 'calculate_log_probs': True, (TaskRunner pid=2099163) 'cudagraph_capture_sizes': None, (TaskRunner pid=2099163) 'data_parallel_size': 1, (TaskRunner pid=2099163) 'disable_log_stats': True, (TaskRunner pid=2099163) 'do_sample': True, (TaskRunner pid=2099163) 'dtype': 'bfloat16', (TaskRunner pid=2099163) 'enable_chunked_prefill': True, (TaskRunner pid=2099163) 'enable_prefix_caching': True, (TaskRunner pid=2099163) 'enable_rollout_routing_replay': False, (TaskRunner pid=2099163) 'enforce_eager': False, (TaskRunner pid=2099163) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2099163) 'expert_parallel_size': 1, (TaskRunner pid=2099163) 'free_cache_engine': True, (TaskRunner pid=2099163) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2099163) 'ignore_eos': False, (TaskRunner pid=2099163) 'layered_summon': False, (TaskRunner pid=2099163) 'load_format': 'dummy', (TaskRunner pid=2099163) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2099163) 'log_prob_micro_batch_size': None, (TaskRunner pid=2099163) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2099163) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2099163) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2099163) 'max_model_len': 10240, (TaskRunner pid=2099163) 'max_num_batched_tokens': 10240, (TaskRunner pid=2099163) 'max_num_seqs': 1024, (TaskRunner pid=2099163) 'mode': 'async', (TaskRunner pid=2099163) 'multi_stage_wake_up': False, (TaskRunner pid=2099163) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'format': 'hermes', (TaskRunner pid=2099163) 'interaction_config_path': None, (TaskRunner pid=2099163) 'max_assistant_turns': None, (TaskRunner pid=2099163) 'max_parallel_calls': 1, (TaskRunner pid=2099163) 'max_tool_response_length': 256, (TaskRunner pid=2099163) 'max_user_turns': None, (TaskRunner pid=2099163) 'num_repeat_rollouts': None, (TaskRunner pid=2099163) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2099163) 'tool_config_path': None, (TaskRunner pid=2099163) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2099163) 'use_inference_chat_template': False}, (TaskRunner pid=2099163) 'n': 8, (TaskRunner pid=2099163) 'name': 'vllm', (TaskRunner pid=2099163) 'over_sample_rate': 0, (TaskRunner pid=2099163) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2099163) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2099163) 'all_ranks': False, (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'ranks': [], (TaskRunner pid=2099163) 'save_path': 'outputs/profile', (TaskRunner pid=2099163) 'tool': None, (TaskRunner pid=2099163) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2099163) 'analysis': True, (TaskRunner pid=2099163) 'contents': [], (TaskRunner pid=2099163) 'discrete': False, (TaskRunner pid=2099163) 'level': 'level0'}, (TaskRunner pid=2099163) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2099163) 'discrete': False}, (TaskRunner pid=2099163) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2099163) 'step_end': None, (TaskRunner pid=2099163) 'step_start': 0}, (TaskRunner pid=2099163) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2099163) 'stack_depth': 32, (TaskRunner pid=2099163) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2099163) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2099163) 'port': 9090, (TaskRunner pid=2099163) 'served_model_name': 'Qwen/Qwen3-4B'}, (TaskRunner pid=2099163) 'prompt_length': 2048, (TaskRunner pid=2099163) 'quantization': None, (TaskRunner pid=2099163) 'quantization_config_file': None, (TaskRunner pid=2099163) 'response_length': 8192, (TaskRunner pid=2099163) 'scheduling_policy': 'fcfs', (TaskRunner pid=2099163) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2099163) 'skip_rollout': False, (TaskRunner pid=2099163) 'skip_tokenizer_init': True, (TaskRunner pid=2099163) 'temperature': 1.0, (TaskRunner pid=2099163) 'tensor_model_parallel_size': 2, (TaskRunner pid=2099163) 'top_k': -1, (TaskRunner pid=2099163) 'top_p': 1.0, (TaskRunner pid=2099163) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2099163) 'backend': None, (TaskRunner pid=2099163) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2099163) 'token2text': False}, (TaskRunner pid=2099163) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2099163) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2099163) 'do_sample': True, (TaskRunner pid=2099163) 'n': 16, (TaskRunner pid=2099163) 'temperature': 0.6, (TaskRunner pid=2099163) 'top_k': -1, (TaskRunner pid=2099163) 'top_p': 0.95}}}, (TaskRunner pid=2099163) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2099163) 'adv_estimator': 'grpo', (TaskRunner pid=2099163) 'gamma': 1.0, (TaskRunner pid=2099163) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2099163) 'horizon': 10000, (TaskRunner pid=2099163) 'kl_coef': 0.001, (TaskRunner pid=2099163) 'target_kl': 0.1, (TaskRunner pid=2099163) 'type': 'fixed'}, (TaskRunner pid=2099163) 'kl_penalty': 'kl', (TaskRunner pid=2099163) 'lam': 1.0, (TaskRunner pid=2099163) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2099163) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2099163) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2099163) 'loss_type': 'ppo_clip', (TaskRunner pid=2099163) 'rollout_is': 'token', (TaskRunner pid=2099163) 'rollout_is_batch_normalize': False, (TaskRunner pid=2099163) 'rollout_is_threshold': 2.0, (TaskRunner pid=2099163) 'rollout_rs': None, (TaskRunner pid=2099163) 'rollout_rs_threshold': None}, (TaskRunner pid=2099163) 'use_kl_in_reward': False, (TaskRunner pid=2099163) 'use_pf_ppo': False}, (TaskRunner pid=2099163) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2099163) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2099163) 'async_save': False, (TaskRunner pid=2099163) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2099163) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2099163) 'cliprange_value': 0.5, (TaskRunner pid=2099163) 'data_loader_seed': 42, (TaskRunner pid=2099163) 'enable': None, (TaskRunner pid=2099163) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2099163) 'forward_micro_batch_size': None, (TaskRunner pid=2099163) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2099163) 'grad_clip': 1.0, (TaskRunner pid=2099163) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2099163) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2099163) 'enable_activation_offload': False, (TaskRunner pid=2099163) 'enable_gradient_checkpointing': True, (TaskRunner pid=2099163) 'external_lib': None, (TaskRunner pid=2099163) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2099163) 'dtype': 'bfloat16', (TaskRunner pid=2099163) 'entropy_checkpointing': False, (TaskRunner pid=2099163) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2099163) 'forward_only': False, (TaskRunner pid=2099163) 'forward_prefetch': False, (TaskRunner pid=2099163) 'fsdp_size': -1, (TaskRunner pid=2099163) 'full_determinism': False, (TaskRunner pid=2099163) 'model_dtype': 'fp32', (TaskRunner pid=2099163) 'offload_policy': False, (TaskRunner pid=2099163) 'optimizer_offload': False, (TaskRunner pid=2099163) 'param_offload': False, (TaskRunner pid=2099163) 'reshard_after_forward': True, (TaskRunner pid=2099163) 'seed': 42, (TaskRunner pid=2099163) 'strategy': 'fsdp', (TaskRunner pid=2099163) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2099163) 'use_orig_params': False, (TaskRunner pid=2099163) 'use_torch_compile': True, (TaskRunner pid=2099163) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2099163) 'lora_alpha': 16, (TaskRunner pid=2099163) 'lora_rank': 0, (TaskRunner pid=2099163) 'override_config': {}, (TaskRunner pid=2099163) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2099163) 'target_modules': 'all-linear', (TaskRunner pid=2099163) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2099163) 'tokenizer_path': 'Qwen/Qwen3-4B', (TaskRunner pid=2099163) 'trust_remote_code': True, (TaskRunner pid=2099163) 'use_remove_padding': False, (TaskRunner pid=2099163) 'use_shm': False}, (TaskRunner pid=2099163) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2099163) 'betas': [0.9, 0.999], (TaskRunner pid=2099163) 'clip_grad': 1.0, (TaskRunner pid=2099163) 'lr': 1e-05, (TaskRunner pid=2099163) 'lr_scheduler_type': 'constant', (TaskRunner pid=2099163) 'lr_warmup_steps': -1, (TaskRunner pid=2099163) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2099163) 'min_lr_ratio': 0.0, (TaskRunner pid=2099163) 'num_cycles': 0.5, (TaskRunner pid=2099163) 'optimizer': 'AdamW', (TaskRunner pid=2099163) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2099163) 'override_optimizer_config': None, (TaskRunner pid=2099163) 'total_training_steps': -1, (TaskRunner pid=2099163) 'warmup_style': None, (TaskRunner pid=2099163) 'weight_decay': 0.01}, (TaskRunner pid=2099163) 'ppo_epochs': 1, (TaskRunner pid=2099163) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2099163) 'ppo_micro_batch_size': None, (TaskRunner pid=2099163) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2099163) 'ppo_mini_batch_size': 8, (TaskRunner pid=2099163) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2099163) 'all_ranks': False, (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'ranks': [], (TaskRunner pid=2099163) 'save_path': 'outputs/profile', (TaskRunner pid=2099163) 'tool': None, (TaskRunner pid=2099163) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2099163) 'analysis': True, (TaskRunner pid=2099163) 'contents': [], (TaskRunner pid=2099163) 'discrete': False, (TaskRunner pid=2099163) 'level': 'level0'}, (TaskRunner pid=2099163) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2099163) 'discrete': False}, (TaskRunner pid=2099163) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2099163) 'step_end': None, (TaskRunner pid=2099163) 'step_start': 0}, (TaskRunner pid=2099163) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2099163) 'stack_depth': 32, (TaskRunner pid=2099163) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2099163) 'rollout_n': 8, (TaskRunner pid=2099163) 'shuffle': False, (TaskRunner pid=2099163) 'strategy': 'fsdp', (TaskRunner pid=2099163) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2099163) 'use_dynamic_bsz': False}, (TaskRunner pid=2099163) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2099163) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2099163) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2099163) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2099163) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2099163) 'dataloader_num_workers': 8, (TaskRunner pid=2099163) 'filter_overlong_prompts': True, (TaskRunner pid=2099163) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2099163) 'image_key': 'images', (TaskRunner pid=2099163) 'image_patch_size': 14, (TaskRunner pid=2099163) 'max_prompt_length': 2048, (TaskRunner pid=2099163) 'max_response_length': 8192, (TaskRunner pid=2099163) 'prompt_key': 'prompt', (TaskRunner pid=2099163) 'return_full_prompt': False, (TaskRunner pid=2099163) 'return_multi_modal_inputs': True, (TaskRunner pid=2099163) 'return_raw_chat': True, (TaskRunner pid=2099163) 'return_raw_input_ids': False, (TaskRunner pid=2099163) 'reward_fn_key': 'data_source', (TaskRunner pid=2099163) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2099163) 'seed': None, (TaskRunner pid=2099163) 'shuffle': True, (TaskRunner pid=2099163) 'tokenizer': None, (TaskRunner pid=2099163) 'tool_config_path': None, (TaskRunner pid=2099163) 'train_batch_size': 32, (TaskRunner pid=2099163) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet'], (TaskRunner pid=2099163) 'train_max_samples': 3200, (TaskRunner pid=2099163) 'truncation': 'error', (TaskRunner pid=2099163) 'trust_remote_code': True, (TaskRunner pid=2099163) 'use_shm': False, (TaskRunner pid=2099163) 'val_batch_size': None, (TaskRunner pid=2099163) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet'], (TaskRunner pid=2099163) 'val_max_samples': -1, (TaskRunner pid=2099163) 'validation_shuffle': False, (TaskRunner pid=2099163) 'video_key': 'videos'}, (TaskRunner pid=2099163) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2099163) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2099163) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2099163) 'cuda-memory-usage': 'true', (TaskRunner pid=2099163) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2099163) 'discrete': False, (TaskRunner pid=2099163) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2099163) 'capture-range-end': None, (TaskRunner pid=2099163) 'cuda-graph-trace': 'graph', (TaskRunner pid=2099163) 'cuda-memory-usage': 'true', (TaskRunner pid=2099163) 'kill': 'none', (TaskRunner pid=2099163) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2099163) 'torch_memory': {'context': 'all', (TaskRunner pid=2099163) 'kw_args': {}, (TaskRunner pid=2099163) 'stack_depth': 32, (TaskRunner pid=2099163) 'stacks': 'all', (TaskRunner pid=2099163) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2099163) 'profile_continuous_steps': False, (TaskRunner pid=2099163) 'save_path': 'outputs/profile', (TaskRunner pid=2099163) 'steps': None, (TaskRunner pid=2099163) 'tool': None}, (TaskRunner pid=2099163) 'max_model_len': 10240, (TaskRunner pid=2099163) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_biology_grpo_Qwen3_4B', (TaskRunner pid=2099163) 'include_dashboard': False, (TaskRunner pid=2099163) 'num_cpus': None, (TaskRunner pid=2099163) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2099163) 'TASK': 'datasets/sciknoweval/biology', (TaskRunner pid=2099163) 'USER': 'root'}}}, (TaskRunner pid=2099163) 'timeline_json_file': None}, (TaskRunner pid=2099163) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2099163) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2099163) 'name': 'custom_reward_manager', (TaskRunner pid=2099163) 'path': None}, (TaskRunner pid=2099163) 'name': 'naive', (TaskRunner pid=2099163) 'source': 'register'}, (TaskRunner pid=2099163) 'reward_model': {'enable': False, (TaskRunner pid=2099163) 'enable_resource_pool': False, (TaskRunner pid=2099163) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2099163) 'launch_reward_fn_async': False, (TaskRunner pid=2099163) 'max_length': None, (TaskRunner pid=2099163) 'micro_batch_size': None, (TaskRunner pid=2099163) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2099163) 'model': {'external_lib': None, (TaskRunner pid=2099163) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2099163) 'forward_prefetch': False, (TaskRunner pid=2099163) 'fsdp_size': -1, (TaskRunner pid=2099163) 'param_offload': False, (TaskRunner pid=2099163) 'reshard_after_forward': True, (TaskRunner pid=2099163) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2099163) 'input_tokenizer': 'Qwen/Qwen3-4B', (TaskRunner pid=2099163) 'override_config': {}, (TaskRunner pid=2099163) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2099163) 'trust_remote_code': False, (TaskRunner pid=2099163) 'use_fused_kernels': False, (TaskRunner pid=2099163) 'use_remove_padding': False, (TaskRunner pid=2099163) 'use_shm': False}, (TaskRunner pid=2099163) 'n_gpus_per_node': 8, (TaskRunner pid=2099163) 'nnodes': 0, (TaskRunner pid=2099163) 'num_workers': 1, (TaskRunner pid=2099163) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2099163) 'all_ranks': False, (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'ranks': [], (TaskRunner pid=2099163) 'save_path': 'outputs/profile', (TaskRunner pid=2099163) 'tool': None, (TaskRunner pid=2099163) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2099163) 'analysis': True, (TaskRunner pid=2099163) 'contents': [], (TaskRunner pid=2099163) 'discrete': False, (TaskRunner pid=2099163) 'level': 'level0'}, (TaskRunner pid=2099163) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2099163) 'discrete': False}, (TaskRunner pid=2099163) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2099163) 'step_end': None, (TaskRunner pid=2099163) 'step_start': 0}, (TaskRunner pid=2099163) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2099163) 'stack_depth': 32, (TaskRunner pid=2099163) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2099163) 'reward_loop_class_name': None, (TaskRunner pid=2099163) 'reward_loop_module_path': None, (TaskRunner pid=2099163) 'reward_loop_source': 'register', (TaskRunner pid=2099163) 'reward_manager': 'naive', (TaskRunner pid=2099163) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2099163) 'cudagraph_capture_sizes': None, (TaskRunner pid=2099163) 'data_parallel_size': 1, (TaskRunner pid=2099163) 'disable_log_stats': True, (TaskRunner pid=2099163) 'dtype': 'bfloat16', (TaskRunner pid=2099163) 'enable_chunked_prefill': True, (TaskRunner pid=2099163) 'enable_prefix_caching': True, (TaskRunner pid=2099163) 'enforce_eager': True, (TaskRunner pid=2099163) 'engine_kwargs': {}, (TaskRunner pid=2099163) 'expert_parallel_size': 1, (TaskRunner pid=2099163) 'free_cache_engine': True, (TaskRunner pid=2099163) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2099163) 'limit_images': None, (TaskRunner pid=2099163) 'load_format': 'auto', (TaskRunner pid=2099163) 'max_model_len': None, (TaskRunner pid=2099163) 'max_num_batched_tokens': 8192, (TaskRunner pid=2099163) 'max_num_seqs': 1024, (TaskRunner pid=2099163) 'name': '???', (TaskRunner pid=2099163) 'prompt_length': 2048, (TaskRunner pid=2099163) 'response_length': 2048, (TaskRunner pid=2099163) 'skip_tokenizer_init': False, (TaskRunner pid=2099163) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2099163) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2099163) 'memory_limit_mb': 1024, (TaskRunner pid=2099163) 'url': None}, (TaskRunner pid=2099163) 'strategy': 'fsdp', (TaskRunner pid=2099163) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2099163) 'use_dynamic_bsz': False, (TaskRunner pid=2099163) 'use_reward_loop': False}, (TaskRunner pid=2099163) 'trainer': {'balance_batch': True, (TaskRunner pid=2099163) 'critic_warmup': 0, (TaskRunner pid=2099163) 'default_hdfs_dir': None, (TaskRunner pid=2099163) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2099163) 'del_local_ckpt_after_load': False, (TaskRunner pid=2099163) 'device': 'cuda', (TaskRunner pid=2099163) 'esi_redundant_time': 0, (TaskRunner pid=2099163) 'experiment_name': 'qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2099163) 'group_name': 'QWEN3-GRPO-generalization', (TaskRunner pid=2099163) 'log_val_generations': 0, (TaskRunner pid=2099163) 'logger': ['console', 'wandb'], (TaskRunner pid=2099163) 'max_actor_ckpt_to_keep': 1, (TaskRunner pid=2099163) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2099163) 'n_gpus_per_node': 8, (TaskRunner pid=2099163) 'nnodes': 1, (TaskRunner pid=2099163) 'project_name': 'SDPO-root', (TaskRunner pid=2099163) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2099163) 'resume_from_path': None, (TaskRunner pid=2099163) 'resume_mode': 'auto', (TaskRunner pid=2099163) 'rollout_data_dir': None, (TaskRunner pid=2099163) 'save_freq': 10, (TaskRunner pid=2099163) 'test_freq': 10, (TaskRunner pid=2099163) 'total_epochs': 30, (TaskRunner pid=2099163) 'total_training_steps': 100, (TaskRunner pid=2099163) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2099163) 'val_before_train': False, (TaskRunner pid=2099163) 'val_only': False, (TaskRunner pid=2099163) 'validation_data_dir': None}, (TaskRunner pid=2099163) 'transfer_queue': {'enable': False}, (TaskRunner pid=2099163) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/biology', (TaskRunner pid=2099163) 'dir': '/users/root/SDPO', (TaskRunner pid=2099163) 'log_dir': '/users/root/output', (TaskRunner pid=2099163) 'task': 'datasets/sciknoweval/biology'}} (TaskRunner pid=2099163) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2099163) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2099163) use_critic=need_critic(config), (TaskRunner pid=2099163) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2099163) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2099163) Using dataset class: RLHFDataset (TaskRunner pid=2099163) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2099163) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2099163) dataset len: 450 (TaskRunner pid=2099163) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2099163) WARNING:2026-07-02 10:22:45,455:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2099163) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/450 [00:00 (TaskRunner pid=2099163) bind role actor_rollout method chat_completion to class .WorkerDict'> (TaskRunner pid=2099163) bind role actor_rollout method generate to class .WorkerDict'> (TaskRunner pid=2099163) bind role actor_rollout method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2099163) bind role actor_rollout method sleep to class .WorkerDict'> (TaskRunner pid=2099163) bind role actor_rollout method wake_up to class .WorkerDict'> (TaskRunner pid=2099163) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 50/50 [00:00<00:00, 79.25 examples/s] (TaskRunner pid=2099163) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2099163) self.use_critic = need_critic(self.config) (WorkerDict pid=2099540) [W702 10:22:53.887126096 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:53015 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2099539) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2099546) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2099546) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2099543) [W702 10:22:54.133676817 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:53015 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2099539) Model config after override: Qwen3Config { (WorkerDict pid=2099539) "architectures": [ (WorkerDict pid=2099539) "Qwen3ForCausalLM" (WorkerDict pid=2099539) ], (WorkerDict pid=2099539) "attention_bias": false, (WorkerDict pid=2099539) "attention_dropout": 0.0, (WorkerDict pid=2099539) "dtype": "bfloat16", (WorkerDict pid=2099539) "eos_token_id": 151645, (WorkerDict pid=2099539) "head_dim": 128, (WorkerDict pid=2099539) "hidden_act": "silu", (WorkerDict pid=2099539) "hidden_size": 2560, (WorkerDict pid=2099539) "initializer_range": 0.02, (WorkerDict pid=2099539) "intermediate_size": 9728, (WorkerDict pid=2099539) "layer_types": [ (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention" (WorkerDict pid=2099539) ], (WorkerDict pid=2099539) "max_position_embeddings": 40960, (WorkerDict pid=2099539) "max_window_layers": 36, (WorkerDict pid=2099539) "model_type": "qwen3", (WorkerDict pid=2099539) "num_attention_heads": 32, (WorkerDict pid=2099539) "num_hidden_layers": 36, (WorkerDict pid=2099539) "num_key_value_heads": 8, (WorkerDict pid=2099539) "pad_token_id": 151643, (WorkerDict pid=2099539) "rms_norm_eps": 1e-06, (WorkerDict pid=2099539) "rope_scaling": null, (WorkerDict pid=2099539) "rope_theta": 1000000, (WorkerDict pid=2099539) "sliding_window": null, (WorkerDict pid=2099539) "tie_word_embeddings": true, (WorkerDict pid=2099539) "transformers_version": "4.57.1", (WorkerDict pid=2099539) "use_cache": true, (WorkerDict pid=2099539) "use_sliding_window": false, (WorkerDict pid=2099539) "vocab_size": 151936 (WorkerDict pid=2099539) } (WorkerDict pid=2099539) (WorkerDict pid=2099540) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2099540) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2099540) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2099546) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2099539) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2099540) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2099540) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2099539) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.93s/it] [repeated 14x across cluster] (WorkerDict pid=2099539) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.97s/it] [repeated 6x across cluster] (WorkerDict pid=2099539) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2099539) Actor use_remove_padding=True (WorkerDict pid=2099539) Actor use_fused_kernels=False (WorkerDict pid=2099539) Actor use_prefix_grouper=False (WorkerDict pid=2099543) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2099543) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2099540) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2099540) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2099539) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2099544) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2099544) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2099542) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2099542) warnings.warn( (WorkerDict pid=2099544) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster] (WorkerDict pid=2099544) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster] (TaskRunner pid=2099163) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2099163) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2099543) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster] (WorkerDict pid=2099543) warnings.warn( [repeated 7x across cluster] (vLLMHttpServer pid=2100408) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2100408) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2100407) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2100407) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2100407) ['serve', (vLLMHttpServer pid=2100407) 'Qwen/Qwen3-4B', (vLLMHttpServer pid=2100407) '--dtype', (vLLMHttpServer pid=2100407) 'bfloat16', (vLLMHttpServer pid=2100407) '--load_format', (vLLMHttpServer pid=2100407) 'dummy', (vLLMHttpServer pid=2100407) '--trust_remote_code', (vLLMHttpServer pid=2100407) '--max_model_len', (vLLMHttpServer pid=2100407) '40960', (vLLMHttpServer pid=2100407) '--max_num_seqs', (vLLMHttpServer pid=2100407) '1024', (vLLMHttpServer pid=2100407) '--enable_chunked_prefill', (vLLMHttpServer pid=2100407) '--max_num_batched_tokens', (vLLMHttpServer pid=2100407) '10240', (vLLMHttpServer pid=2100407) '--enable_prefix_caching', (vLLMHttpServer pid=2100407) '--enable_sleep_mode', (vLLMHttpServer pid=2100407) '--logprobs_mode', (vLLMHttpServer pid=2100407) 'processed_logprobs', (vLLMHttpServer pid=2100407) '--disable_custom_all_reduce', (vLLMHttpServer pid=2100407) '--gpu_memory_utilization', (vLLMHttpServer pid=2100407) '0.8', (vLLMHttpServer pid=2100407) '--disable_log_stats', (vLLMHttpServer pid=2100407) '--tensor_parallel_size', (vLLMHttpServer pid=2100407) '2', (vLLMHttpServer pid=2100407) '--seed', (vLLMHttpServer pid=2100407) '0', (vLLMHttpServer pid=2100407) '--override_generation_config', (vLLMHttpServer pid=2100407) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2100407) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2100407) '--hf_overrides', (vLLMHttpServer pid=2100407) '{}', (vLLMHttpServer pid=2100407) '--scheduling_policy', (vLLMHttpServer pid=2100407) 'fcfs'] (WorkerDict pid=2099544) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (vLLMHttpServer pid=2100408) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2100408) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2100407) WARNING 07-02 10:23:39 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2099540) WARNING 07-02 10:23:47 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2100409) WARNING 07-02 10:23:39 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2099541) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2099540) [rank1]:W0702 10:23:58.556000 2099540 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph (WorkerDict pid=2099540) [rank1]:W0702 10:23:58.556000 2099540 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last): (WorkerDict pid=2099540) [rank1]:W0702 10:23:58.556000 2099540 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates (WorkerDict pid=2099540) [rank1]:W0702 10:23:58.556000 2099540 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f: (WorkerDict pid=2099540) [rank1]:W0702 10:23:58.556000 2099540 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (WorkerDict pid=2099540) [rank1]:W0702 10:23:58.556000 2099540 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/fxgraph/vs/fvscv5dg7y3y3dcup4ddfpgbmpglg73u5zi54vpf2q4orgekbtq7/.2099541.136359408498368.tmp' (vLLMHttpServer pid=2100410) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 2x across cluster] (vLLMHttpServer pid=2100410) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 2x across cluster] (vLLMHttpServer pid=2100410) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2100410) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2099539) 2026-07-02 10:24:01,345 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2099539) 2026-07-02 10:24:01,370 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2099539) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) user (TaskRunner pid=2099163) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2099163) (TaskRunner pid=2099163) A: -0.01 (TaskRunner pid=2099163) B: 1.69 (TaskRunner pid=2099163) C: 2.49 (TaskRunner pid=2099163) D: 0.45 (TaskRunner pid=2099163) Please reason step by step. (TaskRunner pid=2099163) assistant (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) [response] (TaskRunner pid=2099163) The folding stability score of a protein sequence is typically determined through computational methods such as energy minimization, molecular dynamics simulations, or machine learning models trained on known protein structures. However, without access to specific computational tools or databases that can calculate the folding stability score for the given sequence, it is not possible to determine the exact score. The provided options (A: -0.01, B: 1.69, C: 2.49, D: 0.45) are likely hypothetical values, and there is no standard or universally accepted method to calculate this score from the sequence alone. Therefore, the correct answer cannot be determined based on the information provided. (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) B (TaskRunner pid=2099163) (TaskRunner pid=2099163) [ground_truth] B (TaskRunner pid=2099163) [score] 1.0 (TaskRunner pid=2099163) [acc] 1.0 (TaskRunner pid=2099163) [pred] B (TaskRunner pid=2099163) [incorrect_format] 1 (TaskRunner pid=2099163) [feedback] (TaskRunner pid=2099163) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2099163) step:10 - global_seqlen/min:12666 - global_seqlen/max:26901 - global_seqlen/minmax_diff:14235 - global_seqlen/balanced_min:18269 - global_seqlen/balanced_max:18382 - global_seqlen/mean:18293.25 - actor/entropy:0.192046120762825 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9020472168922424 - training/rollout_probs_diff_mean:0.00564963323995471 - training/rollout_probs_diff_std:0.0170443132519722 - training/rollout_actor_probs_pearson_corr:0.9967436790466309 - rollout_corr/rollout_is_mean:0.9999064207077026 - rollout_corr/rollout_is_ratio_fraction_high:6.567023228853941e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00027581496397033334 - rollout_corr/rollout_is_max:3.955087184906006 - rollout_corr/rollout_is_min:0.025797156617045403 - rollout_corr/rollout_is_std:0.045984067022800446 - rollout_corr/rollout_is_eff_sample_size:0.997889452616177 - rollout_corr/rollout_is_seq_mean:0.9998351335525513 - rollout_corr/rollout_is_seq_std:0.0035011477302759886 - rollout_corr/rollout_is_seq_max:1.0109554529190063 - rollout_corr/rollout_is_seq_min:0.9882628321647644 - rollout_corr/rollout_is_seq_max_deviation:0.011737167835235596 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2451128889806569) - rollout_corr/training_log_ppl:np.float64(0.21647670843231026) - rollout_corr/kl:np.float64(0.00146516180673828) - rollout_corr/k3_kl:np.float64(0.0013313775715459997) - rollout_corr/rollout_ppl:np.float64(1.2432878208346665) - rollout_corr/rollout_log_ppl:np.float64(0.21501154491852503) - rollout_corr/log_ppl_diff:np.float64(0.0014651635137852281) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028982661897316575) - rollout_corr/log_ppl_diff_max:np.float64(0.014495670795440674) - rollout_corr/log_ppl_diff_min:np.float64(-0.009721904993057251) - rollout_corr/ppl_ratio:np.float64(1.0014725218061358) - rollout_corr/chi2_token:np.float64(0.0023953597992658615) - rollout_corr/chi2_seq:np.float64(1.8521191228646785) - actor/pg_loss:np.float64(-0.0003558297175914049) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015348545880442543) - actor/ppo_kl:np.float64(3.0528818129482715e-05) - actor/pg_clipfrac_lower:np.float64(1.3610627320304047e-05) - actor/grad_norm:np.float64(0.7854970991611481) - perf/mfu/actor:np.float64(0.052699151040128595) - perf/max_memory_allocated_gb:np.float64(123.58113050460815) - perf/max_memory_reserved_gb:np.float64(129.3828125) - perf/cpu_memory_used_gb:np.float64(105.79932022094727) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.32625) - val-aux/sciknoweval/reward/std@16:np.float64(0.26460828298638245) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.44315999999999994) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.23037433915304933) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.21118000000000003) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.21955087215134436) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.3275) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.266025449388232) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.53736) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.16763225376953209) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.12658) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.14989660984549052) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.3408) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.22535669809810538) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.60002) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.10074795072317197) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.07074) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08389263468998247) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.34962000000000004) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.17799526365500928) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.6341) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.052678976599392024) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.03935999999999999) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.044642921667641916) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.35164) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.13534911056216117) - val-aux/sciknoweval/score/mean@16:np.float64(0.32625) - val-aux/sciknoweval/score/std@16:np.float64(0.26460828298638245) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.44315999999999994) - val-aux/sciknoweval/score/best@2/std:np.float64(0.23037433915304933) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.21118000000000003) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.21955087215134436) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.3275) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.266025449388232) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.53736) - val-aux/sciknoweval/score/best@4/std:np.float64(0.16763225376953209) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.12658) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.14989660984549052) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.3408) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.22535669809810538) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.60002) - val-aux/sciknoweval/score/best@8/std:np.float64(0.10074795072317197) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.07074) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08389263468998247) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.34962000000000004) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.17799526365500928) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.6341) - val-aux/sciknoweval/score/best@16/std:np.float64(0.052678976599392024) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.03935999999999999) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.044642921667641916) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.35164) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.13534911056216117) - val-core/sciknoweval/acc/mean@16:np.float64(0.32625) - val-aux/sciknoweval/acc/std@16:np.float64(0.26460828298638245) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.44315999999999994) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.23037433915304933) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.21118000000000003) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.21955087215134436) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.3275) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.266025449388232) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.53736) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.16763225376953209) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.12658) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.14989660984549052) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.3408) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.22535669809810538) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.60002) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.10074795072317197) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.07074) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08389263468998247) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.34962000000000004) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.17799526365500928) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.6341) - val-core/sciknoweval/acc/best@16/std:np.float64(0.052678976599392024) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.03935999999999999) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.044642921667641916) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.35164) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.13534911056216117) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.97875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.051687330456951015) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9963400000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.020813886045474284) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.96174) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.06405299669111861) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9787599999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.05137839781051554) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9997800000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.003963742623570884) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.93396) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.07265674213411831) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99216) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.03188606302646686) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.90036) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.06915111790505142) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99804) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.015647532793846276) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.86874) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.05102367714768366) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99992) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.002157810766489876) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.25 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.25 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036463395226746798 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0036463395226746798 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:297.4140625 - response_length/max:1569.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:297.4140625 - response_length_non_aborted/max:1569.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.25 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015241466462612152 - timing_s/agent_loop/generate_sequences/min:np.float64(1.17083459533751) - timing_s/agent_loop/generate_sequences/max:np.float64(9.452028851956129) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0214375181894866) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.452028851956129) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1569 - timing_s/gen:15.476865164935589 - timing_s/reward:0.045249899849295616 - timing_s/old_log_prob:2.511210959404707 - timing_s/adv:0.15661288052797318 - timing_s/update_actor:10.127284951508045 - timing_s/step:28.350071949884295 - timing_s/testing:84.79899363964796 - timing_s/save_checkpoint:6.5641801711171865 - timing_s/stop_profile:0.00011404044926166534 - timing_per_token_ms/gen:0.20327386016096546 - timing_per_token_ms/update_actor:0.06920096860527822 - timing_per_token_ms/adv:0.0010701548421410436 - perf/total_num_tokens:146346 - perf/time_per_step:28.350071949884295 - perf/throughput:645.2629126422609 (TaskRunner pid=2099163) Training Progress: 10%|█ | 10/100 [10:46<1:44:31, 69.68s/it] (TaskRunner pid=2099163) step:11 - global_seqlen/min:16676 - global_seqlen/max:36461 - global_seqlen/minmax_diff:19785 - global_seqlen/balanced_min:22290 - global_seqlen/balanced_max:27044 - global_seqlen/mean:22898.125 - actor/entropy:0.16072751581668854 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.32411471009254456 - training/rollout_probs_diff_mean:0.0046199411153793335 - training/rollout_probs_diff_std:0.014347893185913563 - training/rollout_actor_probs_pearson_corr:0.9974071383476257 - rollout_corr/rollout_is_mean:0.9998364448547363 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00011153358354931697 - rollout_corr/rollout_is_max:1.8010543584823608 - rollout_corr/rollout_is_min:0.18950599431991577 - rollout_corr/rollout_is_std:0.040931250900030136 - rollout_corr/rollout_is_eff_sample_size:0.9983268407518708 - rollout_corr/rollout_is_seq_mean:0.9997417330741882 - rollout_corr/rollout_is_seq_std:0.0031297144014388323 - rollout_corr/rollout_is_seq_max:1.0107784271240234 - rollout_corr/rollout_is_seq_min:0.9905034303665161 - rollout_corr/rollout_is_seq_max_deviation:0.010778427124023438 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2281816471368074) - rollout_corr/training_log_ppl:np.float64(0.2024953516520327) - rollout_corr/kl:np.float64(0.001424252133454651) - rollout_corr/k3_kl:np.float64(0.0011623206726625313) - rollout_corr/rollout_ppl:np.float64(1.226403083652258) - rollout_corr/rollout_log_ppl:np.float64(0.20107109841774218) - rollout_corr/log_ppl_diff:np.float64(0.0014242532342905179) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025687814049888402) - rollout_corr/log_ppl_diff_max:np.float64(0.01213720440864563) - rollout_corr/log_ppl_diff_min:np.float64(-0.009074568748474121) - rollout_corr/ppl_ratio:np.float64(1.0014300209004432) - rollout_corr/chi2_token:np.float64(0.0017990793567150831) - rollout_corr/chi2_seq:np.float64(0.5897399557288736) - actor/pg_loss:np.float64(0.00016360939480364323) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016948245472576673) - actor/ppo_kl:np.float64(9.670323938681946e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6953910440206528) - perf/mfu/actor:np.float64(0.06338107402915627) - perf/max_memory_allocated_gb:np.float64(123.58113050460815) - perf/max_memory_reserved_gb:np.float64(129.3828125) - perf/cpu_memory_used_gb:np.float64(111.92782211303711) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.44140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03447655215859413 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.03447655215859413 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:385.25390625 - response_length/max:6669.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:385.25390625 - response_length_non_aborted/max:6669.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.3125 - prompt_length/max:500.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.39831817150116e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2415860313922167) - timing_s/agent_loop/generate_sequences/max:np.float64(33.86591459810734) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7203849007346435) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(33.86591459810734) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:432 - timing_s/agent_loop/slowest/response_length:6669 - timing_s/gen:39.782698061317205 - timing_s/reward:0.061702121049165726 - timing_s/old_log_prob:2.763470709323883 - timing_s/adv:0.13837043195962906 - timing_s/update_actor:10.400962248444557 - timing_s/step:53.240667348727584 - timing_s/stop_profile:0.00013394653797149658 - timing_per_token_ms/gen:0.403373364373305 - timing_per_token_ms/update_actor:0.05677846029120592 - timing_per_token_ms/adv:0.0007553589647603737 - perf/total_num_tokens:183185 - perf/time_per_step:53.240667348727584 - perf/throughput:430.0871146114071 (TaskRunner pid=2099163) Training Progress: 11%|█ | 11/100 [11:39<1:35:55, 64.66s/it] (TaskRunner pid=2099163) step:12 - global_seqlen/min:20156 - global_seqlen/max:34994 - global_seqlen/minmax_diff:14838 - global_seqlen/balanced_min:26866 - global_seqlen/balanced_max:32121 - global_seqlen/mean:28280.875 - actor/entropy:0.12687404453754425 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5569577217102051 - training/rollout_probs_diff_mean:0.00370349595323205 - training/rollout_probs_diff_std:0.013641891069710255 - training/rollout_actor_probs_pearson_corr:0.9971849918365479 - rollout_corr/rollout_is_mean:0.999925971031189 - rollout_corr/rollout_is_ratio_fraction_high:2.9888440622016788e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00020174698147457093 - rollout_corr/rollout_is_max:2.8711585998535156 - rollout_corr/rollout_is_min:0.05395948514342308 - rollout_corr/rollout_is_std:0.038417838513851166 - rollout_corr/rollout_is_eff_sample_size:0.9985261260875207 - rollout_corr/rollout_is_seq_mean:0.9998483657836914 - rollout_corr/rollout_is_seq_std:0.002559172920882702 - rollout_corr/rollout_is_seq_max:1.0071240663528442 - rollout_corr/rollout_is_seq_min:0.9910182356834412 - rollout_corr/rollout_is_seq_max_deviation:0.008981764316558838 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2076720832847059) - rollout_corr/training_log_ppl:np.float64(0.1855281984044268) - rollout_corr/kl:np.float64(0.0015311053801809749) - rollout_corr/k3_kl:np.float64(0.0011974603096120973) - rollout_corr/rollout_ppl:np.float64(1.205766570288688) - rollout_corr/rollout_log_ppl:np.float64(0.1839970920154883) - rollout_corr/log_ppl_diff:np.float64(0.0015311063889384968) - rollout_corr/log_ppl_abs_diff:np.float64(0.002373620836806367) - rollout_corr/log_ppl_diff_max:np.float64(0.015954285860061646) - rollout_corr/log_ppl_diff_min:np.float64(-0.005851656198501587) - rollout_corr/ppl_ratio:np.float64(1.0015365811996162) - rollout_corr/chi2_token:np.float64(0.0016909947153180838) - rollout_corr/chi2_seq:np.float64(0.3893565512262285) - actor/pg_loss:np.float64(0.0003301744582131505) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001968728217093485) - actor/ppo_kl:np.float64(0.0002511454556559656) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.8092714995145798) - perf/mfu/actor:np.float64(0.07718086515597372) - perf/max_memory_allocated_gb:np.float64(123.58113050460815) - perf/max_memory_reserved_gb:np.float64(129.3828125) - perf/cpu_memory_used_gb:np.float64(110.70772171020508) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.45703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.45703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.019058551639318466 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.019058551639318466 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:522.77734375 - response_length/max:8192.0 - response_length/min:106.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:522.77734375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:361.0 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.196065366268158e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2030387092381716) - timing_s/agent_loop/generate_sequences/max:np.float64(49.85608519241214) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.7326578640859225) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.85608519241214) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.627652468159795 - timing_s/reward:0.06745622679591179 - timing_s/old_log_prob:2.655902164056897 - timing_s/adv:0.12508256547152996 - timing_s/update_actor:10.585711555555463 - timing_s/step:69.14367071911693 - timing_s/stop_profile:0.00011152029037475586 - timing_per_token_ms/gen:0.41565595764927254 - timing_per_token_ms/update_actor:0.04678829578096268 - timing_per_token_ms/adv:0.0005528584488259732 - perf/total_num_tokens:226247 - perf/time_per_step:69.14367071911693 - perf/throughput:409.016106693058 (TaskRunner pid=2099163) Training Progress: 12%|█▏ | 12/100 [12:48<1:36:51, 66.04s/it] (TaskRunner pid=2099163) step:13 - global_seqlen/min:14490 - global_seqlen/max:27440 - global_seqlen/minmax_diff:12950 - global_seqlen/balanced_min:21991 - global_seqlen/balanced_max:22034 - global_seqlen/mean:22017.0 - actor/entropy:0.1795348972082138 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3799695372581482 - training/rollout_probs_diff_mean:0.004953271709382534 - training/rollout_probs_diff_std:0.014711440540850163 - training/rollout_actor_probs_pearson_corr:0.9974781274795532 - rollout_corr/rollout_is_mean:1.0001226663589478 - rollout_corr/rollout_is_ratio_fraction_high:4.09433341701515e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.165083661675453e-05 - rollout_corr/rollout_is_max:2.9943156242370605 - rollout_corr/rollout_is_min:0.36588054895401 - rollout_corr/rollout_is_std:0.04241306707262993 - rollout_corr/rollout_is_eff_sample_size:0.9982047182816606 - rollout_corr/rollout_is_seq_mean:1.0001139640808105 - rollout_corr/rollout_is_seq_std:0.003129767719656229 - rollout_corr/rollout_is_seq_max:1.0108983516693115 - rollout_corr/rollout_is_seq_min:0.9917095303535461 - rollout_corr/rollout_is_seq_max_deviation:0.010898351669311523 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2477836473844945) - rollout_corr/training_log_ppl:np.float64(0.2179029462276958) - rollout_corr/kl:np.float64(0.0012722289181801294) - rollout_corr/k3_kl:np.float64(0.0011847673267197933) - rollout_corr/rollout_ppl:np.float64(1.2461294089443982) - rollout_corr/rollout_log_ppl:np.float64(0.2166307152074296) - rollout_corr/log_ppl_diff:np.float64(0.0012722310202661902) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027390209725126624) - rollout_corr/log_ppl_diff_max:np.float64(0.01292330026626587) - rollout_corr/log_ppl_diff_min:np.float64(-0.009444236755371094) - rollout_corr/ppl_ratio:np.float64(1.0012791764456779) - rollout_corr/chi2_token:np.float64(0.002235463121905923) - rollout_corr/chi2_seq:np.float64(1.5766265538986772) - actor/pg_loss:np.float64(-0.0002433034824207425) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002133346634764166) - actor/ppo_kl:np.float64(0.00032201276727761297) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.8120427280664444) - perf/mfu/actor:np.float64(0.06433296210788333) - perf/max_memory_allocated_gb:np.float64(123.58113050460815) - perf/max_memory_reserved_gb:np.float64(129.3828125) - perf/cpu_memory_used_gb:np.float64(110.85671615600586) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007056328002363443 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007056328002363443 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:381.625 - response_length/max:1800.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:381.625 - response_length_non_aborted/max:1800.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:306.40625 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013713166117668152 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2604616954922676) - timing_s/agent_loop/generate_sequences/max:np.float64(11.544187435880303) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8230461675921106) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.544187435880303) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:415 - timing_s/agent_loop/slowest/response_length:1800 - timing_s/gen:17.578441470861435 - timing_s/reward:0.061520595103502274 - timing_s/old_log_prob:2.475315697491169 - timing_s/adv:0.12054650858044624 - timing_s/update_actor:9.704637175425887 - timing_s/step:30.0266327932477 - timing_s/stop_profile:0.0001079123467206955 - timing_per_token_ms/gen:0.17993000195362588 - timing_per_token_ms/update_actor:0.05509740868093909 - timing_per_token_ms/adv:0.0006843944939163274 - perf/total_num_tokens:176136 - perf/time_per_step:30.0266327932477 - perf/throughput:733.2490509875325 (TaskRunner pid=2099163) Training Progress: 13%|█▎ | 13/100 [13:18<1:19:57, 55.14s/it] (TaskRunner pid=2099163) step:14 - global_seqlen/min:12976 - global_seqlen/max:34060 - global_seqlen/minmax_diff:21084 - global_seqlen/balanced_min:22596 - global_seqlen/balanced_max:22654 - global_seqlen/mean:22633.375 - actor/entropy:0.16276216506958008 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31926417350769043 - training/rollout_probs_diff_mean:0.004311886616051197 - training/rollout_probs_diff_std:0.013316148892045021 - training/rollout_actor_probs_pearson_corr:0.9977449178695679 - rollout_corr/rollout_is_mean:0.9999713897705078 - rollout_corr/rollout_is_ratio_fraction_high:3.8242746086325496e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.692480383208022e-05 - rollout_corr/rollout_is_max:2.2574832439422607 - rollout_corr/rollout_is_min:0.28150883316993713 - rollout_corr/rollout_is_std:0.040303174406290054 - rollout_corr/rollout_is_eff_sample_size:0.9983781696179648 - rollout_corr/rollout_is_seq_mean:0.9999383687973022 - rollout_corr/rollout_is_seq_std:0.0024390218313783407 - rollout_corr/rollout_is_seq_max:1.008014440536499 - rollout_corr/rollout_is_seq_min:0.9899135828018188 - rollout_corr/rollout_is_seq_max_deviation:0.010086417198181152 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2327516474761069) - rollout_corr/training_log_ppl:np.float64(0.20615991155500524) - rollout_corr/kl:np.float64(0.0009014457496316908) - rollout_corr/k3_kl:np.float64(0.0010392485548891273) - rollout_corr/rollout_ppl:np.float64(1.231638588476926) - rollout_corr/rollout_log_ppl:np.float64(0.2052584641496651) - rollout_corr/log_ppl_diff:np.float64(0.00090144740534015) - rollout_corr/log_ppl_abs_diff:np.float64(0.00216034529148601) - rollout_corr/log_ppl_diff_max:np.float64(0.010936662554740906) - rollout_corr/log_ppl_diff_min:np.float64(-0.008887723088264465) - rollout_corr/ppl_ratio:np.float64(1.0009056760463864) - rollout_corr/chi2_token:np.float64(0.002428637119010091) - rollout_corr/chi2_seq:np.float64(1.1035329343285412) - actor/pg_loss:np.float64(0.0004473176086321473) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011060538008678122) - actor/ppo_kl:np.float64(-0.0001549605085848782) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.575629249215126) - perf/mfu/actor:np.float64(0.0652755181152363) - perf/max_memory_allocated_gb:np.float64(123.58113050460815) - perf/max_memory_reserved_gb:np.float64(129.3828125) - perf/cpu_memory_used_gb:np.float64(109.99267196655273) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.26953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.26953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.010551412589848042 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.010551412589848042 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:408.57421875 - response_length/max:2188.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:408.57421875 - response_length_non_aborted/max:2188.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:498.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.820424020290375e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0610912814736366) - timing_s/agent_loop/generate_sequences/max:np.float64(13.642793396487832) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8376170305346022) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.642793396487832) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:2188 - timing_s/gen:19.731356034055352 - timing_s/reward:0.06039970554411411 - timing_s/old_log_prob:2.502957135438919 - timing_s/adv:0.12832614220678806 - timing_s/update_actor:9.91347792558372 - timing_s/step:32.42584792152047 - timing_s/stop_profile:0.00012178905308246613 - timing_per_token_ms/gen:0.18864530841871363 - timing_per_token_ms/update_actor:0.05475032957735932 - timing_per_token_ms/adv:0.0007087218665288985 - perf/total_num_tokens:181067 - perf/time_per_step:32.42584792152047 - perf/throughput:698.0041063160178 (TaskRunner pid=2099163) Training Progress: 14%|█▍ | 14/100 [13:51<1:09:13, 48.30s/it] (TaskRunner pid=2099163) step:15 - global_seqlen/min:15918 - global_seqlen/max:28170 - global_seqlen/minmax_diff:12252 - global_seqlen/balanced_min:19653 - global_seqlen/balanced_max:19679 - global_seqlen/mean:19673.25 - actor/entropy:0.19017662107944489 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6307171583175659 - training/rollout_probs_diff_mean:0.005055586341768503 - training/rollout_probs_diff_std:0.014726489782333374 - training/rollout_actor_probs_pearson_corr:0.9975889921188354 - rollout_corr/rollout_is_mean:1.0000306367874146 - rollout_corr/rollout_is_ratio_fraction_high:2.42359619733179e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013329778448678553 - rollout_corr/rollout_is_max:2.5952959060668945 - rollout_corr/rollout_is_min:0.22091145813465118 - rollout_corr/rollout_is_std:0.04240182042121887 - rollout_corr/rollout_is_eff_sample_size:0.9982053121902402 - rollout_corr/rollout_is_seq_mean:1.0000522136688232 - rollout_corr/rollout_is_seq_std:0.0030719421338289976 - rollout_corr/rollout_is_seq_max:1.0085277557373047 - rollout_corr/rollout_is_seq_min:0.9906306266784668 - rollout_corr/rollout_is_seq_max_deviation:0.009369373321533203 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2559455754235387) - rollout_corr/training_log_ppl:np.float64(0.22441870825423393) - rollout_corr/kl:np.float64(0.001127970071593154) - rollout_corr/k3_kl:np.float64(0.001125821751145395) - rollout_corr/rollout_ppl:np.float64(1.254524188581854) - rollout_corr/rollout_log_ppl:np.float64(0.22329073723813053) - rollout_corr/log_ppl_diff:np.float64(0.0011279710161034018) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024534623080398887) - rollout_corr/log_ppl_diff_max:np.float64(0.014268159866333008) - rollout_corr/log_ppl_diff_min:np.float64(-0.008966684341430664) - rollout_corr/ppl_ratio:np.float64(1.00113360513933) - rollout_corr/chi2_token:np.float64(0.0023321015760302544) - rollout_corr/chi2_seq:np.float64(0.6067947293631732) - actor/pg_loss:np.float64(-0.00015570060350000858) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014229919640911248) - actor/ppo_kl:np.float64(0.00016318686129324078) - actor/pg_clipfrac_lower:np.float64(5.865240382263437e-06) - actor/grad_norm:np.float64(0.6899215131998062) - perf/mfu/actor:np.float64(0.05693643757760258) - perf/max_memory_allocated_gb:np.float64(123.58113050460815) - perf/max_memory_reserved_gb:np.float64(129.3828125) - perf/cpu_memory_used_gb:np.float64(110.72046184539795) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:1 - critic/score/mean:0.34765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.012832941487431526 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.012832941487431526 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:322.3515625 - response_length/max:1553.0 - response_length/min:91.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:322.3515625 - response_length_non_aborted/max:1553.0 - response_length_non_aborted/min:91.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:292.4375 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003503747284412384 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2059585116803646) - timing_s/agent_loop/generate_sequences/max:np.float64(9.883644664660096) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.228372771802242) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.883644664660096) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:1553 - timing_s/gen:15.838685717433691 - timing_s/reward:0.051979394629597664 - timing_s/old_log_prob:2.7615113761276007 - timing_s/adv:0.1317354552447796 - timing_s/update_actor:9.826176561415195 - timing_s/step:28.72526820190251 - timing_s/stop_profile:0.0001147240400314331 - timing_per_token_ms/gen:0.1919328871989735 - timing_per_token_ms/update_actor:0.06243361265560594 - timing_per_token_ms/adv:0.0008370214329405384 - perf/total_num_tokens:157386 - perf/time_per_step:28.72526820190251 - perf/throughput:684.876111920759 (TaskRunner pid=2099163) Training Progress: 15%|█▌ | 15/100 [14:20<1:00:12, 42.50s/it] (TaskRunner pid=2099163) step:16 - global_seqlen/min:17621 - global_seqlen/max:49777 - global_seqlen/minmax_diff:32156 - global_seqlen/balanced_min:23638 - global_seqlen/balanced_max:29431 - global_seqlen/mean:27074.0 - actor/entropy:0.12440122663974762 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7334084510803223 - training/rollout_probs_diff_mean:0.003286235500127077 - training/rollout_probs_diff_std:0.012201991863548756 - training/rollout_actor_probs_pearson_corr:0.997710645198822 - rollout_corr/rollout_is_mean:0.999930739402771 - rollout_corr/rollout_is_ratio_fraction_high:7.548765097453725e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.793888314859942e-05 - rollout_corr/rollout_is_max:2.695401906967163 - rollout_corr/rollout_is_min:0.02908293716609478 - rollout_corr/rollout_is_std:0.034535329788923264 - rollout_corr/rollout_is_eff_sample_size:0.9988087318703696 - rollout_corr/rollout_is_seq_mean:0.9999657869338989 - rollout_corr/rollout_is_seq_std:0.002763616619631648 - rollout_corr/rollout_is_seq_max:1.0094408988952637 - rollout_corr/rollout_is_seq_min:0.9909818768501282 - rollout_corr/rollout_is_seq_max_deviation:0.009440898895263672 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2334859780967236) - rollout_corr/training_log_ppl:np.float64(0.20614377667880035) - rollout_corr/kl:np.float64(0.0010040062433311903) - rollout_corr/k3_kl:np.float64(0.0012267602566424785) - rollout_corr/rollout_ppl:np.float64(1.2322044824250042) - rollout_corr/rollout_log_ppl:np.float64(0.20513977024347696) - rollout_corr/log_ppl_diff:np.float64(0.0010040064353233902) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025442858877795516) - rollout_corr/log_ppl_diff_max:np.float64(0.014998257160186768) - rollout_corr/log_ppl_diff_min:np.float64(-0.008154168725013733) - rollout_corr/ppl_ratio:np.float64(1.0010099613573402) - rollout_corr/chi2_token:np.float64(0.002799068344756961) - rollout_corr/chi2_seq:np.float64(0.6977890518028289) - actor/pg_loss:np.float64(0.0005094839725643396) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0022031578653241013) - actor/ppo_kl:np.float64(-6.424526176118661e-05) - actor/pg_clipfrac_lower:np.float64(1.4852661479380913e-05) - actor/grad_norm:np.float64(0.759627491235733) - perf/mfu/actor:np.float64(0.07361790070915576) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(110.84224700927734) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:1 - critic/score/mean:0.546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0865541398525238 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0865541398525238 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:517.46875 - response_length/max:8192.0 - response_length/min:85.0 - response_length/clip_ratio:0.015625 - response_length_non_aborted/mean:517.46875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.015625 - response/aborted_ratio:0.0 - prompt_length/mean:328.59375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.716115891933441e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6790603306144476) - timing_s/agent_loop/generate_sequences/max:np.float64(51.488350415602326) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.263302836421644) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(51.488350415602326) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:57.936794666573405 - timing_s/reward:0.06626211851835251 - timing_s/old_log_prob:2.6734151635318995 - timing_s/adv:0.12808487936854362 - timing_s/update_actor:10.97300206311047 - timing_s/step:71.86347073316574 - timing_s/stop_profile:0.00010935962200164795 - timing_per_token_ms/gen:0.43735124906828166 - timing_per_token_ms/update_actor:0.05066208384017171 - timing_per_token_ms/adv:0.0005913647751003898 - perf/total_num_tokens:216592 - perf/time_per_step:71.86347073316574 - perf/throughput:376.7421712837628 (TaskRunner pid=2099163) Training Progress: 16%|█▌ | 16/100 [15:32<1:11:53, 51.36s/it] (TaskRunner pid=2099163) step:17 - global_seqlen/min:11966 - global_seqlen/max:39775 - global_seqlen/minmax_diff:27809 - global_seqlen/balanced_min:20593 - global_seqlen/balanced_max:26443 - global_seqlen/mean:22197.625 - actor/entropy:0.15671347081661224 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34504491090774536 - training/rollout_probs_diff_mean:0.003963190596550703 - training/rollout_probs_diff_std:0.012594124302268028 - training/rollout_actor_probs_pearson_corr:0.9979807138442993 - rollout_corr/rollout_is_mean:0.9999871253967285 - rollout_corr/rollout_is_ratio_fraction_high:9.640877578931395e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.640877578931395e-06 - rollout_corr/rollout_is_max:2.198077917098999 - rollout_corr/rollout_is_min:0.4044226408004761 - rollout_corr/rollout_is_std:0.03752783685922623 - rollout_corr/rollout_is_eff_sample_size:0.9985936420813145 - rollout_corr/rollout_is_seq_mean:0.9999175071716309 - rollout_corr/rollout_is_seq_std:0.0031496852170675993 - rollout_corr/rollout_is_seq_max:1.0094223022460938 - rollout_corr/rollout_is_seq_min:0.9900813102722168 - rollout_corr/rollout_is_seq_max_deviation:0.009918689727783203 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2560751209966838) - rollout_corr/training_log_ppl:np.float64(0.22452797560981708) - rollout_corr/kl:np.float64(0.0013395728782059635) - rollout_corr/k3_kl:np.float64(0.0012000854812015405) - rollout_corr/rollout_ppl:np.float64(1.2543696821667254) - rollout_corr/rollout_log_ppl:np.float64(0.2231884019674908) - rollout_corr/log_ppl_diff:np.float64(0.001339573642326286) - rollout_corr/log_ppl_abs_diff:np.float64(0.002711944489419693) - rollout_corr/log_ppl_diff_max:np.float64(0.01098942756652832) - rollout_corr/log_ppl_diff_min:np.float64(-0.0078660249710083) - rollout_corr/ppl_ratio:np.float64(1.0013459962792695) - rollout_corr/chi2_token:np.float64(0.0021593261044472456) - rollout_corr/chi2_seq:np.float64(0.8300942496862262) - actor/pg_loss:np.float64(0.0006044314941391349) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015271970464709739) - actor/ppo_kl:np.float64(0.00025955110856701324) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.73015196621418) - perf/mfu/actor:np.float64(0.06131922719985038) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(110.87406206130981) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:1 - critic/score/mean:0.28515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.28515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.06812485307455063 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.06812485307455063 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:405.17578125 - response_length/max:8192.0 - response_length/min:85.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:405.17578125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:288.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014323554933071136 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9844553098082542) - timing_s/agent_loop/generate_sequences/max:np.float64(49.343952156603336) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.660478450343362) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.343952156603336) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.40832286886871 - timing_s/reward:0.06138620153069496 - timing_s/old_log_prob:2.554391412064433 - timing_s/adv:0.17900926247239113 - timing_s/update_actor:10.665983552113175 - timing_s/step:68.9551175814122 - timing_s/stop_profile:0.0001098737120628357 - timing_per_token_ms/gen:0.5341848432766325 - timing_per_token_ms/update_actor:0.06006263931452788 - timing_per_token_ms/adv:0.001008042878868748 - perf/total_num_tokens:177581 - perf/time_per_step:68.9551175814122 - perf/throughput:321.9141055599284 (TaskRunner pid=2099163) Training Progress: 17%|█▋ | 17/100 [16:41<1:18:23, 56.67s/it] (TaskRunner pid=2099163) step:18 - global_seqlen/min:11273 - global_seqlen/max:36015 - global_seqlen/minmax_diff:24742 - global_seqlen/balanced_min:21872 - global_seqlen/balanced_max:27887 - global_seqlen/mean:22719.125 - actor/entropy:0.1572161316871643 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7516714334487915 - training/rollout_probs_diff_mean:0.003971869591623545 - training/rollout_probs_diff_std:0.01294691488146782 - training/rollout_actor_probs_pearson_corr:0.9978299140930176 - rollout_corr/rollout_is_mean:1.0000897645950317 - rollout_corr/rollout_is_ratio_fraction_high:3.9406924770446494e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.9703462385223247e-05 - rollout_corr/rollout_is_max:2.524366617202759 - rollout_corr/rollout_is_min:0.09788884967565536 - rollout_corr/rollout_is_std:0.037467435002326965 - rollout_corr/rollout_is_eff_sample_size:0.998598397072895 - rollout_corr/rollout_is_seq_mean:1.0000863075256348 - rollout_corr/rollout_is_seq_std:0.0033069117926061153 - rollout_corr/rollout_is_seq_max:1.0111510753631592 - rollout_corr/rollout_is_seq_min:0.9837021827697754 - rollout_corr/rollout_is_seq_max_deviation:0.01629781723022461 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2480866289697587) - rollout_corr/training_log_ppl:np.float64(0.2177488712950435) - rollout_corr/kl:np.float64(0.0012160134455356797) - rollout_corr/k3_kl:np.float64(0.0012282074120264497) - rollout_corr/rollout_ppl:np.float64(1.2464866428636014) - rollout_corr/rollout_log_ppl:np.float64(0.21653285443244386) - rollout_corr/log_ppl_diff:np.float64(0.0012160168625996448) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027001788184861653) - rollout_corr/log_ppl_diff_max:np.float64(0.01847335696220398) - rollout_corr/log_ppl_diff_min:np.float64(-0.008770108222961426) - rollout_corr/ppl_ratio:np.float64(1.0012231352739036) - rollout_corr/chi2_token:np.float64(0.0024735613260418177) - rollout_corr/chi2_seq:np.float64(1.276715693064034) - actor/pg_loss:np.float64(0.0005491708870977163) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0019209535612390027) - actor/ppo_kl:np.float64(0.00030758063345004416) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.7345165759325027) - perf/mfu/actor:np.float64(0.06209275037892638) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(110.8076171875) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:1 - critic/score/mean:0.3359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02325008623301983 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02325008623301983 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:396.50390625 - response_length/max:8192.0 - response_length/min:92.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:396.50390625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:313.46875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001363120973110199 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0805652625858784) - timing_s/agent_loop/generate_sequences/max:np.float64(49.01962508261204) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7108128744512214) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.01962508261204) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:54.90000583231449 - timing_s/reward:0.06206467002630234 - timing_s/old_log_prob:2.6254268158227205 - timing_s/adv:0.12246500886976719 - timing_s/update_actor:10.576374450698495 - timing_s/step:68.37408596090972 - timing_s/stop_profile:0.00011956505477428436 - timing_per_token_ms/gen:0.5408601136132652 - timing_per_token_ms/update_actor:0.05819092092399297 - timing_per_token_ms/adv:0.0006737991057631357 - perf/total_num_tokens:181753 - perf/time_per_step:68.37408596090972 - perf/throughput:332.27683676808186 (TaskRunner pid=2099163) Training Progress: 18%|█▊ | 18/100 [17:49<1:22:16, 60.20s/it] (TaskRunner pid=2099163) step:19 - global_seqlen/min:14639 - global_seqlen/max:26849 - global_seqlen/minmax_diff:12210 - global_seqlen/balanced_min:21052 - global_seqlen/balanced_max:21251 - global_seqlen/mean:21094.25 - actor/entropy:0.18874603509902954 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7417160868644714 - training/rollout_probs_diff_mean:0.004822679795324802 - training/rollout_probs_diff_std:0.014397743158042431 - training/rollout_actor_probs_pearson_corr:0.9977278113365173 - rollout_corr/rollout_is_mean:0.9999311566352844 - rollout_corr/rollout_is_ratio_fraction_high:1.0811979336722288e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014055574138183147 - rollout_corr/rollout_is_max:2.0913357734680176 - rollout_corr/rollout_is_min:0.040095582604408264 - rollout_corr/rollout_is_std:0.0410170778632164 - rollout_corr/rollout_is_eff_sample_size:0.998320068587282 - rollout_corr/rollout_is_seq_mean:1.0000311136245728 - rollout_corr/rollout_is_seq_std:0.0029985024593770504 - rollout_corr/rollout_is_seq_max:1.010836124420166 - rollout_corr/rollout_is_seq_min:0.987970232963562 - rollout_corr/rollout_is_seq_max_deviation:0.012029767036437988 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.255211434327066) - rollout_corr/training_log_ppl:np.float64(0.22383148300286848) - rollout_corr/kl:np.float64(0.0012053162868781442) - rollout_corr/k3_kl:np.float64(0.0011591173564511337) - rollout_corr/rollout_ppl:np.float64(1.253677450120449) - rollout_corr/rollout_log_ppl:np.float64(0.2226261663454352) - rollout_corr/log_ppl_diff:np.float64(0.0012053166574332863) - rollout_corr/log_ppl_abs_diff:np.float64(0.002684240898815915) - rollout_corr/log_ppl_diff_max:np.float64(0.01323661208152771) - rollout_corr/log_ppl_diff_min:np.float64(-0.010094761848449707) - rollout_corr/ppl_ratio:np.float64(1.0012121815234423) - rollout_corr/chi2_token:np.float64(0.0021450589410960674) - rollout_corr/chi2_seq:np.float64(1.123602865729481) - actor/pg_loss:np.float64(0.00028023216873407364) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009424093339021056) - actor/ppo_kl:np.float64(0.0002521510247390779) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5630562603473663) - perf/mfu/actor:np.float64(0.06105586170219322) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(110.85847091674805) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:1 - critic/score/mean:0.40625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012870309874415398 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012870309874415398 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:361.2890625 - response_length/max:1883.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:361.2890625 - response_length_non_aborted/max:1883.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:297.90625 - prompt_length/max:498.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.548874080181122e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0813855342566967) - timing_s/agent_loop/generate_sequences/max:np.float64(11.688677107915282) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5753794663687586) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.688677107915282) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:424 - timing_s/agent_loop/slowest/response_length:1883 - timing_s/gen:17.670184712857008 - timing_s/reward:0.06079593487083912 - timing_s/old_log_prob:2.4477415289729834 - timing_s/adv:0.12715278938412666 - timing_s/update_actor:9.766590259969234 - timing_s/step:30.157636089250445 - timing_s/stop_profile:0.00011079944670200348 - timing_per_token_ms/gen:0.19104967794201544 - timing_per_token_ms/update_actor:0.05787471858426606 - timing_per_token_ms/adv:0.0007534801508949516 - perf/total_num_tokens:168754 - perf/time_per_step:30.157636089250445 - perf/throughput:699.4662956198663 (TaskRunner pid=2099163) Training Progress: 19%|█▉ | 19/100 [18:20<1:09:06, 51.19s/it] (TaskRunner pid=2099163) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2099163) validation generation end (TaskRunner pid=2099163) [prompt] system (TaskRunner pid=2099163) (TaskRunner pid=2099163) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) user (TaskRunner pid=2099163) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2099163) (TaskRunner pid=2099163) A: -0.01 (TaskRunner pid=2099163) B: 1.69 (TaskRunner pid=2099163) C: 2.49 (TaskRunner pid=2099163) D: 0.45 (TaskRunner pid=2099163) Please reason step by step. (TaskRunner pid=2099163) assistant (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) [response] (TaskRunner pid=2099163) To determine the folding stability score of the given protein sequence, one would typically use a computational method or software that calculates the stability based on the amino acid sequence. This score is often derived from energy calculations that consider factors such as hydrogen bonding, hydrophobic interactions, and van der Waals forces. However, without access to specific computational tools or databases that provide such scores for the given sequence, it is not possible to calculate the exact folding stability score. Therefore, the correct answer cannot be determined from the information provided. (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) D (TaskRunner pid=2099163) (TaskRunner pid=2099163) [ground_truth] B (TaskRunner pid=2099163) [score] 0.0 (TaskRunner pid=2099163) [acc] 0.0 (TaskRunner pid=2099163) [pred] D (TaskRunner pid=2099163) [incorrect_format] 1 (TaskRunner pid=2099163) [feedback] (TaskRunner pid=2099163) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (WorkerDict pid=2099539) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10/actor (TaskRunner pid=2099163) step:20 - global_seqlen/min:12298 - global_seqlen/max:30479 - global_seqlen/minmax_diff:18181 - global_seqlen/balanced_min:21810 - global_seqlen/balanced_max:21871 - global_seqlen/mean:21839.375 - actor/entropy:0.18792696297168732 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6172393560409546 - training/rollout_probs_diff_mean:0.004675295669585466 - training/rollout_probs_diff_std:0.013787684962153435 - training/rollout_actor_probs_pearson_corr:0.9978795051574707 - rollout_corr/rollout_is_mean:1.0000436305999756 - rollout_corr/rollout_is_ratio_fraction_high:5.289270120556466e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010578540241112933 - rollout_corr/rollout_is_max:2.6348955631256104 - rollout_corr/rollout_is_min:0.11010421812534332 - rollout_corr/rollout_is_std:0.040804363787174225 - rollout_corr/rollout_is_eff_sample_size:0.998337890270291 - rollout_corr/rollout_is_seq_mean:1.000036597251892 - rollout_corr/rollout_is_seq_std:0.0025871675461530685 - rollout_corr/rollout_is_seq_max:1.009564757347107 - rollout_corr/rollout_is_seq_min:0.9896543622016907 - rollout_corr/rollout_is_seq_max_deviation:0.010345637798309326 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2705717207863927) - rollout_corr/training_log_ppl:np.float64(0.2352731467835838) - rollout_corr/kl:np.float64(0.00124775865324378) - rollout_corr/k3_kl:np.float64(0.001086297427093541) - rollout_corr/rollout_ppl:np.float64(1.2689147526398301) - rollout_corr/rollout_log_ppl:np.float64(0.23402538590016775) - rollout_corr/log_ppl_diff:np.float64(0.0012477608834160492) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022727364994352683) - rollout_corr/log_ppl_diff_max:np.float64(0.012642994523048401) - rollout_corr/log_ppl_diff_min:np.float64(-0.007874488830566406) - rollout_corr/ppl_ratio:np.float64(1.0012527105864137) - rollout_corr/chi2_token:np.float64(0.0018538862932473421) - rollout_corr/chi2_seq:np.float64(0.5317726316861808) - actor/pg_loss:np.float64(-4.45549376308918e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001396393277673269) - actor/ppo_kl:np.float64(0.0002731104437279086) - actor/pg_clipfrac_lower:np.float64(7.614522473886609e-06) - actor/grad_norm:np.float64(0.5186081603169441) - perf/mfu/actor:np.float64(0.06376455117784033) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(110.90739440917969) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.3575) - val-aux/sciknoweval/reward/std@16:np.float64(0.2845272325003602) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.4821000000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.24641293690229316) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.23617999999999997) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.23810031156810768) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.35924000000000006) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2844295099662947) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.57886) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.17803900782144286) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.13665999999999998) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.16515577551517271) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.38042000000000004) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.23790217711971715) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.64558) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.11228107051862579) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.07342) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08529636863102308) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.39696) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.18341197265688536) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.6885199999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.06561633303370275) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.04667999999999999) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03078753156755329) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.4047799999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.13095914178704102) - val-aux/sciknoweval/score/mean@16:np.float64(0.3575) - val-aux/sciknoweval/score/std@16:np.float64(0.2845272325003602) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.4821000000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.24641293690229316) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.23617999999999997) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.23810031156810768) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.35924000000000006) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2844295099662947) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.57886) - val-aux/sciknoweval/score/best@4/std:np.float64(0.17803900782144286) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.13665999999999998) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.16515577551517271) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.38042000000000004) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.23790217711971715) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.64558) - val-aux/sciknoweval/score/best@8/std:np.float64(0.11228107051862579) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.07342) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08529636863102308) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.39696) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.18341197265688536) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.6885199999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.06561633303370275) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.04667999999999999) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03078753156755329) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.4047799999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.13095914178704102) - val-core/sciknoweval/acc/mean@16:np.float64(0.3575) - val-aux/sciknoweval/acc/std@16:np.float64(0.2845272325003602) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.4821000000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.24641293690229316) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.23617999999999997) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.23810031156810768) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.35924000000000006) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2844295099662947) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.57886) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.17803900782144286) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.13665999999999998) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.16515577551517271) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.38042000000000004) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.23790217711971715) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.64558) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.11228107051862579) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.07342) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08529636863102308) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.39696) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.18341197265688536) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.6885199999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.06561633303370275) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.04667999999999999) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03078753156755329) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.4047799999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.13095914178704102) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.04906017482220243) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99724) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.017382402655940103) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9658800000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.06179985484869913) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.98052) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.04963623931745396) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.99988) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.001544538766104626) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9399599999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.07259562722286844) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99358) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.02825209907076314) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.90684) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.07207216802183986) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99862) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.010856231271909862) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8730199999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.05670226605201907) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99992) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.001787064632295094) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:1 - critic/score/mean:0.3359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0017243020702153444 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0017243020702153444 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:369.26171875 - response_length/max:1936.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:369.26171875 - response_length_non_aborted/max:1936.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.21875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011486373841762543 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1602171901613474) - timing_s/agent_loop/generate_sequences/max:np.float64(12.04150439798832) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.66143894479319) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.04150439798832) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:1936 - timing_s/gen:17.918031726032495 - timing_s/reward:0.06160684861242771 - timing_s/old_log_prob:2.4307430386543274 - timing_s/adv:0.11749370023608208 - timing_s/update_actor:9.837341289967299 - timing_s/step:30.453087836503983 - timing_s/testing:84.54373020865023 - timing_s/save_checkpoint:6.91516363248229 - timing_s/stop_profile:0.00016070157289505005 - timing_per_token_ms/gen:0.18954662201851769 - timing_per_token_ms/update_actor:0.05630507563727956 - timing_per_token_ms/adv:0.0006724877671412418 - perf/total_num_tokens:174715 - perf/time_per_step:30.453087836503983 - perf/throughput:717.148130174873 (TaskRunner pid=2099163) Training Progress: 20%|██ | 20/100 [20:22<1:36:35, 72.45s/it] (TaskRunner pid=2099163) step:21 - global_seqlen/min:13059 - global_seqlen/max:30220 - global_seqlen/minmax_diff:17161 - global_seqlen/balanced_min:20197 - global_seqlen/balanced_max:20234 - global_seqlen/mean:20222.625 - actor/entropy:0.2039741724729538 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2915150225162506 - training/rollout_probs_diff_mean:0.005005266051739454 - training/rollout_probs_diff_std:0.013932553119957447 - training/rollout_actor_probs_pearson_corr:0.9979515671730042 - rollout_corr/rollout_is_mean:0.9997883439064026 - rollout_corr/rollout_is_ratio_fraction_high:1.1129289305245038e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.3997433185577393 - rollout_corr/rollout_is_min:0.5131387114524841 - rollout_corr/rollout_is_std:0.04085326939821243 - rollout_corr/rollout_is_eff_sample_size:0.9983332565714974 - rollout_corr/rollout_is_seq_mean:0.9999094605445862 - rollout_corr/rollout_is_seq_std:0.0028472074773162603 - rollout_corr/rollout_is_seq_max:1.011277198791504 - rollout_corr/rollout_is_seq_min:0.9916235208511353 - rollout_corr/rollout_is_seq_max_deviation:0.011277198791503906 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2804982154630125) - rollout_corr/training_log_ppl:np.float64(0.2435035327434889) - rollout_corr/kl:np.float64(0.0010719042137878887) - rollout_corr/k3_kl:np.float64(0.0011022668704754324) - rollout_corr/rollout_ppl:np.float64(1.2791178887709975) - rollout_corr/rollout_log_ppl:np.float64(0.24243162775383098) - rollout_corr/log_ppl_diff:np.float64(0.0010719049896579236) - rollout_corr/log_ppl_abs_diff:np.float64(0.002581465640105307) - rollout_corr/log_ppl_diff_max:np.float64(0.009952932596206665) - rollout_corr/log_ppl_diff_min:np.float64(-0.010108351707458496) - rollout_corr/ppl_ratio:np.float64(1.0010773693211377) - rollout_corr/chi2_token:np.float64(0.0022938339971005917) - rollout_corr/chi2_seq:np.float64(1.2692464508581907) - actor/pg_loss:np.float64(0.0004962929524481297) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0018514558880724508) - actor/ppo_kl:np.float64(6.505578917792576e-07) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.7440718114376068) - perf/mfu/actor:np.float64(0.05869452164939222) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(105.24159622192383) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:1 - critic/score/mean:0.40625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0015914883697405457 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0015914883697405457 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:350.98828125 - response_length/max:1700.0 - response_length/min:102.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:350.98828125 - response_length_non_aborted/max:1700.0 - response_length_non_aborted/min:102.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.96875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.297362804412842e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.264117207378149) - timing_s/agent_loop/generate_sequences/max:np.float64(10.401357980445027) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5284495416854043) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.401357980445027) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1700 - timing_s/gen:16.154271429404616 - timing_s/reward:0.05521078407764435 - timing_s/old_log_prob:2.5382862519472837 - timing_s/adv:0.14272014237940311 - timing_s/update_actor:9.808351395651698 - timing_s/step:28.792014434933662 - timing_s/stop_profile:0.00015562400221824646 - timing_per_token_ms/gen:0.17978555451019573 - timing_per_token_ms/update_actor:0.060627338164875345 - timing_per_token_ms/adv:0.0008821811113752735 - perf/total_num_tokens:161781 - perf/time_per_step:28.792014434933662 - perf/throughput:702.3692296938998 (TaskRunner pid=2099163) Training Progress: 21%|██ | 21/100 [20:50<1:18:09, 59.36s/it] (TaskRunner pid=2099163) step:22 - global_seqlen/min:18027 - global_seqlen/max:36166 - global_seqlen/minmax_diff:18139 - global_seqlen/balanced_min:25060 - global_seqlen/balanced_max:28210 - global_seqlen/mean:25542.25 - actor/entropy:0.17578984797000885 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4386941194534302 - training/rollout_probs_diff_mean:0.004188444931060076 - training/rollout_probs_diff_std:0.012796149589121342 - training/rollout_actor_probs_pearson_corr:0.9980850219726562 - rollout_corr/rollout_is_mean:1.000032901763916 - rollout_corr/rollout_is_ratio_fraction_high:1.730672738631256e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010384036431787536 - rollout_corr/rollout_is_max:2.1636812686920166 - rollout_corr/rollout_is_min:0.21608301997184753 - rollout_corr/rollout_is_std:0.038447305560112 - rollout_corr/rollout_is_eff_sample_size:0.9985239866454121 - rollout_corr/rollout_is_seq_mean:0.9999626874923706 - rollout_corr/rollout_is_seq_std:0.0027287364937365055 - rollout_corr/rollout_is_seq_max:1.0086798667907715 - rollout_corr/rollout_is_seq_min:0.990822970867157 - rollout_corr/rollout_is_seq_max_deviation:0.009177029132843018 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.265514174476266) - rollout_corr/training_log_ppl:np.float64(0.23049156166962348) - rollout_corr/kl:np.float64(0.001140852084887456) - rollout_corr/k3_kl:np.float64(0.0010895646327071518) - rollout_corr/rollout_ppl:np.float64(1.2640215558931231) - rollout_corr/rollout_log_ppl:np.float64(0.22935070820676629) - rollout_corr/log_ppl_diff:np.float64(0.0011408534628571942) - rollout_corr/log_ppl_abs_diff:np.float64(0.002326310393982567) - rollout_corr/log_ppl_diff_max:np.float64(0.016114696860313416) - rollout_corr/log_ppl_diff_min:np.float64(-0.005415067076683044) - rollout_corr/ppl_ratio:np.float64(1.0011460261885077) - rollout_corr/chi2_token:np.float64(0.0020373596344143152) - rollout_corr/chi2_seq:np.float64(1.662300607888028) - actor/pg_loss:np.float64(-4.751852247864008e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013797034152958076) - actor/ppo_kl:np.float64(0.0001431179896584922) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6352894306182861) - perf/mfu/actor:np.float64(0.07072783775794689) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(107.67041778564453) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:1 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.024204539135098457 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.024204539135098457 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:451.4140625 - response_length/max:5103.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:451.4140625 - response_length_non_aborted/max:5103.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:346.78125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.669573485851288e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2452805154025555) - timing_s/agent_loop/generate_sequences/max:np.float64(27.396138545125723) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.402181767611182) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.396138545125723) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:5103 - timing_s/gen:33.46731911972165 - timing_s/reward:0.068209582939744 - timing_s/old_log_prob:2.507258925586939 - timing_s/adv:0.13793614879250526 - timing_s/update_actor:10.272249413654208 - timing_s/step:46.539894139394164 - timing_s/stop_profile:3.1733885407447815e-05 - timing_per_token_ms/gen:0.2896048798023715 - timing_per_token_ms/update_actor:0.05027087185767801 - timing_per_token_ms/adv:0.0006750391449094406 - perf/total_num_tokens:204338 - perf/time_per_step:46.539894139394164 - perf/throughput:548.8248409740044 (TaskRunner pid=2099163) Training Progress: 22%|██▏ | 22/100 [21:37<1:12:10, 55.52s/it] (TaskRunner pid=2099163) step:23 - global_seqlen/min:18253 - global_seqlen/max:33113 - global_seqlen/minmax_diff:14860 - global_seqlen/balanced_min:23569 - global_seqlen/balanced_max:24069 - global_seqlen/mean:23647.375 - actor/entropy:0.20427009463310242 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.403209388256073 - training/rollout_probs_diff_mean:0.004876875318586826 - training/rollout_probs_diff_std:0.013606295920908451 - training/rollout_actor_probs_pearson_corr:0.9980311393737793 - rollout_corr/rollout_is_mean:0.9999285936355591 - rollout_corr/rollout_is_ratio_fraction_high:3.8674621464451775e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.76805866532959e-05 - rollout_corr/rollout_is_max:3.7063148021698 - rollout_corr/rollout_is_min:0.2912478744983673 - rollout_corr/rollout_is_std:0.0409952737390995 - rollout_corr/rollout_is_eff_sample_size:0.9983218507269501 - rollout_corr/rollout_is_seq_mean:0.9999192357063293 - rollout_corr/rollout_is_seq_std:0.002459433162584901 - rollout_corr/rollout_is_seq_max:1.008528470993042 - rollout_corr/rollout_is_seq_min:0.9915392994880676 - rollout_corr/rollout_is_seq_max_deviation:0.008528470993041992 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.277975270524621) - rollout_corr/training_log_ppl:np.float64(0.24159730068640783) - rollout_corr/kl:np.float64(0.0014997820293984887) - rollout_corr/k3_kl:np.float64(0.0011694539433619866) - rollout_corr/rollout_ppl:np.float64(1.2760190423578024) - rollout_corr/rollout_log_ppl:np.float64(0.2400975169875892) - rollout_corr/log_ppl_diff:np.float64(0.0014997836988186464) - rollout_corr/log_ppl_abs_diff:np.float64(0.002539189110393636) - rollout_corr/log_ppl_diff_max:np.float64(0.011486321687698364) - rollout_corr/log_ppl_diff_min:np.float64(-0.00882810354232788) - rollout_corr/ppl_ratio:np.float64(1.001505514839664) - rollout_corr/chi2_token:np.float64(0.0016600224189460278) - rollout_corr/chi2_seq:np.float64(0.7354465429671109) - actor/pg_loss:np.float64(0.00017330946866422892) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015464231032638054) - actor/ppo_kl:np.float64(0.0004333707376389384) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6127719506621361) - perf/mfu/actor:np.float64(0.06741122513113522) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(106.24171829223633) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:1 - critic/score/mean:0.25390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.25390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00273381220176816 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00273381220176816 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:404.01171875 - response_length/max:2216.0 - response_length/min:139.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:404.01171875 - response_length_non_aborted/max:2216.0 - response_length_non_aborted/min:139.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:334.96875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.080232560634613e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.536420190706849) - timing_s/agent_loop/generate_sequences/max:np.float64(13.316242065280676) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.049403940458433) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.316242065280676) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:2216 - timing_s/gen:19.068678671494126 - timing_s/reward:0.05746416188776493 - timing_s/old_log_prob:2.5150575172156096 - timing_s/adv:0.1416328214108944 - timing_s/update_actor:10.08072930201888 - timing_s/step:31.964212140068412 - timing_s/stop_profile:0.00014040805399417877 - timing_per_token_ms/gen:0.1843684789416122 - timing_per_token_ms/update_actor:0.0532867247528472 - timing_per_token_ms/adv:0.0007486709487358237 - perf/total_num_tokens:189179 - perf/time_per_step:31.964212140068412 - perf/throughput:739.8078481138934 (TaskRunner pid=2099163) Training Progress: 23%|██▎ | 23/100 [22:09<1:02:12, 48.47s/it] (TaskRunner pid=2099163) step:24 - global_seqlen/min:14555 - global_seqlen/max:31176 - global_seqlen/minmax_diff:16621 - global_seqlen/balanced_min:20123 - global_seqlen/balanced_max:20634 - global_seqlen/mean:20209.125 - actor/entropy:0.21117329597473145 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29998478293418884 - training/rollout_probs_diff_mean:0.005020960234105587 - training/rollout_probs_diff_std:0.013722977600991726 - training/rollout_actor_probs_pearson_corr:0.9980407357215881 - rollout_corr/rollout_is_mean:0.9998765587806702 - rollout_corr/rollout_is_ratio_fraction_high:3.355366789037362e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.5922781029948965e-05 - rollout_corr/rollout_is_max:2.400970697402954 - rollout_corr/rollout_is_min:0.23983050882816315 - rollout_corr/rollout_is_std:0.04097927734255791 - rollout_corr/rollout_is_eff_sample_size:0.9983230388235971 - rollout_corr/rollout_is_seq_mean:0.9998420476913452 - rollout_corr/rollout_is_seq_std:0.002557374769821763 - rollout_corr/rollout_is_seq_max:1.0084576606750488 - rollout_corr/rollout_is_seq_min:0.9929150342941284 - rollout_corr/rollout_is_seq_max_deviation:0.008457660675048828 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2810057001188397) - rollout_corr/training_log_ppl:np.float64(0.2444376761704916) - rollout_corr/kl:np.float64(0.0012209779730945058) - rollout_corr/k3_kl:np.float64(0.0010693179642657924) - rollout_corr/rollout_ppl:np.float64(1.2793983216397464) - rollout_corr/rollout_log_ppl:np.float64(0.24321669599157758) - rollout_corr/log_ppl_diff:np.float64(0.0012209801789140329) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025764188467292115) - rollout_corr/log_ppl_diff_max:np.float64(0.011098459362983704) - rollout_corr/log_ppl_diff_min:np.float64(-0.008527815341949463) - rollout_corr/ppl_ratio:np.float64(1.0012267434503883) - rollout_corr/chi2_token:np.float64(0.0018536767456680536) - rollout_corr/chi2_seq:np.float64(1.0982552233617753) - actor/pg_loss:np.float64(0.0001084102550521493) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001204582532409404) - actor/ppo_kl:np.float64(9.153693397934148e-05) - actor/pg_clipfrac_lower:np.float64(1.0643732821336016e-05) - actor/grad_norm:np.float64(0.641492635011673) - perf/mfu/actor:np.float64(0.056898364927665215) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(105.3012466430664) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:1 - critic/score/mean:0.36328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.36328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.010942690074443817 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.010942690074443817 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:349.25390625 - response_length/max:2150.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:349.25390625 - response_length_non_aborted/max:2150.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:282.28125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.868107736110687e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0074804201722145) - timing_s/agent_loop/generate_sequences/max:np.float64(12.549119051545858) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2516225457002292) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.549119051545858) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:2150 - timing_s/gen:18.52240716293454 - timing_s/reward:0.070555554702878 - timing_s/old_log_prob:2.476743496954441 - timing_s/adv:0.16664943657815456 - timing_s/update_actor:10.124946380034089 - timing_s/step:31.44615630246699 - timing_s/stop_profile:2.8703361749649048e-05 - timing_per_token_ms/gen:0.2071649069213898 - timing_per_token_ms/update_actor:0.06262608091662857 - timing_per_token_ms/adv:0.0010307808760779756 - perf/total_num_tokens:161673 - perf/time_per_step:31.44615630246699 - perf/throughput:642.6580344388408 (TaskRunner pid=2099163) Training Progress: 24%|██▍ | 24/100 [22:41<54:56, 43.37s/it] (TaskRunner pid=2099163) step:25 - global_seqlen/min:19247 - global_seqlen/max:30356 - global_seqlen/minmax_diff:11109 - global_seqlen/balanced_min:26672 - global_seqlen/balanced_max:27390 - global_seqlen/mean:26767.875 - actor/entropy:0.18973146378993988 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.33718350529670715 - training/rollout_probs_diff_mean:0.004361078143119812 - training/rollout_probs_diff_std:0.012972155585885048 - training/rollout_actor_probs_pearson_corr:0.9981417059898376 - rollout_corr/rollout_is_mean:0.999921441078186 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.597230822080746e-05 - rollout_corr/rollout_is_max:1.7342352867126465 - rollout_corr/rollout_is_min:0.2947933077812195 - rollout_corr/rollout_is_std:0.038447305560112 - rollout_corr/rollout_is_eff_sample_size:0.9985237489301881 - rollout_corr/rollout_is_seq_mean:0.9999851584434509 - rollout_corr/rollout_is_seq_std:0.0023914307821542025 - rollout_corr/rollout_is_seq_max:1.012315273284912 - rollout_corr/rollout_is_seq_min:0.9918898344039917 - rollout_corr/rollout_is_seq_max_deviation:0.01231527328491211 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2648270050995052) - rollout_corr/training_log_ppl:np.float64(0.23086260134004988) - rollout_corr/kl:np.float64(0.0009177963597579719) - rollout_corr/k3_kl:np.float64(0.001010191431845442) - rollout_corr/rollout_ppl:np.float64(1.263638133648783) - rollout_corr/rollout_log_ppl:np.float64(0.22994480418856256) - rollout_corr/log_ppl_diff:np.float64(0.0009177971514873207) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021577174484264106) - rollout_corr/log_ppl_diff_max:np.float64(0.01115649938583374) - rollout_corr/log_ppl_diff_min:np.float64(-0.010651543736457825) - rollout_corr/ppl_ratio:np.float64(1.000921874307096) - rollout_corr/chi2_token:np.float64(0.0022637455258518457) - rollout_corr/chi2_seq:np.float64(0.9759733870159835) - actor/pg_loss:np.float64(0.0001767532667145133) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011058543057060888) - actor/ppo_kl:np.float64(-2.0648154892555226e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.553518146276474) - perf/mfu/actor:np.float64(0.07675197121879002) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(105.21955585479736) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:1 - critic/score/mean:0.3828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.021515218541026115 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.021515218541026115 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:473.68359375 - response_length/max:2715.0 - response_length/min:106.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:473.68359375 - response_length_non_aborted/max:2715.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:362.8125 - prompt_length/max:499.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.77803361415863e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.284107943996787) - timing_s/agent_loop/generate_sequences/max:np.float64(15.839517647400498) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.669048825882783) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.839517647400498) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:2715 - timing_s/gen:21.6606355458498 - timing_s/reward:0.05967309698462486 - timing_s/old_log_prob:2.458761554211378 - timing_s/adv:0.14967934787273407 - timing_s/update_actor:9.980403181165457 - timing_s/step:34.411740532144904 - timing_s/stop_profile:0.00013089925050735474 - timing_per_token_ms/gen:0.17862526529815195 - timing_per_token_ms/update_actor:0.04660625461100973 - timing_per_token_ms/adv:0.0006989691368512352 - perf/total_num_tokens:214143 - perf/time_per_step:34.411740532144904 - perf/throughput:777.8704182369221 (TaskRunner pid=2099163) Training Progress: 25%|██▌ | 25/100 [23:15<50:52, 40.70s/it] (TaskRunner pid=2099163) step:26 - global_seqlen/min:14395 - global_seqlen/max:27159 - global_seqlen/minmax_diff:12764 - global_seqlen/balanced_min:22165 - global_seqlen/balanced_max:22193 - global_seqlen/mean:22185.625 - actor/entropy:0.1997486650943756 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.33525192737579346 - training/rollout_probs_diff_mean:0.004470215644687414 - training/rollout_probs_diff_std:0.012864506803452969 - training/rollout_actor_probs_pearson_corr:0.9982172250747681 - rollout_corr/rollout_is_mean:1.0001190900802612 - rollout_corr/rollout_is_ratio_fraction_high:1.0165391358896159e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.0992344515398145e-05 - rollout_corr/rollout_is_max:3.4254493713378906 - rollout_corr/rollout_is_min:0.33838894963264465 - rollout_corr/rollout_is_std:0.03890654817223549 - rollout_corr/rollout_is_eff_sample_size:0.9984888060234459 - rollout_corr/rollout_is_seq_mean:1.0002080202102661 - rollout_corr/rollout_is_seq_std:0.0026749917306005955 - rollout_corr/rollout_is_seq_max:1.0075410604476929 - rollout_corr/rollout_is_seq_min:0.9928621649742126 - rollout_corr/rollout_is_seq_max_deviation:0.007541060447692871 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2674571070820093) - rollout_corr/training_log_ppl:np.float64(0.23334544732642826) - rollout_corr/kl:np.float64(0.0006923889370149539) - rollout_corr/k3_kl:np.float64(0.0010116354250726545) - rollout_corr/rollout_ppl:np.float64(1.266531832050532) - rollout_corr/rollout_log_ppl:np.float64(0.23265305937093217) - rollout_corr/log_ppl_diff:np.float64(0.0006923879554960877) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022414404957089573) - rollout_corr/log_ppl_diff_max:np.float64(0.009537696838378906) - rollout_corr/log_ppl_diff_min:np.float64(-0.008276775479316711) - rollout_corr/ppl_ratio:np.float64(1.0006968146190047) - rollout_corr/chi2_token:np.float64(0.0026694873813539743) - rollout_corr/chi2_seq:np.float64(0.8543046184349805) - actor/pg_loss:np.float64(-2.134358510375023e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006741704369233048) - actor/ppo_kl:np.float64(8.118165057968696e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.37092382460832596) - perf/mfu/actor:np.float64(0.06470432805583136) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(105.13681030273438) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:1 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005917528178542852 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005917528178542852 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:384.26953125 - response_length/max:1567.0 - response_length/min:83.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:384.26953125 - response_length_non_aborted/max:1567.0 - response_length_non_aborted/min:83.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:309.03125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010296143591403961 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9918509125709534) - timing_s/agent_loop/generate_sequences/max:np.float64(10.191927688196301) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6913370271213353) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.191927688196301) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1567 - timing_s/gen:15.927538784220815 - timing_s/reward:0.06093833968043327 - timing_s/old_log_prob:2.489799652248621 - timing_s/adv:0.15363946184515953 - timing_s/update_actor:9.85607378743589 - timing_s/step:28.577587887644768 - timing_s/stop_profile:0.00011636503040790558 - timing_per_token_ms/gen:0.16190965797750212 - timing_per_token_ms/update_actor:0.055531869101253004 - timing_per_token_ms/adv:0.0008656475862476239 - perf/total_num_tokens:177485 - perf/time_per_step:28.577587887644768 - perf/throughput:776.3295169356029 (TaskRunner pid=2099163) Training Progress: 26%|██▌ | 26/100 [23:44<45:43, 37.08s/it] (TaskRunner pid=2099163) step:27 - global_seqlen/min:12650 - global_seqlen/max:28413 - global_seqlen/minmax_diff:15763 - global_seqlen/balanced_min:22006 - global_seqlen/balanced_max:22028 - global_seqlen/mean:22020.875 - actor/entropy:0.1973375827074051 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553977966308594 - training/rollout_probs_diff_mean:0.00446832412853837 - training/rollout_probs_diff_std:0.012732885777950287 - training/rollout_actor_probs_pearson_corr:0.9982201457023621 - rollout_corr/rollout_is_mean:1.0000790357589722 - rollout_corr/rollout_is_ratio_fraction_high:2.0880532247247174e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.132079655188136e-05 - rollout_corr/rollout_is_max:3.3716399669647217 - rollout_corr/rollout_is_min:0.41315898299217224 - rollout_corr/rollout_is_std:0.03870687633752823 - rollout_corr/rollout_is_eff_sample_size:0.9985042566669848 - rollout_corr/rollout_is_seq_mean:1.0001776218414307 - rollout_corr/rollout_is_seq_std:0.002695156028494239 - rollout_corr/rollout_is_seq_max:1.01008939743042 - rollout_corr/rollout_is_seq_min:0.9929530024528503 - rollout_corr/rollout_is_seq_max_deviation:0.010089397430419922 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2657123478129506) - rollout_corr/training_log_ppl:np.float64(0.23218309982621577) - rollout_corr/kl:np.float64(0.0008745037185420301) - rollout_corr/k3_kl:np.float64(0.0010085490487199422) - rollout_corr/rollout_ppl:np.float64(1.264582925941795) - rollout_corr/rollout_log_ppl:np.float64(0.2313085944915656) - rollout_corr/log_ppl_diff:np.float64(0.0008745053346501663) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021427856263471767) - rollout_corr/log_ppl_diff_max:np.float64(0.011478304862976074) - rollout_corr/log_ppl_diff_min:np.float64(-0.008908957242965698) - rollout_corr/ppl_ratio:np.float64(1.0008788730483502) - rollout_corr/chi2_token:np.float64(0.0022939874324947596) - rollout_corr/chi2_seq:np.float64(0.9378205249086022) - actor/pg_loss:np.float64(4.4271000660955906e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008205796011679922) - actor/ppo_kl:np.float64(0.00014555726753329168) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.377008780837059) - perf/mfu/actor:np.float64(0.0638297454911889) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(105.12808609008789) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:1 - critic/score/mean:0.296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014912614598870277 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014912614598870277 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:374.15234375 - response_length/max:1612.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:374.15234375 - response_length_non_aborted/max:1612.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.0 - prompt_length/max:499.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.013922393321991e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1771093513816595) - timing_s/agent_loop/generate_sequences/max:np.float64(10.33962314389646) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6523388316636556) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.33962314389646) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:414 - timing_s/agent_loop/slowest/response_length:1592 - timing_s/gen:16.076986119151115 - timing_s/reward:0.05911053717136383 - timing_s/old_log_prob:2.4205230213701725 - timing_s/adv:0.15234399028122425 - timing_s/update_actor:9.950848061591387 - timing_s/step:28.75192417949438 - timing_s/stop_profile:0.0001438334584236145 - timing_per_token_ms/gen:0.16784801185127962 - timing_per_token_ms/update_actor:0.05648531258176268 - timing_per_token_ms/adv:0.0008647703047745846 - perf/total_num_tokens:176167 - perf/time_per_step:28.75192417949438 - perf/throughput:765.89221863993 (TaskRunner pid=2099163) Training Progress: 27%|██▋ | 27/100 [24:12<42:05, 34.60s/it] (TaskRunner pid=2099163) step:28 - global_seqlen/min:17510 - global_seqlen/max:32402 - global_seqlen/minmax_diff:14892 - global_seqlen/balanced_min:24287 - global_seqlen/balanced_max:24336 - global_seqlen/mean:24323.75 - actor/entropy:0.19081273674964905 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35365766286849976 - training/rollout_probs_diff_mean:0.004214483313262463 - training/rollout_probs_diff_std:0.012326523661613464 - training/rollout_actor_probs_pearson_corr:0.9983044266700745 - rollout_corr/rollout_is_mean:0.9999059438705444 - rollout_corr/rollout_is_ratio_fraction_high:1.8814322174875997e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.407160541741177e-05 - rollout_corr/rollout_is_max:2.637937545776367 - rollout_corr/rollout_is_min:0.28466564416885376 - rollout_corr/rollout_is_std:0.037621427327394485 - rollout_corr/rollout_is_eff_sample_size:0.9985862719338552 - rollout_corr/rollout_is_seq_mean:0.999921977519989 - rollout_corr/rollout_is_seq_std:0.0030063134618103504 - rollout_corr/rollout_is_seq_max:1.014600157737732 - rollout_corr/rollout_is_seq_min:0.990456759929657 - rollout_corr/rollout_is_seq_max_deviation:0.014600157737731934 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2607678300701082) - rollout_corr/training_log_ppl:np.float64(0.22826670162612572) - rollout_corr/kl:np.float64(0.0009433091491317658) - rollout_corr/k3_kl:np.float64(0.0009751564648468047) - rollout_corr/rollout_ppl:np.float64(1.2595306900329888) - rollout_corr/rollout_log_ppl:np.float64(0.22732339147478342) - rollout_corr/log_ppl_diff:np.float64(0.0009433101513423026) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022531004797201604) - rollout_corr/log_ppl_diff_max:np.float64(0.010257154703140259) - rollout_corr/log_ppl_diff_min:np.float64(-0.012321293354034424) - rollout_corr/ppl_ratio:np.float64(1.0009483909234405) - rollout_corr/chi2_token:np.float64(0.002010138938203454) - rollout_corr/chi2_seq:np.float64(1.263109202729538) - actor/pg_loss:np.float64(0.0003952222177758813) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000980165132887123) - actor/ppo_kl:np.float64(-2.858693859408845e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5482866317033768) - perf/mfu/actor:np.float64(0.07006718373692834) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(105.16790771484375) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:1 - critic/score/mean:0.34765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015940433368086815 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015940433368086815 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:415.2421875 - response_length/max:1856.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:415.2421875 - response_length_non_aborted/max:1856.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:344.875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.274482727050781e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9281809013336897) - timing_s/agent_loop/generate_sequences/max:np.float64(11.828542461618781) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.940465694955492) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.828542461618781) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:1856 - timing_s/gen:17.50202799960971 - timing_s/reward:0.06275253742933273 - timing_s/old_log_prob:2.5015085004270077 - timing_s/adv:0.1598478928208351 - timing_s/update_actor:9.977494372054935 - timing_s/step:30.29320042952895 - timing_s/stop_profile:0.00013585761189460754 - timing_per_token_ms/gen:0.1646443905063847 - timing_per_token_ms/update_actor:0.05127444561413708 - timing_per_token_ms/adv:0.0008214599559115839 - perf/total_num_tokens:194590 - perf/time_per_step:30.29320042952895 - perf/throughput:802.9442137216344 (TaskRunner pid=2099163) Training Progress: 28%|██▊ | 28/100 [24:43<39:59, 33.32s/it] (TaskRunner pid=2099163) step:29 - global_seqlen/min:11576 - global_seqlen/max:26504 - global_seqlen/minmax_diff:14928 - global_seqlen/balanced_min:20443 - global_seqlen/balanced_max:20468 - global_seqlen/mean:20457.125 - actor/entropy:0.19512885808944702 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2994707226753235 - training/rollout_probs_diff_mean:0.004517699591815472 - training/rollout_probs_diff_std:0.012806838378310204 - training/rollout_actor_probs_pearson_corr:0.9982004165649414 - rollout_corr/rollout_is_mean:0.9999142289161682 - rollout_corr/rollout_is_ratio_fraction_high:2.2683709175908007e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.5367418351816013e-05 - rollout_corr/rollout_is_max:2.5106632709503174 - rollout_corr/rollout_is_min:0.35614722967147827 - rollout_corr/rollout_is_std:0.03914633393287659 - rollout_corr/rollout_is_eff_sample_size:0.9984696716584548 - rollout_corr/rollout_is_seq_mean:0.9998313188552856 - rollout_corr/rollout_is_seq_std:0.0028376157861202955 - rollout_corr/rollout_is_seq_max:1.0107405185699463 - rollout_corr/rollout_is_seq_min:0.9908356666564941 - rollout_corr/rollout_is_seq_max_deviation:0.010740518569946289 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2674566823989153) - rollout_corr/training_log_ppl:np.float64(0.23359788639936596) - rollout_corr/kl:np.float64(0.0009326249134602449) - rollout_corr/k3_kl:np.float64(0.0010803121514300074) - rollout_corr/rollout_ppl:np.float64(1.2662694193422794) - rollout_corr/rollout_log_ppl:np.float64(0.2326652607152937) - rollout_corr/log_ppl_diff:np.float64(0.0009326256840722635) - rollout_corr/log_ppl_abs_diff:np.float64(0.002509462457965128) - rollout_corr/log_ppl_diff_max:np.float64(0.009627044200897217) - rollout_corr/log_ppl_diff_min:np.float64(-0.010430455207824707) - rollout_corr/ppl_ratio:np.float64(1.0009380243718624) - rollout_corr/chi2_token:np.float64(0.0024118823930621147) - rollout_corr/chi2_seq:np.float64(1.6191739679779857) - actor/pg_loss:np.float64(0.00010256993118673563) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001801439915652736) - actor/ppo_kl:np.float64(-0.0001434271987896807) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5583970844745636) - perf/mfu/actor:np.float64(0.05960628937352358) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(105.0120849609375) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:2 - critic/score/mean:0.296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008349022828042507 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008349022828042507 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:344.41015625 - response_length/max:1501.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:344.41015625 - response_length_non_aborted/max:1501.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:294.875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00034629926085472107 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7719959896057844) - timing_s/agent_loop/generate_sequences/max:np.float64(9.23289637081325) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.100054759372142) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.23289637081325) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:1501 - timing_s/gen:15.16298632323742 - timing_s/reward:0.05156240798532963 - timing_s/old_log_prob:2.6052888948470354 - timing_s/adv:0.14285925030708313 - timing_s/update_actor:9.606981605291367 - timing_s/step:27.679332403466105 - timing_s/stop_profile:0.00011761859059333801 - timing_per_token_ms/gen:0.17197638992432054 - timing_per_token_ms/update_actor:0.05870192906683715 - timing_per_token_ms/adv:0.0008729186671335973 - perf/total_num_tokens:163657 - perf/time_per_step:27.679332403466105 - perf/throughput:739.075809409272 (TaskRunner pid=2099163) Training Progress: 29%|██▉ | 29/100 [25:11<37:36, 31.78s/it] (TaskRunner pid=2099163) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2099163) validation generation end (TaskRunner pid=2099163) [prompt] system (TaskRunner pid=2099163) (TaskRunner pid=2099163) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) user (TaskRunner pid=2099163) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2099163) (TaskRunner pid=2099163) A: -0.01 (TaskRunner pid=2099163) B: 1.69 (TaskRunner pid=2099163) C: 2.49 (TaskRunner pid=2099163) D: 0.45 (TaskRunner pid=2099163) Please reason step by step. (TaskRunner pid=2099163) assistant (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) [response] (TaskRunner pid=2099163) The folding stability score of a protein sequence typically requires computational analysis using tools like Rosetta, which calculate the energy of a protein structure. The sequence provided ('GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR') would need to be input into such a tool to determine its folding stability score. Since no computational tool or database is available here to calculate this score, it is not possible to determine the exact folding stability score without additional information or computational resources. (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) D (TaskRunner pid=2099163) (TaskRunner pid=2099163) [ground_truth] B (TaskRunner pid=2099163) [score] 0.0 (TaskRunner pid=2099163) [acc] 0.0 (TaskRunner pid=2099163) [pred] D (TaskRunner pid=2099163) [incorrect_format] 1 (TaskRunner pid=2099163) [feedback] (TaskRunner pid=2099163) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (WorkerDict pid=2099539) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20/actor (TaskRunner pid=2099163) step:30 - global_seqlen/min:14663 - global_seqlen/max:34744 - global_seqlen/minmax_diff:20081 - global_seqlen/balanced_min:23097 - global_seqlen/balanced_max:23449 - global_seqlen/mean:23160.25 - actor/entropy:0.1929744929075241 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34270334243774414 - training/rollout_probs_diff_mean:0.004414478316903114 - training/rollout_probs_diff_std:0.012888044118881226 - training/rollout_actor_probs_pearson_corr:0.9981456995010376 - rollout_corr/rollout_is_mean:1.000051736831665 - rollout_corr/rollout_is_ratio_fraction_high:1.9112783775199205e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.556392615195364e-05 - rollout_corr/rollout_is_max:2.849820375442505 - rollout_corr/rollout_is_min:0.13807697594165802 - rollout_corr/rollout_is_std:0.03877303749322891 - rollout_corr/rollout_is_eff_sample_size:0.9984991460165831 - rollout_corr/rollout_is_seq_mean:0.9999996423721313 - rollout_corr/rollout_is_seq_std:0.002868861658498645 - rollout_corr/rollout_is_seq_max:1.0089269876480103 - rollout_corr/rollout_is_seq_min:0.989052414894104 - rollout_corr/rollout_is_seq_max_deviation:0.010947585105895996 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2640721159987152) - rollout_corr/training_log_ppl:np.float64(0.23040971330192406) - rollout_corr/kl:np.float64(0.001080727721191721) - rollout_corr/k3_kl:np.float64(0.0009677562554202268) - rollout_corr/rollout_ppl:np.float64(1.2626397949643433) - rollout_corr/rollout_log_ppl:np.float64(0.22932898352155462) - rollout_corr/log_ppl_diff:np.float64(0.0010807297803694382) - rollout_corr/log_ppl_abs_diff:np.float64(0.002270795885124244) - rollout_corr/log_ppl_diff_max:np.float64(0.009988278150558472) - rollout_corr/log_ppl_diff_min:np.float64(-0.008100628852844238) - rollout_corr/ppl_ratio:np.float64(1.0010852506384254) - rollout_corr/chi2_token:np.float64(0.001721000298857689) - rollout_corr/chi2_seq:np.float64(0.7308730445802212) - actor/pg_loss:np.float64(0.00017032702453434467) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00098742782915906) - actor/ppo_kl:np.float64(0.0001257946889916539) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.562660925090313) - perf/mfu/actor:np.float64(0.06606835743079029) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.89504623413086) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.3825) - val-aux/sciknoweval/reward/std@16:np.float64(0.2710825289783396) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.49848) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.23249398932762172) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.26538000000000006) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.23000654365019318) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.38264) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2717389471729169) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.5914200000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.16401609741962153) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.17232) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.16309037403563342) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.4011) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.2275492223932495) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.65118) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.1002078489826257) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.11272) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0901338588728539) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.4145999999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.17744782663880587) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.68952) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.05884117194239494) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.08363999999999999) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04403769261963444) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.42375999999999997) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.13151628701261814) - val-aux/sciknoweval/score/mean@16:np.float64(0.3825) - val-aux/sciknoweval/score/std@16:np.float64(0.2710825289783396) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.49848) - val-aux/sciknoweval/score/best@2/std:np.float64(0.23249398932762172) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.26538000000000006) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.23000654365019318) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.38264) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2717389471729169) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.5914200000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.16401609741962153) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.17232) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.16309037403563342) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.4011) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.2275492223932495) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.65118) - val-aux/sciknoweval/score/best@8/std:np.float64(0.1002078489826257) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.11272) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0901338588728539) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.4145999999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.17744782663880587) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.68952) - val-aux/sciknoweval/score/best@16/std:np.float64(0.05884117194239494) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.08363999999999999) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04403769261963444) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.42375999999999997) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.13151628701261814) - val-core/sciknoweval/acc/mean@16:np.float64(0.3825) - val-aux/sciknoweval/acc/std@16:np.float64(0.2710825289783396) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.49848) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.23249398932762172) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.26538000000000006) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.23000654365019318) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.38264) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2717389471729169) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.5914200000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.16401609741962153) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.17232) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.16309037403563342) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.4011) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.2275492223932495) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.65118) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.1002078489826257) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.11272) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0901338588728539) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.4145999999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.17744782663880587) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.68952) - val-core/sciknoweval/acc/best@16/std:np.float64(0.05884117194239494) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.08363999999999999) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04403769261963444) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.42375999999999997) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.13151628701261814) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.985) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.05195859001739712) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9987600000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.014549558441390551) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9708800000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.06937229914476689) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9849800000000001) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.05188589399017503) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.94702) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.08675116093152056) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99582) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.02779570240419907) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9096600000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.09741143940075572) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99928) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.010537634702780806) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8622200000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.08961444045723323) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999800000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0006321392251711644) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:2 - critic/score/mean:0.44921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012717168778181076 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012717168778181076 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:408.7578125 - response_length/max:2136.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:408.7578125 - response_length_non_aborted/max:2136.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.0 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.426474571228027e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.006382143124938) - timing_s/agent_loop/generate_sequences/max:np.float64(13.268630873411894) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9260140853366465) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.268630873411894) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:2136 - timing_s/gen:19.38935353793204 - timing_s/reward:0.06270048767328262 - timing_s/old_log_prob:2.4441376700997353 - timing_s/adv:0.1214206013828516 - timing_s/update_actor:9.896830067038536 - timing_s/step:32.002956476062536 - timing_s/testing:77.46762142144144 - timing_s/save_checkpoint:6.66411729156971 - timing_s/stop_profile:9.995326399803162e-05 - timing_per_token_ms/gen:0.18529226828550713 - timing_per_token_ms/update_actor:0.05341495702247674 - timing_per_token_ms/adv:0.0006553286416535422 - perf/total_num_tokens:185282 - perf/time_per_step:32.002956476062536 - perf/throughput:723.6909507821043 (TaskRunner pid=2099163) Training Progress: 30%|███ | 30/100 [27:07<1:06:36, 57.10s/it] (TaskRunner pid=2099163) step:31 - global_seqlen/min:15610 - global_seqlen/max:32987 - global_seqlen/minmax_diff:17377 - global_seqlen/balanced_min:22697 - global_seqlen/balanced_max:22706 - global_seqlen/mean:22704.125 - actor/entropy:0.20467621088027954 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.36793190240859985 - training/rollout_probs_diff_mean:0.0046606664545834064 - training/rollout_probs_diff_std:0.013037621974945068 - training/rollout_actor_probs_pearson_corr:0.998202919960022 - rollout_corr/rollout_is_mean:0.9999342560768127 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.056140409782529e-05 - rollout_corr/rollout_is_max:1.8634686470031738 - rollout_corr/rollout_is_min:0.08700346946716309 - rollout_corr/rollout_is_std:0.03880991414189339 - rollout_corr/rollout_is_eff_sample_size:0.998495937030278 - rollout_corr/rollout_is_seq_mean:0.9998412132263184 - rollout_corr/rollout_is_seq_std:0.0024558771401643753 - rollout_corr/rollout_is_seq_max:1.008089303970337 - rollout_corr/rollout_is_seq_min:0.9901171326637268 - rollout_corr/rollout_is_seq_max_deviation:0.009882867336273193 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2660416262224317) - rollout_corr/training_log_ppl:np.float64(0.2328464862657711) - rollout_corr/kl:np.float64(0.0014669340525781394) - rollout_corr/k3_kl:np.float64(0.0012662450398579495) - rollout_corr/rollout_ppl:np.float64(1.2641333830542862) - rollout_corr/rollout_log_ppl:np.float64(0.2313795514346566) - rollout_corr/log_ppl_diff:np.float64(0.0014669348311144859) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024422181595582515) - rollout_corr/log_ppl_diff_max:np.float64(0.0128212571144104) - rollout_corr/log_ppl_diff_min:np.float64(-0.01384890079498291) - rollout_corr/ppl_ratio:np.float64(1.0014724703505635) - rollout_corr/chi2_token:np.float64(0.002137396950274706) - rollout_corr/chi2_seq:np.float64(0.24568918230943382) - actor/pg_loss:np.float64(-4.5661814510822296e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002057411175883317) - actor/ppo_kl:np.float64(0.00041522945140526346) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5537090376019478) - perf/mfu/actor:np.float64(0.0655864265337439) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.2952651977539) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:2 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009520757012069225 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009520757012069225 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:387.00390625 - response_length/max:1622.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:387.00390625 - response_length_non_aborted/max:1622.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:322.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000301973894238472 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0549929738044739) - timing_s/agent_loop/generate_sequences/max:np.float64(10.455312624573708) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.841500379559875) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.455312624573708) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:1622 - timing_s/gen:16.36735489219427 - timing_s/reward:0.055881038308143616 - timing_s/old_log_prob:2.546142552047968 - timing_s/adv:0.16691702045500278 - timing_s/update_actor:9.986165532842278 - timing_s/step:29.223748456686735 - timing_s/stop_profile:0.00011165998876094818 - timing_per_token_ms/gen:0.16520499926513046 - timing_per_token_ms/update_actor:0.05497990746638705 - timing_per_token_ms/adv:0.0009189795932182081 - perf/total_num_tokens:181633 - perf/time_per_step:29.223748456686735 - perf/throughput:776.9066666328026 (TaskRunner pid=2099163) Training Progress: 31%|███ | 31/100 [27:36<56:03, 48.75s/it] (TaskRunner pid=2099163) step:32 - global_seqlen/min:16456 - global_seqlen/max:30003 - global_seqlen/minmax_diff:13547 - global_seqlen/balanced_min:22662 - global_seqlen/balanced_max:22713 - global_seqlen/mean:22703.375 - actor/entropy:0.19235235452651978 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6836323142051697 - training/rollout_probs_diff_mean:0.004288580734282732 - training/rollout_probs_diff_std:0.01270546205341816 - training/rollout_actor_probs_pearson_corr:0.9982287287712097 - rollout_corr/rollout_is_mean:1.000108003616333 - rollout_corr/rollout_is_ratio_fraction_high:4.2221261537633836e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.0555315384408459e-05 - rollout_corr/rollout_is_max:3.5419375896453857 - rollout_corr/rollout_is_min:0.4771340489387512 - rollout_corr/rollout_is_std:0.03740055859088898 - rollout_corr/rollout_is_eff_sample_size:0.9986033898627913 - rollout_corr/rollout_is_seq_mean:1.0001899003982544 - rollout_corr/rollout_is_seq_std:0.002721930155530572 - rollout_corr/rollout_is_seq_max:1.009945273399353 - rollout_corr/rollout_is_seq_min:0.9931846857070923 - rollout_corr/rollout_is_seq_max_deviation:0.009945273399353027 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2627140362747014) - rollout_corr/training_log_ppl:np.float64(0.22928791525191627) - rollout_corr/kl:np.float64(0.0009696064506714563) - rollout_corr/k3_kl:np.float64(0.0009657189097538321) - rollout_corr/rollout_ppl:np.float64(1.2614360507577658) - rollout_corr/rollout_log_ppl:np.float64(0.228318307781592) - rollout_corr/log_ppl_diff:np.float64(0.000969607470324263) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024012566718738526) - rollout_corr/log_ppl_diff_max:np.float64(0.008903712034225464) - rollout_corr/log_ppl_diff_min:np.float64(-0.007482409477233887) - rollout_corr/ppl_ratio:np.float64(1.0009746660944074) - rollout_corr/chi2_token:np.float64(0.0019679118413478136) - rollout_corr/chi2_seq:np.float64(0.5128269151318818) - actor/pg_loss:np.float64(0.0006739883683621883) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011445526070019696) - actor/ppo_kl:np.float64(0.00029994351070516956) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5374793410301208) - perf/mfu/actor:np.float64(0.06520734277390705) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.37225103378296) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:2 - critic/score/mean:0.5078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0055389017798006535 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0055389017798006535 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:370.07421875 - response_length/max:1794.0 - response_length/min:84.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:370.07421875 - response_length_non_aborted/max:1794.0 - response_length_non_aborted/min:84.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:339.40625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.293854236602783e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0003391448408365) - timing_s/agent_loop/generate_sequences/max:np.float64(11.236332735046744) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5134957447080524) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.236332735046744) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1794 - timing_s/gen:17.538983415812254 - timing_s/reward:0.05691481754183769 - timing_s/old_log_prob:2.4536229204386473 - timing_s/adv:0.14731214381754398 - timing_s/update_actor:10.00981381163001 - timing_s/step:30.308992091566324 - timing_s/stop_profile:0.0001265406608581543 - timing_per_token_ms/gen:0.18512949699503112 - timing_per_token_ms/update_actor:0.05511192615431632 - timing_per_token_ms/adv:0.0008110696307131868 - perf/total_num_tokens:181627 - perf/time_per_step:30.308992091566324 - perf/throughput:749.0640048804976 (TaskRunner pid=2099163) Training Progress: 32%|███▏ | 32/100 [28:07<49:00, 43.24s/it] (TaskRunner pid=2099163) step:33 - global_seqlen/min:14153 - global_seqlen/max:31822 - global_seqlen/minmax_diff:17669 - global_seqlen/balanced_min:24569 - global_seqlen/balanced_max:24592 - global_seqlen/mean:24586.5 - actor/entropy:0.19219401478767395 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.38829466700553894 - training/rollout_probs_diff_mean:0.004298380576074123 - training/rollout_probs_diff_std:0.012502633035182953 - training/rollout_actor_probs_pearson_corr:0.9982357621192932 - rollout_corr/rollout_is_mean:0.9999257922172546 - rollout_corr/rollout_is_ratio_fraction_high:1.8696480765356682e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.674120282288641e-05 - rollout_corr/rollout_is_max:3.7463631629943848 - rollout_corr/rollout_is_min:0.3203428089618683 - rollout_corr/rollout_is_std:0.03699354827404022 - rollout_corr/rollout_is_eff_sample_size:0.9986331098833465 - rollout_corr/rollout_is_seq_mean:0.9999837875366211 - rollout_corr/rollout_is_seq_std:0.002320910105481744 - rollout_corr/rollout_is_seq_max:1.0067297220230103 - rollout_corr/rollout_is_seq_min:0.9923191666603088 - rollout_corr/rollout_is_seq_max_deviation:0.007680833339691162 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2546508102677763) - rollout_corr/training_log_ppl:np.float64(0.22293072564934846) - rollout_corr/kl:np.float64(0.0007883897724163154) - rollout_corr/k3_kl:np.float64(0.0008786465269849941) - rollout_corr/rollout_ppl:np.float64(1.2536278427578509) - rollout_corr/rollout_log_ppl:np.float64(0.22214233508566394) - rollout_corr/log_ppl_diff:np.float64(0.0007883905636845157) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019512200815370306) - rollout_corr/log_ppl_diff_max:np.float64(0.00915798544883728) - rollout_corr/log_ppl_diff_min:np.float64(-0.007831752300262451) - rollout_corr/ppl_ratio:np.float64(1.0007919785566628) - rollout_corr/chi2_token:np.float64(0.0019339756108820438) - rollout_corr/chi2_seq:np.float64(0.8230726562906057) - actor/pg_loss:np.float64(6.270431913435459e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013706558816011238) - actor/ppo_kl:np.float64(-7.741855117160412e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6440355181694031) - perf/mfu/actor:np.float64(0.0710191135212451) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.29852676391602) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:2 - critic/score/mean:0.453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00058893917594105 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00058893917594105 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:417.859375 - response_length/max:1500.0 - response_length/min:109.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:417.859375 - response_length_non_aborted/max:1500.0 - response_length_non_aborted/min:109.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:350.46875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.15168023109436e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3214661497622728) - timing_s/agent_loop/generate_sequences/max:np.float64(10.396715246140957) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.068895058728231) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.396715246140957) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1500 - timing_s/gen:16.118461679667234 - timing_s/reward:0.06390342302620411 - timing_s/old_log_prob:2.3601754792034626 - timing_s/adv:0.18045086413621902 - timing_s/update_actor:9.997449096292257 - timing_s/step:28.806253915652633 - timing_s/stop_profile:0.00010740384459495544 - timing_per_token_ms/gen:0.1506792588683696 - timing_per_token_ms/update_actor:0.05082793960248641 - timing_per_token_ms/adv:0.00091742858955229 - perf/total_num_tokens:196692 - perf/time_per_step:28.806253915652633 - perf/throughput:853.512576539509 (TaskRunner pid=2099163) Training Progress: 33%|███▎ | 33/100 [28:36<43:27, 38.93s/it] (TaskRunner pid=2099163) step:34 - global_seqlen/min:13165 - global_seqlen/max:31218 - global_seqlen/minmax_diff:18053 - global_seqlen/balanced_min:20844 - global_seqlen/balanced_max:21371 - global_seqlen/mean:20953.375 - actor/entropy:0.20621417462825775 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.47441184520721436 - training/rollout_probs_diff_mean:0.004632072988897562 - training/rollout_probs_diff_std:0.013067140243947506 - training/rollout_actor_probs_pearson_corr:0.9982191324234009 - rollout_corr/rollout_is_mean:0.9996575117111206 - rollout_corr/rollout_is_ratio_fraction_high:1.123557649407303e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.741345714544877e-05 - rollout_corr/rollout_is_max:3.048807144165039 - rollout_corr/rollout_is_min:0.1631394475698471 - rollout_corr/rollout_is_std:0.039033498615026474 - rollout_corr/rollout_is_eff_sample_size:0.9984777531641464 - rollout_corr/rollout_is_seq_mean:0.9995238184928894 - rollout_corr/rollout_is_seq_std:0.003114572959020734 - rollout_corr/rollout_is_seq_max:1.0097143650054932 - rollout_corr/rollout_is_seq_min:0.9892820119857788 - rollout_corr/rollout_is_seq_max_deviation:0.010717988014221191 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2769425138831139) - rollout_corr/training_log_ppl:np.float64(0.24088622524868697) - rollout_corr/kl:np.float64(0.0017434109029323963) - rollout_corr/k3_kl:np.float64(0.0011406559008833028) - rollout_corr/rollout_ppl:np.float64(1.2746948134154081) - rollout_corr/rollout_log_ppl:np.float64(0.23914281158067752) - rollout_corr/log_ppl_diff:np.float64(0.0017434136680094525) - rollout_corr/log_ppl_abs_diff:np.float64(0.002788226425764151) - rollout_corr/log_ppl_diff_max:np.float64(0.012063741683959961) - rollout_corr/log_ppl_diff_min:np.float64(-0.007839560508728027) - rollout_corr/ppl_ratio:np.float64(1.001750238938257) - rollout_corr/chi2_token:np.float64(0.0010821421165019274) - rollout_corr/chi2_seq:np.float64(0.3799931020475924) - actor/pg_loss:np.float64(0.00013993948232382536) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016885243967408314) - actor/ppo_kl:np.float64(0.0003524259806644281) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5224123150110245) - perf/mfu/actor:np.float64(0.06054932786011055) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.48657608032227) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:2 - critic/score/mean:0.3828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01508656982332468 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01508656982332468 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:347.66796875 - response_length/max:2290.0 - response_length/min:83.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:347.66796875 - response_length_non_aborted/max:2290.0 - response_length_non_aborted/min:83.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:307.125 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.014829993247986e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0219873953610659) - timing_s/agent_loop/generate_sequences/max:np.float64(13.463359158486128) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3108865806207177) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.463359158486128) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:414 - timing_s/agent_loop/slowest/response_length:2290 - timing_s/gen:19.49041328765452 - timing_s/reward:0.05500324070453644 - timing_s/old_log_prob:2.408570285886526 - timing_s/adv:0.15743279829621315 - timing_s/update_actor:9.805676659569144 - timing_s/step:32.01260624267161 - timing_s/stop_profile:0.00011429190635681152 - timing_per_token_ms/gen:0.21898602617501117 - timing_per_token_ms/update_actor:0.05849700024202034 - timing_per_token_ms/adv:0.0009391852046282111 - perf/total_num_tokens:167627 - perf/time_per_step:32.01260624267161 - perf/throughput:654.5351178583496 (TaskRunner pid=2099163) Training Progress: 34%|███▍ | 34/100 [29:08<40:33, 36.86s/it] (TaskRunner pid=2099163) step:35 - global_seqlen/min:10930 - global_seqlen/max:29063 - global_seqlen/minmax_diff:18133 - global_seqlen/balanced_min:21182 - global_seqlen/balanced_max:21279 - global_seqlen/mean:21202.375 - actor/entropy:0.20077916979789734 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4047938883304596 - training/rollout_probs_diff_mean:0.004468213766813278 - training/rollout_probs_diff_std:0.012723792344331741 - training/rollout_actor_probs_pearson_corr:0.9982172250747681 - rollout_corr/rollout_is_mean:0.9999480247497559 - rollout_corr/rollout_is_ratio_fraction_high:1.1354991329426412e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.4064974897773936e-05 - rollout_corr/rollout_is_max:2.361192226409912 - rollout_corr/rollout_is_min:0.35472583770751953 - rollout_corr/rollout_is_std:0.03832307830452919 - rollout_corr/rollout_is_eff_sample_size:0.9985334953471807 - rollout_corr/rollout_is_seq_mean:0.9999346137046814 - rollout_corr/rollout_is_seq_std:0.0025622511748224497 - rollout_corr/rollout_is_seq_max:1.0083149671554565 - rollout_corr/rollout_is_seq_min:0.9895864129066467 - rollout_corr/rollout_is_seq_max_deviation:0.010413587093353271 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2627762155607343) - rollout_corr/training_log_ppl:np.float64(0.22958877607015893) - rollout_corr/kl:np.float64(0.001482734944080022) - rollout_corr/k3_kl:np.float64(0.001303993512976831) - rollout_corr/rollout_ppl:np.float64(1.2608598791994154) - rollout_corr/rollout_log_ppl:np.float64(0.22810603884863667) - rollout_corr/log_ppl_diff:np.float64(0.0014827372215222567) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028337313269730657) - rollout_corr/log_ppl_diff_max:np.float64(0.01417878270149231) - rollout_corr/log_ppl_diff_min:np.float64(-0.015369802713394165) - rollout_corr/ppl_ratio:np.float64(1.0014910355675966) - rollout_corr/chi2_token:np.float64(0.002457682741805911) - rollout_corr/chi2_seq:np.float64(1.00893239909783) - actor/pg_loss:np.float64(0.0003223039675503969) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0018587432675758464) - actor/ppo_kl:np.float64(0.000577584969583711) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.604684367775917) - perf/mfu/actor:np.float64(0.06179983949184756) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.36828994750977) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:2 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004048054106533527 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004048054106533527 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:344.01171875 - response_length/max:1733.0 - response_length/min:83.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:344.01171875 - response_length_non_aborted/max:1733.0 - response_length_non_aborted/min:83.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:318.5625 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.777901530265808e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7171750254929066) - timing_s/agent_loop/generate_sequences/max:np.float64(8.89642646163702) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0253594234527554) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.89642646163702) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:1293 - timing_s/gen:15.099879954010248 - timing_s/reward:0.04901489429175854 - timing_s/old_log_prob:2.3395367600023746 - timing_s/adv:0.11664748005568981 - timing_s/update_actor:9.723327906802297 - timing_s/step:27.36307036317885 - timing_s/stop_profile:0.00013891421258449554 - timing_per_token_ms/gen:0.17145900228246957 - timing_per_token_ms/update_actor:0.05732452087798122 - timing_per_token_ms/adv:0.00068770291096923 - perf/total_num_tokens:169619 - perf/time_per_step:27.36307036317885 - perf/throughput:774.853651969225 (TaskRunner pid=2099163) Training Progress: 35%|███▌ | 35/100 [29:35<36:51, 34.03s/it] (TaskRunner pid=2099163) step:36 - global_seqlen/min:14194 - global_seqlen/max:29137 - global_seqlen/minmax_diff:14943 - global_seqlen/balanced_min:20936 - global_seqlen/balanced_max:21371 - global_seqlen/mean:21018.125 - actor/entropy:0.20421844720840454 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5412930250167847 - training/rollout_probs_diff_mean:0.00441008573397994 - training/rollout_probs_diff_std:0.012608489021658897 - training/rollout_actor_probs_pearson_corr:0.9983038306236267 - rollout_corr/rollout_is_mean:0.9999710917472839 - rollout_corr/rollout_is_ratio_fraction_high:2.242931986984331e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.607329876511358e-05 - rollout_corr/rollout_is_max:2.2185213565826416 - rollout_corr/rollout_is_min:0.007402495015412569 - rollout_corr/rollout_is_std:0.03787250071763992 - rollout_corr/rollout_is_eff_sample_size:0.9985674904370404 - rollout_corr/rollout_is_seq_mean:0.999952495098114 - rollout_corr/rollout_is_seq_std:0.0027767117135226727 - rollout_corr/rollout_is_seq_max:1.008007526397705 - rollout_corr/rollout_is_seq_min:0.9892098307609558 - rollout_corr/rollout_is_seq_max_deviation:0.01079016923904419 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.27850745851174) - rollout_corr/training_log_ppl:np.float64(0.24139065398776438) - rollout_corr/kl:np.float64(0.0012924192194674333) - rollout_corr/k3_kl:np.float64(0.0011641595515641256) - rollout_corr/rollout_ppl:np.float64(1.2768099810928106) - rollout_corr/rollout_log_ppl:np.float64(0.24009823182132095) - rollout_corr/log_ppl_diff:np.float64(0.0012924221664434299) - rollout_corr/log_ppl_abs_diff:np.float64(0.002743791919783689) - rollout_corr/log_ppl_diff_max:np.float64(0.017144635319709778) - rollout_corr/log_ppl_diff_min:np.float64(-0.00719086080789566) - rollout_corr/ppl_ratio:np.float64(1.0013000178150833) - rollout_corr/chi2_token:np.float64(0.002002155175432563) - rollout_corr/chi2_seq:np.float64(0.773878968320787) - actor/pg_loss:np.float64(-6.605871021747589e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014383478630861646) - actor/ppo_kl:np.float64(0.00034995766268330897) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5709159523248672) - perf/mfu/actor:np.float64(0.061025477795435934) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.43119430541992) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:2 - critic/score/mean:0.34375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012700601946562529 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0012700601946562529 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:348.31640625 - response_length/max:2327.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:348.31640625 - response_length_non_aborted/max:2327.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:308.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013381987810134888 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9047284964472055) - timing_s/agent_loop/generate_sequences/max:np.float64(13.386295091360807) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3439106781152077) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.386295091360807) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:435 - timing_s/agent_loop/slowest/response_length:2327 - timing_s/gen:19.22059991583228 - timing_s/reward:0.06047280505299568 - timing_s/old_log_prob:2.4186214227229357 - timing_s/adv:0.13746901601552963 - timing_s/update_actor:9.880311040207744 - timing_s/step:31.805876852944493 - timing_s/stop_profile:0.00011717155575752258 - timing_per_token_ms/gen:0.215552489271297 - timing_per_token_ms/update_actor:0.05876065919419396 - timing_per_token_ms/adv:0.0008175623183295943 - perf/total_num_tokens:168145 - perf/time_per_step:31.805876852944493 - perf/throughput:660.8252021215445 (TaskRunner pid=2099163) Training Progress: 36%|███▌ | 36/100 [30:07<35:36, 33.38s/it] (TaskRunner pid=2099163) step:37 - global_seqlen/min:10542 - global_seqlen/max:30603 - global_seqlen/minmax_diff:20061 - global_seqlen/balanced_min:19813 - global_seqlen/balanced_max:26160 - global_seqlen/mean:20656.375 - actor/entropy:0.19215349853038788 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102194607257843 - training/rollout_probs_diff_mean:0.004060864448547363 - training/rollout_probs_diff_std:0.012080005370080471 - training/rollout_actor_probs_pearson_corr:0.9984062314033508 - rollout_corr/rollout_is_mean:0.9999275803565979 - rollout_corr/rollout_is_ratio_fraction_high:1.1167319826199673e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.583660094998777e-05 - rollout_corr/rollout_is_max:2.5254178047180176 - rollout_corr/rollout_is_min:0.23548689484596252 - rollout_corr/rollout_is_std:0.03610971197485924 - rollout_corr/rollout_is_eff_sample_size:0.9986974300643999 - rollout_corr/rollout_is_seq_mean:0.9998934268951416 - rollout_corr/rollout_is_seq_std:0.002943673636764288 - rollout_corr/rollout_is_seq_max:1.009252905845642 - rollout_corr/rollout_is_seq_min:0.9870862364768982 - rollout_corr/rollout_is_seq_max_deviation:0.012913763523101807 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2790742232464254) - rollout_corr/training_log_ppl:np.float64(0.2423434283118695) - rollout_corr/kl:np.float64(0.0010189136970133461) - rollout_corr/k3_kl:np.float64(0.0010748106194000684) - rollout_corr/rollout_ppl:np.float64(1.277731901500374) - rollout_corr/rollout_log_ppl:np.float64(0.24132451312470948) - rollout_corr/log_ppl_diff:np.float64(0.0010189151871600188) - rollout_corr/log_ppl_abs_diff:np.float64(0.002592919896414969) - rollout_corr/log_ppl_diff_max:np.float64(0.015276342630386353) - rollout_corr/log_ppl_diff_min:np.float64(-0.011511757969856262) - rollout_corr/ppl_ratio:np.float64(1.0010252590291202) - rollout_corr/chi2_token:np.float64(0.0022940633352845907) - rollout_corr/chi2_seq:np.float64(0.5807141268160194) - actor/pg_loss:np.float64(4.6521308831870556e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013833769742177537) - actor/ppo_kl:np.float64(4.1046847659931274e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5195856019854546) - perf/mfu/actor:np.float64(0.05586420816152182) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.32366943359375) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:2 - critic/score/mean:0.359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03285565227270126 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.03285565227270126 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:349.79296875 - response_length/max:8192.0 - response_length/min:81.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:349.79296875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:295.71875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016735494136810303 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6763673182576895) - timing_s/agent_loop/generate_sequences/max:np.float64(49.16137577034533) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9194193026196444) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.16137577034533) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.212846806272864 - timing_s/reward:0.05881442688405514 - timing_s/old_log_prob:2.536781370639801 - timing_s/adv:0.1617901399731636 - timing_s/update_actor:10.77561523206532 - timing_s/step:68.83385827951133 - timing_s/stop_profile:0.00013233162462711334 - timing_per_token_ms/gen:0.6165795259056458 - timing_per_token_ms/update_actor:0.06520756444478593 - timing_per_token_ms/adv:0.0009790569495686174 - perf/total_num_tokens:165251 - perf/time_per_step:68.83385827951133 - perf/throughput:300.0903264222289 (TaskRunner pid=2099163) Training Progress: 37%|███▋ | 37/100 [31:16<46:14, 44.04s/it] (TaskRunner pid=2099163) step:38 - global_seqlen/min:19998 - global_seqlen/max:31407 - global_seqlen/minmax_diff:11409 - global_seqlen/balanced_min:24964 - global_seqlen/balanced_max:28950 - global_seqlen/mean:25513.75 - actor/entropy:0.17665337026119232 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29611214995384216 - training/rollout_probs_diff_mean:0.0035523248370736837 - training/rollout_probs_diff_std:0.0110044926404953 - training/rollout_actor_probs_pearson_corr:0.9986032247543335 - rollout_corr/rollout_is_mean:0.999925434589386 - rollout_corr/rollout_is_ratio_fraction_high:1.739039726089686e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.478079452179372e-05 - rollout_corr/rollout_is_max:2.075227975845337 - rollout_corr/rollout_is_min:0.4160393178462982 - rollout_corr/rollout_is_std:0.03343033045530319 - rollout_corr/rollout_is_eff_sample_size:0.9988834226305476 - rollout_corr/rollout_is_seq_mean:0.9999387264251709 - rollout_corr/rollout_is_seq_std:0.0025514268781989813 - rollout_corr/rollout_is_seq_max:1.0072029829025269 - rollout_corr/rollout_is_seq_min:0.9921553730964661 - rollout_corr/rollout_is_seq_max_deviation:0.007844626903533936 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2722632107324898) - rollout_corr/training_log_ppl:np.float64(0.2361144043825334) - rollout_corr/kl:np.float64(0.0008850332726968446) - rollout_corr/k3_kl:np.float64(0.0009130484113342163) - rollout_corr/rollout_ppl:np.float64(1.2710885656997561) - rollout_corr/rollout_log_ppl:np.float64(0.23522936822337215) - rollout_corr/log_ppl_diff:np.float64(0.0008850361591612455) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021792255029140506) - rollout_corr/log_ppl_diff_max:np.float64(0.012408077716827393) - rollout_corr/log_ppl_diff_min:np.float64(-0.010918736457824707) - rollout_corr/ppl_ratio:np.float64(1.000889461953193) - rollout_corr/chi2_token:np.float64(0.001883210614323616) - rollout_corr/chi2_seq:np.float64(1.0299227663781494) - actor/pg_loss:np.float64(0.00017131585627794266) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015263908690030803) - actor/ppo_kl:np.float64(6.009416163976766e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6240686252713203) - perf/mfu/actor:np.float64(0.06947042905322103) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.28501892089844) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:2 - critic/score/mean:0.43359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.43359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014406857080757618 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014406857080757618 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:449.2421875 - response_length/max:6502.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:449.2421875 - response_length_non_aborted/max:6502.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:348.0625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.151758134365082e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.093883166089654) - timing_s/agent_loop/generate_sequences/max:np.float64(34.20732720755041) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.161816134750552) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(34.20732720755041) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:6502 - timing_s/gen:39.98209969326854 - timing_s/reward:0.0648152194917202 - timing_s/old_log_prob:2.563793510198593 - timing_s/adv:0.12993765622377396 - timing_s/update_actor:10.490677770227194 - timing_s/step:53.31818316318095 - timing_s/stop_profile:0.00011186115443706512 - timing_per_token_ms/gen:0.3476522937348359 - timing_per_token_ms/update_actor:0.0513971768665288 - timing_per_token_ms/adv:0.0006366060272587034 - perf/total_num_tokens:204110 - perf/time_per_step:53.31818316318095 - perf/throughput:478.5187432571522 (TaskRunner pid=2099163) Training Progress: 38%|███▊ | 38/100 [32:09<48:23, 46.84s/it] (TaskRunner pid=2099163) step:39 - global_seqlen/min:17054 - global_seqlen/max:28720 - global_seqlen/minmax_diff:11666 - global_seqlen/balanced_min:22500 - global_seqlen/balanced_max:22776 - global_seqlen/mean:22545.375 - actor/entropy:0.19491960108280182 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45386987924575806 - training/rollout_probs_diff_mean:0.004056902602314949 - training/rollout_probs_diff_std:0.012101990170776844 - training/rollout_actor_probs_pearson_corr:0.9984006881713867 - rollout_corr/rollout_is_mean:0.9999191164970398 - rollout_corr/rollout_is_ratio_fraction_high:1.0010110599978361e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012012132356176153 - rollout_corr/rollout_is_max:3.0525314807891846 - rollout_corr/rollout_is_min:0.011025836691260338 - rollout_corr/rollout_is_std:0.03583298996090889 - rollout_corr/rollout_is_eff_sample_size:0.9987174054849994 - rollout_corr/rollout_is_seq_mean:0.9999333024024963 - rollout_corr/rollout_is_seq_std:0.002865663729608059 - rollout_corr/rollout_is_seq_max:1.009408712387085 - rollout_corr/rollout_is_seq_min:0.9888485074043274 - rollout_corr/rollout_is_seq_max_deviation:0.011151492595672607 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2850506003014743) - rollout_corr/training_log_ppl:np.float64(0.24625673811533488) - rollout_corr/kl:np.float64(0.001440951160018411) - rollout_corr/k3_kl:np.float64(0.0010581204401205468) - rollout_corr/rollout_ppl:np.float64(1.2831103513017297) - rollout_corr/rollout_log_ppl:np.float64(0.24481578795530368) - rollout_corr/log_ppl_diff:np.float64(0.001440950160031207) - rollout_corr/log_ppl_abs_diff:np.float64(0.002506036587874405) - rollout_corr/log_ppl_diff_max:np.float64(0.015404284000396729) - rollout_corr/log_ppl_diff_min:np.float64(-0.008438989520072937) - rollout_corr/ppl_ratio:np.float64(1.0014470850583166) - rollout_corr/chi2_token:np.float64(0.0012923125177621841) - rollout_corr/chi2_seq:np.float64(0.49474142445251346) - actor/pg_loss:np.float64(8.401612285524607e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014041442402685789) - actor/ppo_kl:np.float64(0.00043948041806451954) - actor/pg_clipfrac_lower:np.float64(1.9405732018640265e-05) - actor/grad_norm:np.float64(0.6177720576524734) - perf/mfu/actor:np.float64(0.06559690994691629) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.28384017944336) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:2 - critic/score/mean:0.3828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004189230967313051 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004189230967313051 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:390.23046875 - response_length/max:2201.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:390.23046875 - response_length_non_aborted/max:2201.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.3125 - prompt_length/max:498.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016164779663085938 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0495860055088997) - timing_s/agent_loop/generate_sequences/max:np.float64(13.142637204378843) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6693802871523076) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.142637204378843) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:2201 - timing_s/gen:18.87878078967333 - timing_s/reward:0.06111055053770542 - timing_s/old_log_prob:2.4288064669817686 - timing_s/adv:0.12556837499141693 - timing_s/update_actor:9.746203064918518 - timing_s/step:31.32407439313829 - timing_s/stop_profile:3.988295793533325e-05 - timing_per_token_ms/gen:0.18897867635985674 - timing_per_token_ms/update_actor:0.054036598775350365 - timing_per_token_ms/adv:0.0006961980838166194 - perf/total_num_tokens:180363 - perf/time_per_step:31.32407439313829 - perf/throughput:719.7459282288861 (TaskRunner pid=2099163) Training Progress: 39%|███▉ | 39/100 [32:41<42:53, 42.19s/it] (TaskRunner pid=2099163) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2099163) validation generation end (TaskRunner pid=2099163) [prompt] system (TaskRunner pid=2099163) (TaskRunner pid=2099163) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) user (TaskRunner pid=2099163) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2099163) (TaskRunner pid=2099163) A: -0.01 (TaskRunner pid=2099163) B: 1.69 (TaskRunner pid=2099163) C: 2.49 (TaskRunner pid=2099163) D: 0.45 (TaskRunner pid=2099163) Please reason step by step. (TaskRunner pid=2099163) assistant (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) [response] (TaskRunner pid=2099163) The folding stability score of a protein sequence cannot be determined without access to specific computational tools or databases that calculate such scores based on the sequence. The provided sequence is a string of amino acids, and without additional context or computational analysis, it is not possible to determine the folding stability score. Therefore, the question as posed cannot be answered with the given information. (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) [ground_truth] B (TaskRunner pid=2099163) [score] 0.0 (TaskRunner pid=2099163) [acc] 0.0 (TaskRunner pid=2099163) [pred] A (TaskRunner pid=2099163) [incorrect_format] 1 (TaskRunner pid=2099163) [feedback] (TaskRunner pid=2099163) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (WorkerDict pid=2099539) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30/actor (TaskRunner pid=2099163) step:40 - global_seqlen/min:15913 - global_seqlen/max:27702 - global_seqlen/minmax_diff:11789 - global_seqlen/balanced_min:21440 - global_seqlen/balanced_max:22074 - global_seqlen/mean:21638.125 - actor/entropy:0.2002803236246109 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28941571712493896 - training/rollout_probs_diff_mean:0.00437137670814991 - training/rollout_probs_diff_std:0.012427032925188541 - training/rollout_actor_probs_pearson_corr:0.998335063457489 - rollout_corr/rollout_is_mean:1.000082015991211 - rollout_corr/rollout_is_ratio_fraction_high:2.1701152945752256e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.425288327387534e-05 - rollout_corr/rollout_is_max:2.1118814945220947 - rollout_corr/rollout_is_min:0.3157401978969574 - rollout_corr/rollout_is_std:0.037414900958538055 - rollout_corr/rollout_is_eff_sample_size:0.9986024388513428 - rollout_corr/rollout_is_seq_mean:0.9999911785125732 - rollout_corr/rollout_is_seq_std:0.002646325621753931 - rollout_corr/rollout_is_seq_max:1.0076828002929688 - rollout_corr/rollout_is_seq_min:0.99074786901474 - rollout_corr/rollout_is_seq_max_deviation:0.00925213098526001 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2884388542734087) - rollout_corr/training_log_ppl:np.float64(0.24909779269364662) - rollout_corr/kl:np.float64(0.001254979046022564) - rollout_corr/k3_kl:np.float64(0.0012163868613583873) - rollout_corr/rollout_ppl:np.float64(1.2867698702029884) - rollout_corr/rollout_log_ppl:np.float64(0.2478428122412879) - rollout_corr/log_ppl_diff:np.float64(0.0012549804523587227) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025399263540748507) - rollout_corr/log_ppl_diff_max:np.float64(0.013394579291343689) - rollout_corr/log_ppl_diff_min:np.float64(-0.006626993417739868) - rollout_corr/ppl_ratio:np.float64(1.001261416822672) - rollout_corr/chi2_token:np.float64(0.002296414924785495) - rollout_corr/chi2_seq:np.float64(0.5827651822473854) - actor/pg_loss:np.float64(0.00018049252685159445) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001993459543427889) - actor/ppo_kl:np.float64(0.00036094990030921537) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.7396467328071594) - perf/mfu/actor:np.float64(0.06226110744290928) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.27344131469727) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.35375) - val-aux/sciknoweval/reward/std@16:np.float64(0.3395231260295747) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5024000000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.31451708561123753) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.20027999999999999) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.26304832865625566) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.34982000000000013) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.33924744416949965) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6401399999999999) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.23702579096468043) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.10339999999999999) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1561668404485443) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.3715800000000001) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.30096314091120385) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7321000000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.1335033733578523) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.05524) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07954253083945868) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.38284) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.25173507835666453) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.77508) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.05977639622221202) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.02734) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.046689905811973874) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.39382) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.21182267262392962) - val-aux/sciknoweval/score/mean@16:np.float64(0.35375) - val-aux/sciknoweval/score/std@16:np.float64(0.3395231260295747) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.5024000000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.31451708561123753) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.20027999999999999) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.26304832865625566) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.34982000000000013) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.33924744416949965) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6401399999999999) - val-aux/sciknoweval/score/best@4/std:np.float64(0.23702579096468043) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.10339999999999999) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1561668404485443) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.3715800000000001) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.30096314091120385) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7321000000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.1335033733578523) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.05524) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07954253083945868) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.38284) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.25173507835666453) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.77508) - val-aux/sciknoweval/score/best@16/std:np.float64(0.05977639622221202) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.02734) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.046689905811973874) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.39382) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.21182267262392962) - val-core/sciknoweval/acc/mean@16:np.float64(0.35375) - val-aux/sciknoweval/acc/std@16:np.float64(0.3395231260295747) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.5024000000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.31451708561123753) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.20027999999999999) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.26304832865625566) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.34982000000000013) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.33924744416949965) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6401399999999999) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.23702579096468043) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.10339999999999999) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1561668404485443) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.3715800000000001) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.30096314091120385) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7321000000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.1335033733578523) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.05524) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07954253083945868) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.38284) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.25173507835666453) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.77508) - val-core/sciknoweval/acc/best@16/std:np.float64(0.05977639622221202) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.02734) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.046689905811973874) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.39382) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.21182267262392962) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.94375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.10821442299182672) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9848399999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.05425277298355868) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.89878) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.12778925051495435) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9417399999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.10929119954355648) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9982199999999999) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.013301438756809114) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.84614) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.13137340985952187) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.97116) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.07631025659993602) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9999600000000001) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0012642784503423287) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.7881) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.11433821446278315) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.98946) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.042506343696145) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.7379399999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.07740453538749054) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99836) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.014501222079305633) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:2 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008019932545721531 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008019932545721531 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:360.00390625 - response_length/max:2721.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:360.00390625 - response_length_non_aborted/max:2721.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:316.1875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.67449426651001e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9381169620901346) - timing_s/agent_loop/generate_sequences/max:np.float64(15.55611569993198) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4685358870920027) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.55611569993198) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:2721 - timing_s/gen:21.971412979066372 - timing_s/reward:0.06056964956223965 - timing_s/old_log_prob:2.4427013341337442 - timing_s/adv:0.12022142671048641 - timing_s/update_actor:9.820722814649343 - timing_s/step:34.50256798416376 - timing_s/testing:96.73482717573643 - timing_s/save_checkpoint:6.927775839343667 - timing_s/stop_profile:3.5040080547332764e-05 - timing_per_token_ms/gen:0.23840250191584697 - timing_per_token_ms/update_actor:0.05673275072730045 - timing_per_token_ms/adv:0.000694500024323309 - perf/total_num_tokens:173105 - perf/time_per_step:34.50256798416376 - perf/throughput:627.1453478457495 (TaskRunner pid=2099163) Training Progress: 40%|████ | 40/100 [34:59<1:10:59, 70.99s/it] (TaskRunner pid=2099163) step:41 - global_seqlen/min:13619 - global_seqlen/max:27244 - global_seqlen/minmax_diff:13625 - global_seqlen/balanced_min:22065 - global_seqlen/balanced_max:22100 - global_seqlen/mean:22093.0 - actor/entropy:0.1972004473209381 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7945776581764221 - training/rollout_probs_diff_mean:0.0044371881522238255 - training/rollout_probs_diff_std:0.013081462122499943 - training/rollout_actor_probs_pearson_corr:0.9981732368469238 - rollout_corr/rollout_is_mean:1.0000603199005127 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.645401299465448e-05 - rollout_corr/rollout_is_max:1.7906994819641113 - rollout_corr/rollout_is_min:0.14579379558563232 - rollout_corr/rollout_is_std:0.037738487124443054 - rollout_corr/rollout_is_eff_sample_size:0.9985780698008809 - rollout_corr/rollout_is_seq_mean:1.0000441074371338 - rollout_corr/rollout_is_seq_std:0.0030684443190693855 - rollout_corr/rollout_is_seq_max:1.007584571838379 - rollout_corr/rollout_is_seq_min:0.9881037473678589 - rollout_corr/rollout_is_seq_max_deviation:0.011896252632141113 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2687957105226815) - rollout_corr/training_log_ppl:np.float64(0.23422167608805466) - rollout_corr/kl:np.float64(0.001997436562771071) - rollout_corr/k3_kl:np.float64(0.001895930330306328) - rollout_corr/rollout_ppl:np.float64(1.2661805939860642) - rollout_corr/rollout_log_ppl:np.float64(0.2322242384107085) - rollout_corr/log_ppl_diff:np.float64(0.00199743767734617) - rollout_corr/log_ppl_abs_diff:np.float64(0.0037411078810691833) - rollout_corr/log_ppl_diff_max:np.float64(0.03718364238739014) - rollout_corr/log_ppl_diff_min:np.float64(-0.030967384576797485) - rollout_corr/ppl_ratio:np.float64(1.002018980216235) - rollout_corr/chi2_token:np.float64(0.0035980751272290945) - rollout_corr/chi2_seq:np.float64(2.2225923924706876) - actor/pg_loss:np.float64(0.0005501225823536515) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0022654881254311476) - actor/ppo_kl:np.float64(0.001161328320471) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.7245654910802841) - perf/mfu/actor:np.float64(0.06373080366245223) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.36378860473633) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:2 - critic/score/mean:0.48828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0027744551189243793 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0027744551189243793 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:352.6875 - response_length/max:1485.0 - response_length/min:79.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:352.6875 - response_length_non_aborted/max:1485.0 - response_length_non_aborted/min:79.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:337.71875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.8796494007110596e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.95299425534904) - timing_s/agent_loop/generate_sequences/max:np.float64(9.864370591938496) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4211922848553513) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.864370591938496) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:1485 - timing_s/gen:15.542456863448024 - timing_s/reward:0.05860104784369469 - timing_s/old_log_prob:2.535149410367012 - timing_s/adv:0.1277296431362629 - timing_s/update_actor:9.861975859850645 - timing_s/step:28.19834844954312 - timing_s/stop_profile:0.00011692754924297333 - timing_per_token_ms/gen:0.17214310720636214 - timing_per_token_ms/update_actor:0.05579808004713396 - timing_per_token_ms/adv:0.0007226816363568941 - perf/total_num_tokens:176744 - perf/time_per_step:28.19834844954312 - perf/throughput:783.4856016313239 (TaskRunner pid=2099163) Training Progress: 41%|████ | 41/100 [35:27<57:11, 58.17s/it] (TaskRunner pid=2099163) step:42 - global_seqlen/min:11327 - global_seqlen/max:31713 - global_seqlen/minmax_diff:20386 - global_seqlen/balanced_min:16962 - global_seqlen/balanced_max:23405 - global_seqlen/mean:17874.375 - actor/entropy:0.1793491095304489 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2369065284729004 - training/rollout_probs_diff_mean:0.003962434828281403 - training/rollout_probs_diff_std:0.011616604402661324 - training/rollout_actor_probs_pearson_corr:0.9984515309333801 - rollout_corr/rollout_is_mean:1.0001260042190552 - rollout_corr/rollout_is_ratio_fraction_high:1.2704059372481424e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.3973875045776367 - rollout_corr/rollout_is_min:0.5620211958885193 - rollout_corr/rollout_is_std:0.03498118743300438 - rollout_corr/rollout_is_eff_sample_size:0.9987781689679683 - rollout_corr/rollout_is_seq_mean:1.0004432201385498 - rollout_corr/rollout_is_seq_std:0.003099439898505807 - rollout_corr/rollout_is_seq_max:1.010513424873352 - rollout_corr/rollout_is_seq_min:0.9920833706855774 - rollout_corr/rollout_is_seq_max_deviation:0.01051342487335205 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2783533367328346) - rollout_corr/training_log_ppl:np.float64(0.24186358234874206) - rollout_corr/kl:np.float64(0.0008186687481881449) - rollout_corr/k3_kl:np.float64(0.0014607259122385585) - rollout_corr/rollout_ppl:np.float64(1.2772352620959282) - rollout_corr/rollout_log_ppl:np.float64(0.24104491147591034) - rollout_corr/log_ppl_diff:np.float64(0.0008186708728317171) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031197732605505735) - rollout_corr/log_ppl_diff_max:np.float64(0.023270994424819946) - rollout_corr/log_ppl_diff_min:np.float64(-0.01981446146965027) - rollout_corr/ppl_ratio:np.float64(1.0008287110831589) - rollout_corr/chi2_token:np.float64(0.004162837052717805) - rollout_corr/chi2_seq:np.float64(0.8917187473271042) - actor/pg_loss:np.float64(0.000158449518494308) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011291635195220806) - actor/ppo_kl:np.float64(0.0004214342557844475) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5698883533477783) - perf/mfu/actor:np.float64(0.0494323141668247) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.41558456420898) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:2 - critic/score/mean:0.328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03640824556350708 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.03640824556350708 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:307.48046875 - response_length/max:8192.0 - response_length/min:79.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:307.48046875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:79.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:251.09375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000103706493973732 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9703596197068691) - timing_s/agent_loop/generate_sequences/max:np.float64(41.32044994644821) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.750317184647429) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(41.32044994644821) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:46.890859078615904 - timing_s/reward:0.05139863304793835 - timing_s/old_log_prob:2.586537519469857 - timing_s/adv:0.1239897795021534 - timing_s/update_actor:10.514672089368105 - timing_s/step:60.26146995835006 - timing_s/stop_profile:0.0001133400946855545 - timing_per_token_ms/gen:0.5957042378023999 - timing_per_token_ms/update_actor:0.07353174649021368 - timing_per_token_ms/adv:0.0008670917130120172 - perf/total_num_tokens:142995 - perf/time_per_step:60.26146995835006 - perf/throughput:296.6136573228954 (TaskRunner pid=2099163) Training Progress: 42%|████▏ | 42/100 [36:27<56:51, 58.81s/it] (TaskRunner pid=2099163) step:43 - global_seqlen/min:14105 - global_seqlen/max:31969 - global_seqlen/minmax_diff:17864 - global_seqlen/balanced_min:21216 - global_seqlen/balanced_max:21234 - global_seqlen/mean:21226.625 - actor/entropy:0.19266550242900848 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44984614849090576 - training/rollout_probs_diff_mean:0.004387952387332916 - training/rollout_probs_diff_std:0.01266084611415863 - training/rollout_actor_probs_pearson_corr:0.9982303977012634 - rollout_corr/rollout_is_mean:0.9999305605888367 - rollout_corr/rollout_is_ratio_fraction_high:3.408587508602068e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.544783223536797e-05 - rollout_corr/rollout_is_max:2.5337352752685547 - rollout_corr/rollout_is_min:0.32041898369789124 - rollout_corr/rollout_is_std:0.038009174168109894 - rollout_corr/rollout_is_eff_sample_size:0.9985570301628549 - rollout_corr/rollout_is_seq_mean:0.9998732805252075 - rollout_corr/rollout_is_seq_std:0.002512736711651087 - rollout_corr/rollout_is_seq_max:1.008451223373413 - rollout_corr/rollout_is_seq_min:0.9910997748374939 - rollout_corr/rollout_is_seq_max_deviation:0.008900225162506104 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2575704725459218) - rollout_corr/training_log_ppl:np.float64(0.22557678240991663) - rollout_corr/kl:np.float64(0.0013313335243312352) - rollout_corr/k3_kl:np.float64(0.0010828630935861838) - rollout_corr/rollout_ppl:np.float64(1.2558402898721397) - rollout_corr/rollout_log_ppl:np.float64(0.22424544939713087) - rollout_corr/log_ppl_diff:np.float64(0.0013313330127857625) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023614601814188063) - rollout_corr/log_ppl_diff_max:np.float64(0.011232882738113403) - rollout_corr/log_ppl_diff_min:np.float64(-0.007203608751296997) - rollout_corr/ppl_ratio:np.float64(1.0013366700150073) - rollout_corr/chi2_token:np.float64(0.0016005998477339745) - rollout_corr/chi2_seq:np.float64(0.9111351130995899) - actor/pg_loss:np.float64(-0.00032677757553756237) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0019711919426299573) - actor/ppo_kl:np.float64(0.00035553486237560605) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6708932965993881) - perf/mfu/actor:np.float64(0.06152323141502927) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.48104095458984) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:3 - critic/score/mean:0.44140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0011432970641180873 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0011432970641180873 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:343.80078125 - response_length/max:1356.0 - response_length/min:83.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:343.80078125 - response_length_non_aborted/max:1356.0 - response_length_non_aborted/min:83.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:319.53125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002473127096891403 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0578165259212255) - timing_s/agent_loop/generate_sequences/max:np.float64(9.275035005062819) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.464938813202025) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.275035005062819) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:1356 - timing_s/gen:15.193364929407835 - timing_s/reward:0.05014399252831936 - timing_s/old_log_prob:2.6355569884181023 - timing_s/adv:0.1556774377822876 - timing_s/update_actor:9.845597103238106 - timing_s/step:27.98137691617012 - timing_s/stop_profile:4.378892481327057e-05 - timing_per_token_ms/gen:0.17262637257459507 - timing_per_token_ms/update_actor:0.05797905403731225 - timing_per_token_ms/adv:0.0009167580678881334 - perf/total_num_tokens:169813 - perf/time_per_step:27.98137691617012 - perf/throughput:758.5983014200197 (TaskRunner pid=2099163) Training Progress: 43%|████▎ | 43/100 [36:56<47:12, 49.70s/it] (TaskRunner pid=2099163) step:44 - global_seqlen/min:14289 - global_seqlen/max:31868 - global_seqlen/minmax_diff:17579 - global_seqlen/balanced_min:23024 - global_seqlen/balanced_max:27562 - global_seqlen/mean:23664.625 - actor/entropy:0.16483263671398163 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5307222604751587 - training/rollout_probs_diff_mean:0.0037482562474906445 - training/rollout_probs_diff_std:0.011952490545809269 - training/rollout_actor_probs_pearson_corr:0.9982149600982666 - rollout_corr/rollout_is_mean:0.9997656941413879 - rollout_corr/rollout_is_ratio_fraction_high:1.877493559732102e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.754987119464204e-05 - rollout_corr/rollout_is_max:2.257253646850586 - rollout_corr/rollout_is_min:0.2560407221317291 - rollout_corr/rollout_is_std:0.03545422852039337 - rollout_corr/rollout_is_eff_sample_size:0.9987439217121525 - rollout_corr/rollout_is_seq_mean:0.9996024966239929 - rollout_corr/rollout_is_seq_std:0.002828868106007576 - rollout_corr/rollout_is_seq_max:1.0087915658950806 - rollout_corr/rollout_is_seq_min:0.9925878047943115 - rollout_corr/rollout_is_seq_max_deviation:0.008791565895080566 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2643110915087163) - rollout_corr/training_log_ppl:np.float64(0.23016513518814463) - rollout_corr/kl:np.float64(0.0011426463696224687) - rollout_corr/k3_kl:np.float64(0.0011375640360711259) - rollout_corr/rollout_ppl:np.float64(1.2628552531823516) - rollout_corr/rollout_log_ppl:np.float64(0.22902248779064394) - rollout_corr/log_ppl_diff:np.float64(0.0011426473975006957) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027303947317705024) - rollout_corr/log_ppl_diff_max:np.float64(0.012277275323867798) - rollout_corr/log_ppl_diff_min:np.float64(-0.010143131017684937) - rollout_corr/ppl_ratio:np.float64(1.0011494110804051) - rollout_corr/chi2_token:np.float64(0.0023959616664797068) - rollout_corr/chi2_seq:np.float64(1.1097935638390481) - actor/pg_loss:np.float64(0.0004005907103419304) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017527463992337289) - actor/ppo_kl:np.float64(-0.0001501098105372023) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6244964599609375) - perf/mfu/actor:np.float64(0.06486412678751474) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.46553421020508) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:3 - critic/score/mean:0.4375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03883712738752365 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.03883712738752365 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:416.11328125 - response_length/max:7121.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:416.11328125 - response_length_non_aborted/max:7121.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:323.40625 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.310292959213257e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.854373425245285) - timing_s/agent_loop/generate_sequences/max:np.float64(35.90070942789316) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.641139167644724) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(35.90070942789316) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:7121 - timing_s/gen:41.78276102989912 - timing_s/reward:0.05910661816596985 - timing_s/old_log_prob:2.6271430384367704 - timing_s/adv:0.12738168239593506 - timing_s/update_actor:10.614613642916083 - timing_s/step:55.30339947156608 - timing_s/stop_profile:0.00013011880218982697 - timing_per_token_ms/gen:0.39223432086270005 - timing_per_token_ms/update_actor:0.0560679370733536 - timing_per_token_ms/adv:0.0006728486210743624 - perf/total_num_tokens:189317 - perf/time_per_step:55.30339947156608 - perf/throughput:427.9054312414742 (TaskRunner pid=2099163) Training Progress: 44%|████▍ | 44/100 [37:51<47:58, 51.40s/it] (TaskRunner pid=2099163) step:45 - global_seqlen/min:10341 - global_seqlen/max:43510 - global_seqlen/minmax_diff:33169 - global_seqlen/balanced_min:21899 - global_seqlen/balanced_max:27874 - global_seqlen/mean:22856.25 - actor/entropy:0.1471649408340454 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3509632647037506 - training/rollout_probs_diff_mean:0.0035214414820075035 - training/rollout_probs_diff_std:0.01150407176464796 - training/rollout_actor_probs_pearson_corr:0.9982120394706726 - rollout_corr/rollout_is_mean:0.9999986290931702 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.89342778059654e-05 - rollout_corr/rollout_is_max:1.867427945137024 - rollout_corr/rollout_is_min:0.030966857448220253 - rollout_corr/rollout_is_std:0.03498629853129387 - rollout_corr/rollout_is_eff_sample_size:0.9987774554595162 - rollout_corr/rollout_is_seq_mean:0.9998350739479065 - rollout_corr/rollout_is_seq_std:0.002756875241175294 - rollout_corr/rollout_is_seq_max:1.0082145929336548 - rollout_corr/rollout_is_seq_min:0.990932822227478 - rollout_corr/rollout_is_seq_max_deviation:0.009067177772521973 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2405082695186138) - rollout_corr/training_log_ppl:np.float64(0.21127635941229528) - rollout_corr/kl:np.float64(0.0012804620336837047) - rollout_corr/k3_kl:np.float64(0.0011797188574576012) - rollout_corr/rollout_ppl:np.float64(1.2388938385993242) - rollout_corr/rollout_log_ppl:np.float64(0.20999589761777315) - rollout_corr/log_ppl_diff:np.float64(0.0012804617945221253) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025750486165634356) - rollout_corr/log_ppl_diff_max:np.float64(0.01426708698272705) - rollout_corr/log_ppl_diff_min:np.float64(-0.016405731439590454) - rollout_corr/ppl_ratio:np.float64(1.001287407707423) - rollout_corr/chi2_token:np.float64(0.0022307620383799076) - rollout_corr/chi2_seq:np.float64(0.5359919595066458) - actor/pg_loss:np.float64(8.05326271802187e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001362902614914674) - actor/ppo_kl:np.float64(0.0002342196375835215) - actor/pg_clipfrac_lower:np.float64(7.356402875302592e-06) - actor/grad_norm:np.float64(0.6136828362941742) - perf/mfu/actor:np.float64(0.062255809404529946) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.4916729927063) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:3 - critic/score/mean:0.31640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.31640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0037901541218161583 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0037901541218161583 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:396.6640625 - response_length/max:8160.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:396.6640625 - response_length_non_aborted/max:8160.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:317.59375 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.038004696369171e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9433151036500931) - timing_s/agent_loop/generate_sequences/max:np.float64(41.46591006591916) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.561986893051653) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(41.46591006591916) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:8160 - timing_s/gen:47.15192976593971 - timing_s/reward:0.059990689158439636 - timing_s/old_log_prob:2.561908131465316 - timing_s/adv:0.12769671715795994 - timing_s/update_actor:10.679077072069049 - timing_s/step:60.668493922799826 - timing_s/stop_profile:0.00013370439410209656 - timing_per_token_ms/gen:0.4643405921054469 - timing_per_token_ms/update_actor:0.0584034841239762 - timing_per_token_ms/adv:0.0006983687019850147 - perf/total_num_tokens:182850 - perf/time_per_step:60.668493922799826 - perf/throughput:376.7400263649926 (TaskRunner pid=2099163) Training Progress: 45%|████▌ | 45/100 [38:52<49:40, 54.19s/it] (TaskRunner pid=2099163) step:46 - global_seqlen/min:11629 - global_seqlen/max:28235 - global_seqlen/minmax_diff:16606 - global_seqlen/balanced_min:17711 - global_seqlen/balanced_max:17919 - global_seqlen/mean:17760.75 - actor/entropy:0.20734380185604095 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5599408149719238 - training/rollout_probs_diff_mean:0.004730684217065573 - training/rollout_probs_diff_std:0.013396089896559715 - training/rollout_actor_probs_pearson_corr:0.9980821013450623 - rollout_corr/rollout_is_mean:1.0000985860824585 - rollout_corr/rollout_is_ratio_fraction_high:2.7359032173990272e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.3679516086995136e-05 - rollout_corr/rollout_is_max:5.697536468505859 - rollout_corr/rollout_is_min:0.4908820390701294 - rollout_corr/rollout_is_std:0.04028098285198212 - rollout_corr/rollout_is_eff_sample_size:0.9983800707884429 - rollout_corr/rollout_is_seq_mean:1.000237226486206 - rollout_corr/rollout_is_seq_std:0.00363980233669281 - rollout_corr/rollout_is_seq_max:1.021539568901062 - rollout_corr/rollout_is_seq_min:0.9916234016418457 - rollout_corr/rollout_is_seq_max_deviation:0.02153956890106201 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2810124992392957) - rollout_corr/training_log_ppl:np.float64(0.24381673078460153) - rollout_corr/kl:np.float64(0.0006558897876516312) - rollout_corr/k3_kl:np.float64(0.0010635649665289293) - rollout_corr/rollout_ppl:np.float64(1.2801448996178806) - rollout_corr/rollout_log_ppl:np.float64(0.24316083974554203) - rollout_corr/log_ppl_diff:np.float64(0.0006558910390594974) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025359629216836765) - rollout_corr/log_ppl_diff_max:np.float64(0.009192198514938354) - rollout_corr/log_ppl_diff_min:np.float64(-0.009751826524734497) - rollout_corr/ppl_ratio:np.float64(1.0006613184232265) - rollout_corr/chi2_token:np.float64(0.003016318893060088) - rollout_corr/chi2_seq:np.float64(0.5328007256612182) - actor/pg_loss:np.float64(-6.973580457270145e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015097170539775107) - actor/ppo_kl:np.float64(-0.00010064278021815198) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.563657782971859) - perf/mfu/actor:np.float64(0.051583663196279474) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.5073127746582) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:3 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.023010656237602234 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.023010656237602234 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:285.5546875 - response_length/max:1823.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:285.5546875 - response_length_non_aborted/max:1823.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.46875 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.830668568611145e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9158236924558878) - timing_s/agent_loop/generate_sequences/max:np.float64(10.958448996767402) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.764639292836364) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.958448996767402) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:1823 - timing_s/gen:16.863597629591823 - timing_s/reward:0.053841013461351395 - timing_s/old_log_prob:2.380986912176013 - timing_s/adv:0.14236685261130333 - timing_s/update_actor:9.776626717299223 - timing_s/step:29.306406520307064 - timing_s/stop_profile:2.9392540454864502e-05 - timing_per_token_ms/gen:0.23068585852085885 - timing_per_token_ms/update_actor:0.06880781158804684 - timing_per_token_ms/adv:0.0010019766381719757 - perf/total_num_tokens:142086 - perf/time_per_step:29.306406520307064 - perf/throughput:606.036430556342 (TaskRunner pid=2099163) Training Progress: 46%|████▌ | 46/100 [39:21<42:03, 46.73s/it] (TaskRunner pid=2099163) step:47 - global_seqlen/min:18567 - global_seqlen/max:38675 - global_seqlen/minmax_diff:20108 - global_seqlen/balanced_min:25878 - global_seqlen/balanced_max:26514 - global_seqlen/mean:25983.75 - actor/entropy:0.14283302426338196 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7308436632156372 - training/rollout_probs_diff_mean:0.003343928838148713 - training/rollout_probs_diff_std:0.011697083711624146 - training/rollout_actor_probs_pearson_corr:0.9980596899986267 - rollout_corr/rollout_is_mean:1.0000288486480713 - rollout_corr/rollout_is_ratio_fraction_high:2.6085595891345292e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001130375821958296 - rollout_corr/rollout_is_max:2.327317714691162 - rollout_corr/rollout_is_min:0.23235759139060974 - rollout_corr/rollout_is_std:0.03357976675033569 - rollout_corr/rollout_is_eff_sample_size:0.9988737883222161 - rollout_corr/rollout_is_seq_mean:0.9999798536300659 - rollout_corr/rollout_is_seq_std:0.0024428016040474176 - rollout_corr/rollout_is_seq_max:1.0096776485443115 - rollout_corr/rollout_is_seq_min:0.9921210408210754 - rollout_corr/rollout_is_seq_max_deviation:0.009677648544311523 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2100329943932593) - rollout_corr/training_log_ppl:np.float64(0.18659601080435095) - rollout_corr/kl:np.float64(0.0011015117767527727) - rollout_corr/k3_kl:np.float64(0.0009561012871870389) - rollout_corr/rollout_ppl:np.float64(1.208669331856072) - rollout_corr/rollout_log_ppl:np.float64(0.18549449724378064) - rollout_corr/log_ppl_diff:np.float64(0.0011015135605703108) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020468592338147573) - rollout_corr/log_ppl_diff_max:np.float64(0.013909026980400085) - rollout_corr/log_ppl_diff_min:np.float64(-0.007575169205665588) - rollout_corr/ppl_ratio:np.float64(1.001105838920921) - rollout_corr/chi2_token:np.float64(0.001610573148354888) - rollout_corr/chi2_seq:np.float64(0.26548866275697947) - actor/pg_loss:np.float64(4.338508006185293e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011870542070937518) - actor/ppo_kl:np.float64(0.0002975921167021056) - actor/pg_clipfrac_lower:np.float64(9.322792720922735e-06) - actor/grad_norm:np.float64(0.5477600991725922) - perf/mfu/actor:np.float64(0.07517353223146761) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.55639266967773) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:3 - critic/score/mean:0.39453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.39453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0002075978700304404 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0002075978700304404 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:449.2421875 - response_length/max:2732.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:449.2421875 - response_length_non_aborted/max:2732.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:362.75 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.532266080379486e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9549492709338665) - timing_s/agent_loop/generate_sequences/max:np.float64(15.872619692236185) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.19089348961279) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.872619692236185) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:2732 - timing_s/gen:21.63072121143341 - timing_s/reward:0.06235244311392307 - timing_s/old_log_prob:2.547062236815691 - timing_s/adv:0.1267538219690323 - timing_s/update_actor:9.819873118773103 - timing_s/step:34.27366493828595 - timing_s/stop_profile:0.00012672320008277893 - timing_per_token_ms/gen:0.18808341487777516 - timing_per_token_ms/update_actor:0.04724045373922693 - timing_per_token_ms/adv:0.0006097744839035565 - perf/total_num_tokens:207870 - perf/time_per_step:34.27366493828595 - perf/throughput:758.1258102040449 (TaskRunner pid=2099163) Training Progress: 47%|████▋ | 47/100 [39:55<37:59, 43.01s/it] (TaskRunner pid=2099163) step:48 - global_seqlen/min:16381 - global_seqlen/max:32235 - global_seqlen/minmax_diff:15854 - global_seqlen/balanced_min:21840 - global_seqlen/balanced_max:27905 - global_seqlen/mean:22746.0 - actor/entropy:0.16609838604927063 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5618129372596741 - training/rollout_probs_diff_mean:0.0038480223156511784 - training/rollout_probs_diff_std:0.012091293931007385 - training/rollout_actor_probs_pearson_corr:0.9981785416603088 - rollout_corr/rollout_is_mean:1.0000433921813965 - rollout_corr/rollout_is_ratio_fraction_high:1.9581733795348555e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.8745201386045665e-05 - rollout_corr/rollout_is_max:2.1106479167938232 - rollout_corr/rollout_is_min:0.0005965463351458311 - rollout_corr/rollout_is_std:0.03607833385467529 - rollout_corr/rollout_is_eff_sample_size:0.9987001647473916 - rollout_corr/rollout_is_seq_mean:1.0000697374343872 - rollout_corr/rollout_is_seq_std:0.0026138562243431807 - rollout_corr/rollout_is_seq_max:1.010244369506836 - rollout_corr/rollout_is_seq_min:0.99269038438797 - rollout_corr/rollout_is_seq_max_deviation:0.010244369506835938 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.260544263292104) - rollout_corr/training_log_ppl:np.float64(0.2275718234195665) - rollout_corr/kl:np.float64(0.000840849202081273) - rollout_corr/k3_kl:np.float64(0.0011277189190366244) - rollout_corr/rollout_ppl:np.float64(1.2594485664740205) - rollout_corr/rollout_log_ppl:np.float64(0.2267309735470917) - rollout_corr/log_ppl_diff:np.float64(0.0008408498724747915) - rollout_corr/log_ppl_abs_diff:np.float64(0.002264020313305082) - rollout_corr/log_ppl_diff_max:np.float64(0.02813515067100525) - rollout_corr/log_ppl_diff_min:np.float64(-0.008843004703521729) - rollout_corr/ppl_ratio:np.float64(1.0008466544095427) - rollout_corr/chi2_token:np.float64(0.0026712415274232626) - rollout_corr/chi2_seq:np.float64(0.9579408683348447) - actor/pg_loss:np.float64(1.6385456547141075e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00142442833248424) - actor/ppo_kl:np.float64(-7.805178177022754e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5536335557699203) - perf/mfu/actor:np.float64(0.06172036956944962) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.46680450439453) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:3 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0621769018471241 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0621769018471241 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:398.96875 - response_length/max:8192.0 - response_length/min:81.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:398.96875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:311.84375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.11615788936615e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5817424338310957) - timing_s/agent_loop/generate_sequences/max:np.float64(47.18641279824078) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3139364701564773) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(47.18641279824078) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:52.73312545567751 - timing_s/reward:0.06915519759058952 - timing_s/old_log_prob:2.6080319173634052 - timing_s/adv:0.13032598234713078 - timing_s/update_actor:10.7602223996073 - timing_s/step:66.35072775743902 - timing_s/stop_profile:0.0001078825443983078 - timing_per_token_ms/gen:0.5163030220067117 - timing_per_token_ms/update_actor:0.05913249801947211 - timing_per_token_ms/adv:0.0007162027518417017 - perf/total_num_tokens:181968 - perf/time_per_step:66.35072775743902 - perf/throughput:342.81462719072886 (TaskRunner pid=2099163) Training Progress: 48%|████▊ | 48/100 [41:02<43:21, 50.03s/it] (TaskRunner pid=2099163) step:49 - global_seqlen/min:13330 - global_seqlen/max:23649 - global_seqlen/minmax_diff:10319 - global_seqlen/balanced_min:19645 - global_seqlen/balanced_max:19656 - global_seqlen/mean:19652.75 - actor/entropy:0.21312448382377625 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29631492495536804 - training/rollout_probs_diff_mean:0.004599376115947962 - training/rollout_probs_diff_std:0.012834436260163784 - training/rollout_actor_probs_pearson_corr:0.9982872009277344 - rollout_corr/rollout_is_mean:0.9999762773513794 - rollout_corr/rollout_is_ratio_fraction_high:1.2280485861992929e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.684145849547349e-05 - rollout_corr/rollout_is_max:2.1531147956848145 - rollout_corr/rollout_is_min:0.16506630182266235 - rollout_corr/rollout_is_std:0.03860819712281227 - rollout_corr/rollout_is_eff_sample_size:0.9985116256038487 - rollout_corr/rollout_is_seq_mean:0.999916136264801 - rollout_corr/rollout_is_seq_std:0.002757440321147442 - rollout_corr/rollout_is_seq_max:1.011631965637207 - rollout_corr/rollout_is_seq_min:0.9919342398643494 - rollout_corr/rollout_is_seq_max_deviation:0.011631965637207031 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.290575869847089) - rollout_corr/training_log_ppl:np.float64(0.2508397557976423) - rollout_corr/kl:np.float64(0.0012681070296984842) - rollout_corr/k3_kl:np.float64(0.0010787070739013416) - rollout_corr/rollout_ppl:np.float64(1.2888763579539955) - rollout_corr/rollout_log_ppl:np.float64(0.24957164909574203) - rollout_corr/log_ppl_diff:np.float64(0.0012681067019002512) - rollout_corr/log_ppl_abs_diff:np.float64(0.002345262750168331) - rollout_corr/log_ppl_diff_max:np.float64(0.009302854537963867) - rollout_corr/log_ppl_diff_min:np.float64(-0.009885609149932861) - rollout_corr/ppl_ratio:np.float64(1.0012729486916214) - rollout_corr/chi2_token:np.float64(0.0018082375172525644) - rollout_corr/chi2_seq:np.float64(0.36839555972255766) - actor/pg_loss:np.float64(4.909001290798187e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016076876192983036) - actor/ppo_kl:np.float64(0.0002880832484706275) - actor/pg_clipfrac_lower:np.float64(4.828491910302546e-06) - actor/grad_norm:np.float64(0.629483163356781) - perf/mfu/actor:np.float64(0.05734176798977645) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.61180114746094) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:3 - critic/score/mean:0.34765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0032481886446475983 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0032481886446475983 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:318.0859375 - response_length/max:1276.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:318.0859375 - response_length_non_aborted/max:1276.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.0625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.732825517654419e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8680097497999668) - timing_s/agent_loop/generate_sequences/max:np.float64(8.407540844753385) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0850135255313944) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.407540844753385) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:1276 - timing_s/gen:14.08630795404315 - timing_s/reward:0.05711162090301514 - timing_s/old_log_prob:2.3311667516827583 - timing_s/adv:0.1224974524229765 - timing_s/update_actor:9.94140362367034 - timing_s/step:26.626789750531316 - timing_s/stop_profile:0.00010458938777446747 - timing_per_token_ms/gen:0.17298671194944307 - timing_per_token_ms/update_actor:0.06323163185604012 - timing_per_token_ms/adv:0.0007791368410462689 - perf/total_num_tokens:157222 - perf/time_per_step:26.626789750531316 - perf/throughput:738.0818410378535 (TaskRunner pid=2099163) Training Progress: 49%|████▉ | 49/100 [41:29<36:34, 43.02s/it] (TaskRunner pid=2099163) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2099163) validation generation end (TaskRunner pid=2099163) [prompt] system (TaskRunner pid=2099163) (TaskRunner pid=2099163) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) user (TaskRunner pid=2099163) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2099163) (TaskRunner pid=2099163) A: -0.01 (TaskRunner pid=2099163) B: 1.69 (TaskRunner pid=2099163) C: 2.49 (TaskRunner pid=2099163) D: 0.45 (TaskRunner pid=2099163) Please reason step by step. (TaskRunner pid=2099163) assistant (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) [response] (TaskRunner pid=2099163) The folding stability score of a protein sequence typically requires computational analysis using tools like I-TASSER, ROSETTA, or other protein structure prediction and stability analysis software. These tools calculate scores based on the predicted 3D structure of the protein, taking into account factors such as secondary structure, solvent accessibility, and other structural properties. Without access to specific computational tools or databases that provide folding stability scores for the given sequence, it is not possible to determine the exact folding stability score for the sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'. Therefore, the score cannot be determined from the information provided. (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) D (TaskRunner pid=2099163) (TaskRunner pid=2099163) [ground_truth] B (TaskRunner pid=2099163) [score] 0.0 (TaskRunner pid=2099163) [acc] 0.0 (TaskRunner pid=2099163) [pred] D (TaskRunner pid=2099163) [incorrect_format] 1 (TaskRunner pid=2099163) [feedback] (TaskRunner pid=2099163) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (WorkerDict pid=2099539) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40/actor (TaskRunner pid=2099163) step:50 - global_seqlen/min:12100 - global_seqlen/max:26431 - global_seqlen/minmax_diff:14331 - global_seqlen/balanced_min:22098 - global_seqlen/balanced_max:22581 - global_seqlen/mean:22179.5 - actor/entropy:0.1872798055410385 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25002074241638184 - training/rollout_probs_diff_mean:0.0043448833748698235 - training/rollout_probs_diff_std:0.012745512649416924 - training/rollout_actor_probs_pearson_corr:0.9981958270072937 - rollout_corr/rollout_is_mean:0.9999893307685852 - rollout_corr/rollout_is_ratio_fraction_high:1.082672861230094e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.248018765589222e-05 - rollout_corr/rollout_is_max:2.2042269706726074 - rollout_corr/rollout_is_min:0.3166545629501343 - rollout_corr/rollout_is_std:0.03807654604315758 - rollout_corr/rollout_is_eff_sample_size:0.9985522755652161 - rollout_corr/rollout_is_seq_mean:0.9999541640281677 - rollout_corr/rollout_is_seq_std:0.0029842411167919636 - rollout_corr/rollout_is_seq_max:1.0160789489746094 - rollout_corr/rollout_is_seq_min:0.986756443977356 - rollout_corr/rollout_is_seq_max_deviation:0.016078948974609375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.264865230768919) - rollout_corr/training_log_ppl:np.float64(0.2308657732064603) - rollout_corr/kl:np.float64(0.001112062272518699) - rollout_corr/k3_kl:np.float64(0.0010981027096477192) - rollout_corr/rollout_ppl:np.float64(1.2634586868807673) - rollout_corr/rollout_log_ppl:np.float64(0.22975370922358707) - rollout_corr/log_ppl_diff:np.float64(0.0011120639828732237) - rollout_corr/log_ppl_abs_diff:np.float64(0.002492617248208262) - rollout_corr/log_ppl_diff_max:np.float64(0.01720789074897766) - rollout_corr/log_ppl_diff_min:np.float64(-0.013499855995178223) - rollout_corr/ppl_ratio:np.float64(1.001118621090427) - rollout_corr/chi2_token:np.float64(0.002215159824118018) - rollout_corr/chi2_seq:np.float64(0.9738334778230637) - actor/pg_loss:np.float64(0.00040720601100474596) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00143988847764831) - actor/ppo_kl:np.float64(0.00017129167446228166) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5727251768112183) - perf/mfu/actor:np.float64(0.06430968389502065) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.6324348449707) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.41875) - val-aux/sciknoweval/reward/std@16:np.float64(0.22869494240339183) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.51456) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.18468184508936822) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.31976) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.20223553757769963) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.41681999999999997) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.22791020438572854) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.58694) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1261578566460394) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.23558) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.15226752550720532) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.4398) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.18398851891396895) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.63446) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07544190394601284) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.17675999999999997) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09854951447854242) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.45147999999999994) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13395224221738825) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.66118) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03956012550827376) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.14170000000000002) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04890733027547041) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.45902) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.09216818857180725) - val-aux/sciknoweval/score/mean@16:np.float64(0.41875) - val-aux/sciknoweval/score/std@16:np.float64(0.22869494240339183) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.51456) - val-aux/sciknoweval/score/best@2/std:np.float64(0.18468184508936822) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.31976) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.20223553757769963) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.41681999999999997) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.22791020438572854) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.58694) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1261578566460394) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.23558) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.15226752550720532) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.4398) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.18398851891396895) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.63446) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07544190394601284) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.17675999999999997) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09854951447854242) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.45147999999999994) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.13395224221738825) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.66118) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03956012550827376) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.14170000000000002) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04890733027547041) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.45902) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.09216818857180725) - val-core/sciknoweval/acc/mean@16:np.float64(0.41875) - val-aux/sciknoweval/acc/std@16:np.float64(0.22869494240339183) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.51456) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.18468184508936822) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.31976) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.20223553757769963) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.41681999999999997) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.22791020438572854) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.58694) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1261578566460394) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.23558) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.15226752550720532) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.4398) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.18398851891396895) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.63446) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07544190394601284) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.17675999999999997) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09854951447854242) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.45147999999999994) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.13395224221738825) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.66118) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03956012550827376) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.14170000000000002) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04890733027547041) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.45902) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.09216818857180725) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.05008237914987659) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99824) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.015941849190385885) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.96934) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.06589422003315963) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.98418) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.04953093796525588) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.94512) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.08018857855516771) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99496) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.02826029051906959) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9103999999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.08613360783081132) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9989199999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.012363736848705086) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8677600000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.07445805556218542) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999600000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0008935323161475471) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:3 - critic/score/mean:0.51171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007015720475465059 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007015720475465059 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:360.796875 - response_length/max:2173.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:360.796875 - response_length_non_aborted/max:2173.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:332.3125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.383952081203461e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9896603114902973) - timing_s/agent_loop/generate_sequences/max:np.float64(12.962509341537952) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5228450542199425) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.962509341537952) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:425 - timing_s/agent_loop/slowest/response_length:2173 - timing_s/gen:18.663984855636954 - timing_s/reward:0.07062600553035736 - timing_s/old_log_prob:2.381566744297743 - timing_s/adv:0.1254611872136593 - timing_s/update_actor:9.75345753878355 - timing_s/step:31.04253143630922 - timing_s/testing:77.66123559139669 - timing_s/save_checkpoint:6.67768957093358 - timing_s/stop_profile:8.756667375564575e-05 - timing_per_token_ms/gen:0.20206990662635826 - timing_per_token_ms/update_actor:0.05496887632038341 - timing_per_token_ms/adv:0.0007070785365633766 - perf/total_num_tokens:177436 - perf/time_per_step:31.04253143630922 - perf/throughput:714.4874781074561 (TaskRunner pid=2099163) Training Progress: 50%|█████ | 50/100 [43:24<53:57, 64.75s/it] (TaskRunner pid=2099163) step:51 - global_seqlen/min:16904 - global_seqlen/max:25515 - global_seqlen/minmax_diff:8611 - global_seqlen/balanced_min:21289 - global_seqlen/balanced_max:21389 - global_seqlen/mean:21315.375 - actor/entropy:0.19681613147258759 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6418529748916626 - training/rollout_probs_diff_mean:0.004624536260962486 - training/rollout_probs_diff_std:0.013267040252685547 - training/rollout_actor_probs_pearson_corr:0.9980368614196777 - rollout_corr/rollout_is_mean:0.9997654557228088 - rollout_corr/rollout_is_ratio_fraction_high:2.3248514480656013e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.855645179748535 - rollout_corr/rollout_is_min:0.5350923538208008 - rollout_corr/rollout_is_std:0.03936876356601715 - rollout_corr/rollout_is_eff_sample_size:0.9984516075893486 - rollout_corr/rollout_is_seq_mean:0.9997847676277161 - rollout_corr/rollout_is_seq_std:0.0029975101351737976 - rollout_corr/rollout_is_seq_max:1.0126070976257324 - rollout_corr/rollout_is_seq_min:0.9907597303390503 - rollout_corr/rollout_is_seq_max_deviation:0.012607097625732422 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2693150429986417) - rollout_corr/training_log_ppl:np.float64(0.2345841708010994) - rollout_corr/kl:np.float64(0.0011821451473537081) - rollout_corr/k3_kl:np.float64(0.0013152157603144587) - rollout_corr/rollout_ppl:np.float64(1.267772528808564) - rollout_corr/rollout_log_ppl:np.float64(0.23340202278632205) - rollout_corr/log_ppl_diff:np.float64(0.00118214801477734) - rollout_corr/log_ppl_abs_diff:np.float64(0.002665369029273279) - rollout_corr/log_ppl_diff_max:np.float64(0.010928303003311157) - rollout_corr/log_ppl_diff_min:np.float64(-0.00843799114227295) - rollout_corr/ppl_ratio:np.float64(1.0011883925180882) - rollout_corr/chi2_token:np.float64(0.0032832033466547728) - rollout_corr/chi2_seq:np.float64(2.628566562430933) - actor/pg_loss:np.float64(0.00014865747652947903) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017586877143003221) - actor/ppo_kl:np.float64(2.5402906045357554e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5996150523424149) - perf/mfu/actor:np.float64(0.061039498769682454) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.6258544921875) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:3 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019761237781494856 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0019761237781494856 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:336.04296875 - response_length/max:1579.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:336.04296875 - response_length_non_aborted/max:1579.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.0625 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010008178651332855 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1094308514147997) - timing_s/agent_loop/generate_sequences/max:np.float64(9.863438200205564) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3283677981307846) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.863438200205564) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1579 - timing_s/gen:15.612013388425112 - timing_s/reward:0.05625380948185921 - timing_s/old_log_prob:2.4509079605340958 - timing_s/adv:0.1235277634114027 - timing_s/update_actor:9.913907954469323 - timing_s/step:28.248313438147306 - timing_s/stop_profile:3.726966679096222e-05 - timing_per_token_ms/gen:0.18147806372912123 - timing_per_token_ms/update_actor:0.0581382450136892 - timing_per_token_ms/adv:0.0007244052908487576 - perf/total_num_tokens:170523 - perf/time_per_step:28.248313438147306 - perf/throughput:754.5715975812958 (TaskRunner pid=2099163) Training Progress: 51%|█████ | 51/100 [43:52<43:56, 53.81s/it] (TaskRunner pid=2099163) step:52 - global_seqlen/min:12379 - global_seqlen/max:22705 - global_seqlen/minmax_diff:10326 - global_seqlen/balanced_min:16786 - global_seqlen/balanced_max:16849 - global_seqlen/mean:16796.75 - actor/entropy:0.22803975641727448 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.36542510986328125 - training/rollout_probs_diff_mean:0.0053499056957662106 - training/rollout_probs_diff_std:0.01391876582056284 - training/rollout_actor_probs_pearson_corr:0.9980417490005493 - rollout_corr/rollout_is_mean:1.0001552104949951 - rollout_corr/rollout_is_ratio_fraction_high:4.7668985644122586e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.355864752549678e-05 - rollout_corr/rollout_is_max:2.923143148422241 - rollout_corr/rollout_is_min:0.39950257539749146 - rollout_corr/rollout_is_std:0.04149097949266434 - rollout_corr/rollout_is_eff_sample_size:0.9982820511233786 - rollout_corr/rollout_is_seq_mean:1.0000829696655273 - rollout_corr/rollout_is_seq_std:0.003001436125487089 - rollout_corr/rollout_is_seq_max:1.011511206626892 - rollout_corr/rollout_is_seq_min:0.990175724029541 - rollout_corr/rollout_is_seq_max_deviation:0.01151120662689209 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2837047763168812) - rollout_corr/training_log_ppl:np.float64(0.24606362084159628) - rollout_corr/kl:np.float64(0.0014448560898543406) - rollout_corr/k3_kl:np.float64(0.0011887111170381104) - rollout_corr/rollout_ppl:np.float64(1.281900660134852) - rollout_corr/rollout_log_ppl:np.float64(0.24461876275017858) - rollout_corr/log_ppl_diff:np.float64(0.0014448580914177) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029126321896910667) - rollout_corr/log_ppl_diff_max:np.float64(0.013406842947006226) - rollout_corr/log_ppl_diff_min:np.float64(-0.008877873420715332) - rollout_corr/ppl_ratio:np.float64(1.001452179858461) - rollout_corr/chi2_token:np.float64(0.0019569701980799437) - rollout_corr/chi2_seq:np.float64(0.4788138067815453) - actor/pg_loss:np.float64(0.00014281971380114555) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0018395553652226226) - actor/ppo_kl:np.float64(0.0005606412955785345) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6134368255734444) - perf/mfu/actor:np.float64(0.047977674968366274) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.7822380065918) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:3 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01040375605225563 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01040375605225563 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:245.8359375 - response_length/max:1193.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:245.8359375 - response_length_non_aborted/max:1193.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.0625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.267452120780945e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9509514290839434) - timing_s/agent_loop/generate_sequences/max:np.float64(7.854211928322911) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.525260150054237) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.854211928322911) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1193 - timing_s/gen:13.733885359019041 - timing_s/reward:0.05357689596712589 - timing_s/old_log_prob:2.40861296094954 - timing_s/adv:0.1382129043340683 - timing_s/update_actor:10.0369862485677 - timing_s/step:26.460144393146038 - timing_s/stop_profile:2.820044755935669e-05 - timing_per_token_ms/gen:0.21822679885306895 - timing_per_token_ms/update_actor:0.07469440701748627 - timing_per_token_ms/adv:0.001028568802998112 - perf/total_num_tokens:134374 - perf/time_per_step:26.460144393146038 - perf/throughput:634.7943439171427 (TaskRunner pid=2099163) Training Progress: 52%|█████▏ | 52/100 [44:19<36:29, 45.61s/it] (TaskRunner pid=2099163) step:53 - global_seqlen/min:15429 - global_seqlen/max:26564 - global_seqlen/minmax_diff:11135 - global_seqlen/balanced_min:20620 - global_seqlen/balanced_max:20653 - global_seqlen/mean:20644.25 - actor/entropy:0.20042356848716736 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3535129427909851 - training/rollout_probs_diff_mean:0.0044761584140360355 - training/rollout_probs_diff_std:0.012780102901160717 - training/rollout_actor_probs_pearson_corr:0.9982099533081055 - rollout_corr/rollout_is_mean:0.9998370409011841 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00012490943481680006 - rollout_corr/rollout_is_max:1.8165900707244873 - rollout_corr/rollout_is_min:0.2722632586956024 - rollout_corr/rollout_is_std:0.03822029009461403 - rollout_corr/rollout_is_eff_sample_size:0.9985411024389872 - rollout_corr/rollout_is_seq_mean:0.9997756481170654 - rollout_corr/rollout_is_seq_std:0.0028406018391251564 - rollout_corr/rollout_is_seq_max:1.0108007192611694 - rollout_corr/rollout_is_seq_min:0.9895545840263367 - rollout_corr/rollout_is_seq_max_deviation:0.010800719261169434 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2743919994682074) - rollout_corr/training_log_ppl:np.float64(0.237915026533301) - rollout_corr/kl:np.float64(0.0014105199338541041) - rollout_corr/k3_kl:np.float64(0.0010296943997332164) - rollout_corr/rollout_ppl:np.float64(1.2725408389233053) - rollout_corr/rollout_log_ppl:np.float64(0.23650450706190895) - rollout_corr/log_ppl_diff:np.float64(0.0014105194713920355) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024846792221069336) - rollout_corr/log_ppl_diff_max:np.float64(0.014649003744125366) - rollout_corr/log_ppl_diff_min:np.float64(-0.0049605220556259155) - rollout_corr/ppl_ratio:np.float64(1.0014160515274853) - rollout_corr/chi2_token:np.float64(0.0013009211979806423) - rollout_corr/chi2_seq:np.float64(0.20528000919148326) - actor/pg_loss:np.float64(0.00033905566670000553) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011068140559018502) - actor/ppo_kl:np.float64(0.00029123531216690424) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.495539128780365) - perf/mfu/actor:np.float64(0.059122989206736365) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.70719528198242) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:3 - critic/score/mean:0.51953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01337155606597662 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01337155606597662 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:312.7265625 - response_length/max:1431.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:312.7265625 - response_length_non_aborted/max:1431.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:332.40625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.622953176498413e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7730677053332329) - timing_s/agent_loop/generate_sequences/max:np.float64(9.259236015379429) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0541376218679943) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.259236015379429) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1431 - timing_s/gen:14.91242471896112 - timing_s/reward:0.059695588424801826 - timing_s/old_log_prob:2.3497717920690775 - timing_s/adv:0.12111933156847954 - timing_s/update_actor:9.819533618167043 - timing_s/step:27.35314425267279 - timing_s/stop_profile:0.00012644007802009583 - timing_per_token_ms/gen:0.18627026304630542 - timing_per_token_ms/update_actor:0.05945683191546704 - timing_per_token_ms/adv:0.0007333720743577482 - perf/total_num_tokens:165154 - perf/time_per_step:27.35314425267279 - perf/throughput:754.7304181669266 (TaskRunner pid=2099163) Training Progress: 53%|█████▎ | 53/100 [44:46<31:26, 40.15s/it] (TaskRunner pid=2099163) step:54 - global_seqlen/min:13022 - global_seqlen/max:20901 - global_seqlen/minmax_diff:7879 - global_seqlen/balanced_min:17654 - global_seqlen/balanced_max:19215 - global_seqlen/mean:17867.875 - actor/entropy:0.21646106243133545 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5828086733818054 - training/rollout_probs_diff_mean:0.004788811318576336 - training/rollout_probs_diff_std:0.013526316732168198 - training/rollout_actor_probs_pearson_corr:0.9981729388237 - rollout_corr/rollout_is_mean:0.9998193979263306 - rollout_corr/rollout_is_ratio_fraction_high:1.4932727935956791e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010452909918967634 - rollout_corr/rollout_is_max:2.679741144180298 - rollout_corr/rollout_is_min:0.20122860372066498 - rollout_corr/rollout_is_std:0.040283203125 - rollout_corr/rollout_is_eff_sample_size:0.9983792390254678 - rollout_corr/rollout_is_seq_mean:0.9996543526649475 - rollout_corr/rollout_is_seq_std:0.0032862715888768435 - rollout_corr/rollout_is_seq_max:1.0084000825881958 - rollout_corr/rollout_is_seq_min:0.9903689622879028 - rollout_corr/rollout_is_seq_max_deviation:0.009631037712097168 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2988018337637186) - rollout_corr/training_log_ppl:np.float64(0.257318063187995) - rollout_corr/kl:np.float64(0.0014489969444486661) - rollout_corr/k3_kl:np.float64(0.001245539906221893) - rollout_corr/rollout_ppl:np.float64(1.2968687978573143) - rollout_corr/rollout_log_ppl:np.float64(0.25586906356329564) - rollout_corr/log_ppl_diff:np.float64(0.0014489996246993542) - rollout_corr/log_ppl_abs_diff:np.float64(0.002985531697049737) - rollout_corr/log_ppl_diff_max:np.float64(0.013407379388809204) - rollout_corr/log_ppl_diff_min:np.float64(-0.011749684810638428) - rollout_corr/ppl_ratio:np.float64(1.0014570830389857) - rollout_corr/chi2_token:np.float64(0.002103503793478012) - rollout_corr/chi2_seq:np.float64(0.6353278271853924) - actor/pg_loss:np.float64(0.0002622948959469795) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012713276441900234) - actor/ppo_kl:np.float64(0.00012942606369392706) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5507104620337486) - perf/mfu/actor:np.float64(0.05088630331246074) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.86227798461914) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:3 - critic/score/mean:0.48046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008711379952728748 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008711379952728748 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:261.58984375 - response_length/max:2899.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:261.58984375 - response_length_non_aborted/max:2899.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.78125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.422190487384796e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6486075595021248) - timing_s/agent_loop/generate_sequences/max:np.float64(15.47397425211966) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.486256126627268) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.47397425211966) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:2899 - timing_s/gen:21.19790812022984 - timing_s/reward:0.046014051884412766 - timing_s/old_log_prob:2.4986167792230844 - timing_s/adv:0.1188246849924326 - timing_s/update_actor:9.975375892594457 - timing_s/step:33.96630131267011 - timing_s/stop_profile:0.00012148916721343994 - timing_per_token_ms/gen:0.3165425974021509 - timing_per_token_ms/update_actor:0.06978569004844208 - timing_per_token_ms/adv:0.000831273199753976 - perf/total_num_tokens:142943 - perf/time_per_step:33.96630131267011 - perf/throughput:526.0471205127926 (TaskRunner pid=2099163) Training Progress: 54%|█████▍ | 54/100 [45:20<29:22, 38.31s/it] (TaskRunner pid=2099163) step:55 - global_seqlen/min:16785 - global_seqlen/max:28225 - global_seqlen/minmax_diff:11440 - global_seqlen/balanced_min:21265 - global_seqlen/balanced_max:22805 - global_seqlen/mean:21496.5 - actor/entropy:0.1851159632205963 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7663038969039917 - training/rollout_probs_diff_mean:0.004267675802111626 - training/rollout_probs_diff_std:0.012877827510237694 - training/rollout_actor_probs_pearson_corr:0.9981168508529663 - rollout_corr/rollout_is_mean:0.9998764991760254 - rollout_corr/rollout_is_ratio_fraction_high:5.7237051805714145e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.578964217216708e-05 - rollout_corr/rollout_is_max:2.459521770477295 - rollout_corr/rollout_is_min:0.02907703071832657 - rollout_corr/rollout_is_std:0.03838213533163071 - rollout_corr/rollout_is_eff_sample_size:0.9985286221149012 - rollout_corr/rollout_is_seq_mean:0.9999008178710938 - rollout_corr/rollout_is_seq_std:0.002929779700934887 - rollout_corr/rollout_is_seq_max:1.008970022201538 - rollout_corr/rollout_is_seq_min:0.9923213124275208 - rollout_corr/rollout_is_seq_max_deviation:0.008970022201538086 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2743566152639687) - rollout_corr/training_log_ppl:np.float64(0.23732106332317926) - rollout_corr/kl:np.float64(0.0011733787810150886) - rollout_corr/k3_kl:np.float64(0.0012601886370475768) - rollout_corr/rollout_ppl:np.float64(1.2727803294546902) - rollout_corr/rollout_log_ppl:np.float64(0.23614768433617428) - rollout_corr/log_ppl_diff:np.float64(0.0011733789870049804) - rollout_corr/log_ppl_abs_diff:np.float64(0.002733354893280193) - rollout_corr/log_ppl_diff_max:np.float64(0.014354616403579712) - rollout_corr/log_ppl_diff_min:np.float64(-0.008024752140045166) - rollout_corr/ppl_ratio:np.float64(1.0011803007218987) - rollout_corr/chi2_token:np.float64(0.0026173368096351624) - rollout_corr/chi2_seq:np.float64(0.7098474926315248) - actor/pg_loss:np.float64(0.00024044979363679886) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017808543852879666) - actor/ppo_kl:np.float64(0.00014058525295723712) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5874743461608887) - perf/mfu/actor:np.float64(0.061257645386096266) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.73693084716797) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:3 - critic/score/mean:0.51171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.010787753388285637 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.010787753388285637 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:341.234375 - response_length/max:3184.0 - response_length/min:83.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:341.234375 - response_length_non_aborted/max:3184.0 - response_length_non_aborted/min:83.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.53125 - prompt_length/max:498.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.620934724807739e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9831281434744596) - timing_s/agent_loop/generate_sequences/max:np.float64(17.57840107381344) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.278305432460911) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.57840107381344) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:432 - timing_s/agent_loop/slowest/response_length:3184 - timing_s/gen:23.38557505980134 - timing_s/reward:0.05253286845982075 - timing_s/old_log_prob:2.4493800587952137 - timing_s/adv:0.12223737873136997 - timing_s/update_actor:10.099678717553616 - timing_s/step:36.21919436752796 - timing_s/stop_profile:0.00011761859059333801 - timing_per_token_ms/gen:0.2677042797266512 - timing_per_token_ms/update_actor:0.05872862278483483 - timing_per_token_ms/adv:0.0007107981458107713 - perf/total_num_tokens:171972 - perf/time_per_step:36.21919436752796 - perf/throughput:593.5112686899662 (TaskRunner pid=2099163) Training Progress: 55%|█████▌ | 55/100 [45:56<28:16, 37.69s/it] (TaskRunner pid=2099163) step:56 - global_seqlen/min:11908 - global_seqlen/max:34677 - global_seqlen/minmax_diff:22769 - global_seqlen/balanced_min:19857 - global_seqlen/balanced_max:21973 - global_seqlen/mean:20175.375 - actor/entropy:0.19302956759929657 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5312970876693726 - training/rollout_probs_diff_mean:0.0045180413872003555 - training/rollout_probs_diff_std:0.01335164438933134 - training/rollout_actor_probs_pearson_corr:0.9980792999267578 - rollout_corr/rollout_is_mean:0.9997986555099487 - rollout_corr/rollout_is_ratio_fraction_high:1.2620685083675198e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.834479376673698e-05 - rollout_corr/rollout_is_max:2.0223238468170166 - rollout_corr/rollout_is_min:0.36936911940574646 - rollout_corr/rollout_is_std:0.03903120756149292 - rollout_corr/rollout_is_eff_sample_size:0.9984784662444588 - rollout_corr/rollout_is_seq_mean:0.9998159408569336 - rollout_corr/rollout_is_seq_std:0.0029684919863939285 - rollout_corr/rollout_is_seq_max:1.0116934776306152 - rollout_corr/rollout_is_seq_min:0.9913424253463745 - rollout_corr/rollout_is_seq_max_deviation:0.011693477630615234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2815037313848734) - rollout_corr/training_log_ppl:np.float64(0.24245586230244953) - rollout_corr/kl:np.float64(0.0015198735749812897) - rollout_corr/k3_kl:np.float64(0.0011902167615289727) - rollout_corr/rollout_ppl:np.float64(1.2794606960378587) - rollout_corr/rollout_log_ppl:np.float64(0.24093598501349334) - rollout_corr/log_ppl_diff:np.float64(0.0015198772889561951) - rollout_corr/log_ppl_abs_diff:np.float64(0.002925883134594187) - rollout_corr/log_ppl_diff_max:np.float64(0.015707731246948242) - rollout_corr/log_ppl_diff_min:np.float64(-0.008237093687057495) - rollout_corr/ppl_ratio:np.float64(1.0015279354993254) - rollout_corr/chi2_token:np.float64(0.001645366894081235) - rollout_corr/chi2_seq:np.float64(0.5515149484854192) - actor/pg_loss:np.float64(-7.30195315554738e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013563435145442782) - actor/ppo_kl:np.float64(0.0003750655004007841) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6355954259634018) - perf/mfu/actor:np.float64(0.057120220065657104) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.86117172241211) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:3 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0028096800670027733 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0028096800670027733 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:309.51171875 - response_length/max:3693.0 - response_length/min:83.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:309.51171875 - response_length_non_aborted/max:3693.0 - response_length_non_aborted/min:83.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:320.96875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010253302752971649 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9995839446783066) - timing_s/agent_loop/generate_sequences/max:np.float64(19.118556136265397) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.989160280507349) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.118556136265397) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:425 - timing_s/agent_loop/slowest/response_length:3693 - timing_s/gen:25.32910566404462 - timing_s/reward:0.05742966942489147 - timing_s/old_log_prob:2.481884676963091 - timing_s/adv:0.12075792998075485 - timing_s/update_actor:10.213837813585997 - timing_s/step:38.28866018541157 - timing_s/stop_profile:0.00011267699301242828 - timing_per_token_ms/gen:0.31967067159771084 - timing_per_token_ms/update_actor:0.06328158592830367 - timing_per_token_ms/adv:0.0007481764897849163 - perf/total_num_tokens:161403 - perf/time_per_step:38.28866018541157 - perf/throughput:526.9282054347531 (TaskRunner pid=2099163) Training Progress: 56%|█████▌ | 56/100 [46:35<27:47, 37.89s/it] (TaskRunner pid=2099163) step:57 - global_seqlen/min:18300 - global_seqlen/max:23742 - global_seqlen/minmax_diff:5442 - global_seqlen/balanced_min:20262 - global_seqlen/balanced_max:20589 - global_seqlen/mean:20356.625 - actor/entropy:0.1873641461133957 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9385040998458862 - training/rollout_probs_diff_mean:0.004480709787458181 - training/rollout_probs_diff_std:0.013861753977835178 - training/rollout_actor_probs_pearson_corr:0.9978734850883484 - rollout_corr/rollout_is_mean:1.0002038478851318 - rollout_corr/rollout_is_ratio_fraction_high:2.4369737730012275e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.529407932655886e-05 - rollout_corr/rollout_is_max:2.595475912094116 - rollout_corr/rollout_is_min:0.0481024831533432 - rollout_corr/rollout_is_std:0.03987497463822365 - rollout_corr/rollout_is_eff_sample_size:0.9984132236126418 - rollout_corr/rollout_is_seq_mean:1.000412940979004 - rollout_corr/rollout_is_seq_std:0.0028782689478248358 - rollout_corr/rollout_is_seq_max:1.0121479034423828 - rollout_corr/rollout_is_seq_min:0.992722749710083 - rollout_corr/rollout_is_seq_max_deviation:0.012147903442382812 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2728352225385606) - rollout_corr/training_log_ppl:np.float64(0.23661376311792992) - rollout_corr/kl:np.float64(0.0012007082295639737) - rollout_corr/k3_kl:np.float64(0.0011471591212455223) - rollout_corr/rollout_ppl:np.float64(1.2712574996985495) - rollout_corr/rollout_log_ppl:np.float64(0.23541305179242045) - rollout_corr/log_ppl_diff:np.float64(0.0012007113255094737) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025620612141210586) - rollout_corr/log_ppl_diff_max:np.float64(0.013712316751480103) - rollout_corr/log_ppl_diff_min:np.float64(-0.00932343304157257) - rollout_corr/ppl_ratio:np.float64(1.001206706976518) - rollout_corr/chi2_token:np.float64(0.002160461153835058) - rollout_corr/chi2_seq:np.float64(0.438829617574811) - actor/pg_loss:np.float64(0.00015424867160618305) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00141477917463817) - actor/ppo_kl:np.float64(0.0006155347940577371) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5178738236427307) - perf/mfu/actor:np.float64(0.058537288240303825) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.82574844360352) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:4 - critic/score/mean:0.44140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.013144426979124546 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.013144426979124546 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:320.58203125 - response_length/max:2049.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:320.58203125 - response_length_non_aborted/max:2049.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.5625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000209735706448555 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0480106547474861) - timing_s/agent_loop/generate_sequences/max:np.float64(12.178743667900562) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1250689009248163) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.178743667900562) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:2049 - timing_s/gen:18.099144043400884 - timing_s/reward:0.04921534284949303 - timing_s/old_log_prob:2.4653547890484333 - timing_s/adv:0.13351497240364552 - timing_s/update_actor:9.881367065012455 - timing_s/step:30.751196237280965 - timing_s/stop_profile:2.9180198907852173e-05 - timing_per_token_ms/gen:0.22053569610207122 - timing_per_token_ms/update_actor:0.06067660445317222 - timing_per_token_ms/adv:0.0008198496337411378 - perf/total_num_tokens:162853 - perf/time_per_step:30.751196237280965 - perf/throughput:661.9783127435156 (TaskRunner pid=2099163) Training Progress: 57%|█████▋ | 57/100 [47:06<25:43, 35.88s/it] (TaskRunner pid=2099163) step:58 - global_seqlen/min:13787 - global_seqlen/max:26334 - global_seqlen/minmax_diff:12547 - global_seqlen/balanced_min:19233 - global_seqlen/balanced_max:19250 - global_seqlen/mean:19242.625 - actor/entropy:0.21358489990234375 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4141245484352112 - training/rollout_probs_diff_mean:0.00496778916567564 - training/rollout_probs_diff_std:0.013776712119579315 - training/rollout_actor_probs_pearson_corr:0.9979900121688843 - rollout_corr/rollout_is_mean:1.0003639459609985 - rollout_corr/rollout_is_ratio_fraction_high:2.7048592528444715e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.762148404959589e-05 - rollout_corr/rollout_is_max:2.116508960723877 - rollout_corr/rollout_is_min:0.3293769061565399 - rollout_corr/rollout_is_std:0.04176446422934532 - rollout_corr/rollout_is_eff_sample_size:0.9982600735978991 - rollout_corr/rollout_is_seq_mean:1.000337839126587 - rollout_corr/rollout_is_seq_std:0.0029638041742146015 - rollout_corr/rollout_is_seq_max:1.0089969635009766 - rollout_corr/rollout_is_seq_min:0.9919787049293518 - rollout_corr/rollout_is_seq_max_deviation:0.008996963500976562 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2812516475096345) - rollout_corr/training_log_ppl:np.float64(0.2435778763901908) - rollout_corr/kl:np.float64(0.0010009155997465768) - rollout_corr/k3_kl:np.float64(0.0012113901442489805) - rollout_corr/rollout_ppl:np.float64(1.2799442182295024) - rollout_corr/rollout_log_ppl:np.float64(0.24257695997948758) - rollout_corr/log_ppl_diff:np.float64(0.001000916410703212) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027250831481069326) - rollout_corr/log_ppl_diff_max:np.float64(0.01267920434474945) - rollout_corr/log_ppl_diff_min:np.float64(-0.010362982749938965) - rollout_corr/ppl_ratio:np.float64(1.0010073960293084) - rollout_corr/chi2_token:np.float64(0.002911286661401391) - rollout_corr/chi2_seq:np.float64(0.7661588522605598) - actor/pg_loss:np.float64(0.0003306339494884014) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00199477257979197) - actor/ppo_kl:np.float64(0.0003630212834728752) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5990487411618233) - perf/mfu/actor:np.float64(0.05602923257711591) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.89770126342773) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:4 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001112373429350555 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001112373429350555 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:288.83203125 - response_length/max:1110.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:288.83203125 - response_length_non_aborted/max:1110.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.475431680679321e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0492791645228863) - timing_s/agent_loop/generate_sequences/max:np.float64(7.552903410047293) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9448991375684272) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.552903410047293) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1110 - timing_s/gen:13.404031103476882 - timing_s/reward:0.057840101420879364 - timing_s/old_log_prob:2.417235178872943 - timing_s/adv:0.11892589367926121 - timing_s/update_actor:9.727152958512306 - timing_s/step:25.811423553153872 - timing_s/stop_profile:0.00012199766933917999 - timing_per_token_ms/gen:0.18128008957786454 - timing_per_token_ms/update_actor:0.06318753911246715 - timing_per_token_ms/adv:0.0007725420367495418 - perf/total_num_tokens:153941 - perf/time_per_step:25.811423553153872 - perf/throughput:745.5080871604528 (TaskRunner pid=2099163) Training Progress: 58%|█████▊ | 58/100 [47:32<23:00, 32.88s/it] (TaskRunner pid=2099163) step:59 - global_seqlen/min:14785 - global_seqlen/max:28128 - global_seqlen/minmax_diff:13343 - global_seqlen/balanced_min:20039 - global_seqlen/balanced_max:23101 - global_seqlen/mean:20570.0 - actor/entropy:0.19363823533058167 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3583551049232483 - training/rollout_probs_diff_mean:0.004458777606487274 - training/rollout_probs_diff_std:0.012819879688322544 - training/rollout_actor_probs_pearson_corr:0.9981608390808105 - rollout_corr/rollout_is_mean:1.0000730752944946 - rollout_corr/rollout_is_ratio_fraction_high:4.982560858479701e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.719482138985768e-05 - rollout_corr/rollout_is_max:2.615692138671875 - rollout_corr/rollout_is_min:0.29617154598236084 - rollout_corr/rollout_is_std:0.038851357996463776 - rollout_corr/rollout_is_eff_sample_size:0.9984930846153224 - rollout_corr/rollout_is_seq_mean:0.9999926686286926 - rollout_corr/rollout_is_seq_std:0.0030627932865172625 - rollout_corr/rollout_is_seq_max:1.009785532951355 - rollout_corr/rollout_is_seq_min:0.9908705949783325 - rollout_corr/rollout_is_seq_max_deviation:0.00978553295135498 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2848408012650907) - rollout_corr/training_log_ppl:np.float64(0.24571586469028261) - rollout_corr/kl:np.float64(0.001712628750690115) - rollout_corr/k3_kl:np.float64(0.0012789307371718905) - rollout_corr/rollout_ppl:np.float64(1.2825767188332975) - rollout_corr/rollout_log_ppl:np.float64(0.24400323344525532) - rollout_corr/log_ppl_diff:np.float64(0.0017126312450272962) - rollout_corr/log_ppl_abs_diff:np.float64(0.002915053817559965) - rollout_corr/log_ppl_diff_max:np.float64(0.011791497468948364) - rollout_corr/log_ppl_diff_min:np.float64(-0.008301064372062683) - rollout_corr/ppl_ratio:np.float64(1.0017199639696628) - rollout_corr/chi2_token:np.float64(0.0019151861779391766) - rollout_corr/chi2_seq:np.float64(0.34929756191559136) - actor/pg_loss:np.float64(2.256268635392189e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014333776648527419) - actor/ppo_kl:np.float64(0.0007561920642880082) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5470746606588364) - perf/mfu/actor:np.float64(0.05836452796553154) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.88081741333008) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:4 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03495578095316887 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.03495578095316887 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:313.59375 - response_length/max:4644.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:313.59375 - response_length_non_aborted/max:4644.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:329.21875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001398380845785141 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8809522464871407) - timing_s/agent_loop/generate_sequences/max:np.float64(23.754080295562744) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0718314567784546) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.754080295562744) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:4644 - timing_s/gen:29.604787712916732 - timing_s/reward:0.04805128648877144 - timing_s/old_log_prob:2.5187726486474276 - timing_s/adv:0.11889026686549187 - timing_s/update_actor:10.145638149231672 - timing_s/step:42.470872562378645 - timing_s/stop_profile:0.00012224912643432617 - timing_per_token_ms/gen:0.3687691543711601 - timing_per_token_ms/update_actor:0.06165312438764993 - timing_per_token_ms/adv:0.0007224736683610347 - perf/total_num_tokens:164560 - perf/time_per_step:42.470872562378645 - perf/throughput:484.33193760707485 (TaskRunner pid=2099163) Training Progress: 59%|█████▉ | 59/100 [48:14<24:26, 35.77s/it] (TaskRunner pid=2099163) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2099163) validation generation end (TaskRunner pid=2099163) [prompt] system (TaskRunner pid=2099163) (TaskRunner pid=2099163) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) user (TaskRunner pid=2099163) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2099163) (TaskRunner pid=2099163) A: -0.01 (TaskRunner pid=2099163) B: 1.69 (TaskRunner pid=2099163) C: 2.49 (TaskRunner pid=2099163) D: 0.45 (TaskRunner pid=2099163) Please reason step by step. (TaskRunner pid=2099163) assistant (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) [response] (TaskRunner pid=2099163) The folding stability score for a protein sequence is typically determined using computational methods such as those implemented in tools like DeepFold, Rosetta, or other protein structure prediction software. The provided sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR' is a specific amino acid sequence, and without access to specific computational tools or databases that calculate folding stability scores, it is not possible to determine the exact folding stability score for this sequence. Therefore, the score cannot be determined from the given information. (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) D (TaskRunner pid=2099163) (TaskRunner pid=2099163) [ground_truth] B (TaskRunner pid=2099163) [score] 0.0 (TaskRunner pid=2099163) [acc] 0.0 (TaskRunner pid=2099163) [pred] D (TaskRunner pid=2099163) [incorrect_format] 1 (TaskRunner pid=2099163) [feedback] (TaskRunner pid=2099163) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (WorkerDict pid=2099539) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50/actor (TaskRunner pid=2099163) step:60 - global_seqlen/min:11482 - global_seqlen/max:23421 - global_seqlen/minmax_diff:11939 - global_seqlen/balanced_min:16763 - global_seqlen/balanced_max:16948 - global_seqlen/mean:16792.25 - actor/entropy:0.24822477996349335 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30099523067474365 - training/rollout_probs_diff_mean:0.005300878547132015 - training/rollout_probs_diff_std:0.01358079444617033 - training/rollout_actor_probs_pearson_corr:0.9982614517211914 - rollout_corr/rollout_is_mean:1.000307321548462 - rollout_corr/rollout_is_ratio_fraction_high:3.160056803608313e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.160056803608313e-05 - rollout_corr/rollout_is_max:2.7097694873809814 - rollout_corr/rollout_is_min:0.2010701298713684 - rollout_corr/rollout_is_std:0.041654426604509354 - rollout_corr/rollout_is_eff_sample_size:0.9982688644919676 - rollout_corr/rollout_is_seq_mean:1.000412106513977 - rollout_corr/rollout_is_seq_std:0.003237298224121332 - rollout_corr/rollout_is_seq_max:1.0087321996688843 - rollout_corr/rollout_is_seq_min:0.9887025952339172 - rollout_corr/rollout_is_seq_max_deviation:0.011297404766082764 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3292968641035259) - rollout_corr/training_log_ppl:np.float64(0.2800096425926313) - rollout_corr/kl:np.float64(0.0010625352792885678) - rollout_corr/k3_kl:np.float64(0.0012574967236105294) - rollout_corr/rollout_ppl:np.float64(1.3278319356031716) - rollout_corr/rollout_log_ppl:np.float64(0.2789471071882872) - rollout_corr/log_ppl_diff:np.float64(0.0010625354043440893) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028606804116861895) - rollout_corr/log_ppl_diff_max:np.float64(0.01503533124923706) - rollout_corr/log_ppl_diff_min:np.float64(-0.010933548212051392) - rollout_corr/ppl_ratio:np.float64(1.0010700710117817) - rollout_corr/chi2_token:np.float64(0.0029621769208461046) - rollout_corr/chi2_seq:np.float64(1.0398543775081635) - actor/pg_loss:np.float64(0.0007010122062638402) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012734923266179976) - actor/ppo_kl:np.float64(0.0005196989214417691) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6372415572404861) - perf/mfu/actor:np.float64(0.049047718363570796) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.9422492980957) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.3825) - val-aux/sciknoweval/reward/std@16:np.float64(0.2596834756971686) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.49317999999999995) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.2221376792011573) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.27333999999999997) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.22152763913536153) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.38454) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2597535466020497) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.5833200000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.15799078019111001) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.18596000000000001) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.15643648114578526) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.39120000000000005) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.21626416144931568) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.6409199999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09541131973399612) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.12792) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09909837524937166) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.39025999999999994) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.16916725228527277) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.67412) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.05028767227181808) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.08998) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.058669001814606235) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.38689999999999997) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.13557260155312797) - val-aux/sciknoweval/score/mean@16:np.float64(0.3825) - val-aux/sciknoweval/score/std@16:np.float64(0.2596834756971686) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.49317999999999995) - val-aux/sciknoweval/score/best@2/std:np.float64(0.2221376792011573) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.27333999999999997) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.22152763913536153) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.38454) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2597535466020497) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.5833200000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.15799078019111001) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.18596000000000001) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.15643648114578526) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.39120000000000005) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.21626416144931568) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.6409199999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09541131973399612) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.12792) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09909837524937166) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.39025999999999994) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.16916725228527277) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.67412) - val-aux/sciknoweval/score/best@16/std:np.float64(0.05028767227181808) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.08998) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.058669001814606235) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.38689999999999997) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.13557260155312797) - val-core/sciknoweval/acc/mean@16:np.float64(0.3825) - val-aux/sciknoweval/acc/std@16:np.float64(0.2596834756971686) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.49317999999999995) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.2221376792011573) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.27333999999999997) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.22152763913536153) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.38454) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2597535466020497) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.5833200000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.15799078019111001) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.18596000000000001) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.15643648114578526) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.39120000000000005) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.21626416144931568) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.6409199999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09541131973399612) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.12792) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09909837524937166) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.39025999999999994) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.16916725228527277) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.67412) - val-core/sciknoweval/acc/best@16/std:np.float64(0.05028767227181808) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.08998) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.058669001814606235) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.38689999999999997) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.13557260155312797) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.02113806582593929) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99956) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.00561792069220043) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9884999999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0273368559769371) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.99374) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.020753176673987798) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.97858) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.03448809285937782) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9984999999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.010504955437387428) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.96292) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.038690345327705815) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9998600000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0028808653454051197) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9443) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03543438166339959) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:4 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01805972494184971 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01805972494184971 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:247.2265625 - response_length/max:1516.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:247.2265625 - response_length_non_aborted/max:1516.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013054721057415009 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6093897819519043) - timing_s/agent_loop/generate_sequences/max:np.float64(8.942918440327048) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.4281441207785974) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.942918440327048) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:424 - timing_s/agent_loop/slowest/response_length:1516 - timing_s/gen:14.714311951771379 - timing_s/reward:0.05381259322166443 - timing_s/old_log_prob:2.322748025879264 - timing_s/adv:0.12639613077044487 - timing_s/update_actor:9.668666616082191 - timing_s/step:26.97286426089704 - timing_s/testing:71.51457652077079 - timing_s/save_checkpoint:6.8972090892493725 - timing_s/stop_profile:0.00016043893992900848 - timing_per_token_ms/gen:0.23249031366363374 - timing_per_token_ms/update_actor:0.07197268543585725 - timing_per_token_ms/adv:0.0009408814391344584 - perf/total_num_tokens:134338 - perf/time_per_step:26.97286426089704 - perf/throughput:622.5608759075681 (TaskRunner pid=2099163) Training Progress: 60%|██████ | 60/100 [50:00<37:46, 56.67s/it] (TaskRunner pid=2099163) step:61 - global_seqlen/min:11756 - global_seqlen/max:28743 - global_seqlen/minmax_diff:16987 - global_seqlen/balanced_min:17559 - global_seqlen/balanced_max:20581 - global_seqlen/mean:18308.5 - actor/entropy:0.22188033163547516 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7668329477310181 - training/rollout_probs_diff_mean:0.004735196474939585 - training/rollout_probs_diff_std:0.01345571968704462 - training/rollout_actor_probs_pearson_corr:0.9982025623321533 - rollout_corr/rollout_is_mean:1.000156283378601 - rollout_corr/rollout_is_ratio_fraction_high:2.8573062081704848e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.285959221306257e-05 - rollout_corr/rollout_is_max:10.01133918762207 - rollout_corr/rollout_is_min:0.23175029456615448 - rollout_corr/rollout_is_std:0.03982262313365936 - rollout_corr/rollout_is_eff_sample_size:0.9984169073956883 - rollout_corr/rollout_is_seq_mean:1.000264286994934 - rollout_corr/rollout_is_seq_std:0.0033271322026848793 - rollout_corr/rollout_is_seq_max:1.0181152820587158 - rollout_corr/rollout_is_seq_min:0.9914125800132751 - rollout_corr/rollout_is_seq_max_deviation:0.01811528205871582 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3211746821179986) - rollout_corr/training_log_ppl:np.float64(0.2735010301985312) - rollout_corr/kl:np.float64(0.0008562939778675727) - rollout_corr/k3_kl:np.float64(0.0012789235939010268) - rollout_corr/rollout_ppl:np.float64(1.3200623537413776) - rollout_corr/rollout_log_ppl:np.float64(0.27264473447576165) - rollout_corr/log_ppl_diff:np.float64(0.0008562957227695733) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028399873117450625) - rollout_corr/log_ppl_diff_max:np.float64(0.014549434185028076) - rollout_corr/log_ppl_diff_min:np.float64(-0.008745625615119934) - rollout_corr/ppl_ratio:np.float64(1.0008633281104267) - rollout_corr/chi2_token:np.float64(0.004113542148843408) - rollout_corr/chi2_seq:np.float64(0.8988656422588974) - actor/pg_loss:np.float64(0.0002667094813659787) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015468199947576977) - actor/ppo_kl:np.float64(9.708452350309926e-05) - actor/pg_clipfrac_lower:np.float64(1.4575559362128843e-05) - actor/grad_norm:np.float64(0.5922280102968216) - perf/mfu/actor:np.float64(0.050955802459431765) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(105.23041534423828) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:4 - critic/score/mean:0.28125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.28125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.020483313128352165 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.020483313128352165 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:273.421875 - response_length/max:4322.0 - response_length/min:80.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:273.421875 - response_length_non_aborted/max:4322.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.061714470386505e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9486130196601152) - timing_s/agent_loop/generate_sequences/max:np.float64(22.567378267645836) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.5980923264069133) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.567378267645836) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:4322 - timing_s/gen:28.375739235430956 - timing_s/reward:0.053623124957084656 - timing_s/old_log_prob:2.4645230006426573 - timing_s/adv:0.11367073655128479 - timing_s/update_actor:10.312344249337912 - timing_s/step:41.40952676907182 - timing_s/stop_profile:0.00012328848242759705 - timing_per_token_ms/gen:0.40539086855578826 - timing_per_token_ms/update_actor:0.07040680728444378 - timing_per_token_ms/adv:0.0007760789834727367 - perf/total_num_tokens:146468 - perf/time_per_step:41.40952676907182 - perf/throughput:442.13255809709847 (TaskRunner pid=2099163) Training Progress: 61%|██████ | 61/100 [50:41<33:52, 52.11s/it] (TaskRunner pid=2099163) step:62 - global_seqlen/min:11720 - global_seqlen/max:23390 - global_seqlen/minmax_diff:11670 - global_seqlen/balanced_min:17845 - global_seqlen/balanced_max:18228 - global_seqlen/mean:17903.375 - actor/entropy:0.23856085538864136 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7297425270080566 - training/rollout_probs_diff_mean:0.004906587768346071 - training/rollout_probs_diff_std:0.013448494486510754 - training/rollout_actor_probs_pearson_corr:0.9983062148094177 - rollout_corr/rollout_is_mean:0.9999780654907227 - rollout_corr/rollout_is_ratio_fraction_high:1.470869392505847e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.825216355035082e-05 - rollout_corr/rollout_is_max:2.1296794414520264 - rollout_corr/rollout_is_min:0.06101953238248825 - rollout_corr/rollout_is_std:0.0396200455725193 - rollout_corr/rollout_is_eff_sample_size:0.9984327123217758 - rollout_corr/rollout_is_seq_mean:1.0001113414764404 - rollout_corr/rollout_is_seq_std:0.0030740199144929647 - rollout_corr/rollout_is_seq_max:1.014378547668457 - rollout_corr/rollout_is_seq_min:0.9906818866729736 - rollout_corr/rollout_is_seq_max_deviation:0.014378547668457031 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3296263343654573) - rollout_corr/training_log_ppl:np.float64(0.28007101351977326) - rollout_corr/kl:np.float64(0.0011143518837499755) - rollout_corr/k3_kl:np.float64(0.0012758465617253023) - rollout_corr/rollout_ppl:np.float64(1.3281300985254347) - rollout_corr/rollout_log_ppl:np.float64(0.27895666120457463) - rollout_corr/log_ppl_diff:np.float64(0.00111435231519863) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026538760284893215) - rollout_corr/log_ppl_diff_max:np.float64(0.03442579507827759) - rollout_corr/log_ppl_diff_min:np.float64(-0.012630432844161987) - rollout_corr/ppl_ratio:np.float64(1.0011220904998481) - rollout_corr/chi2_token:np.float64(0.0027193170972168446) - rollout_corr/chi2_seq:np.float64(0.22215013159438968) - actor/pg_loss:np.float64(0.0003498832229524851) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012161810750512814) - actor/ppo_kl:np.float64(0.0002681101446029288) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5286872908473015) - perf/mfu/actor:np.float64(0.051028239589326306) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.716552734375) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:4 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004962345585227013 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004962345585227013 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:265.57421875 - response_length/max:1654.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:265.57421875 - response_length_non_aborted/max:1654.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.90625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.049180567264557e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.962077796459198) - timing_s/agent_loop/generate_sequences/max:np.float64(9.180661825463176) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.6732208340472425) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.180661825463176) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:436 - timing_s/agent_loop/slowest/response_length:1654 - timing_s/gen:14.704399591311812 - timing_s/reward:0.054863808676600456 - timing_s/old_log_prob:2.536904999986291 - timing_s/adv:0.11875544860959053 - timing_s/update_actor:9.82179962284863 - timing_s/step:27.321818605065346 - timing_s/stop_profile:0.0001788027584552765 - timing_per_token_ms/gen:0.21628251858902162 - timing_per_token_ms/update_actor:0.06857505653856208 - timing_per_token_ms/adv:0.0008291414929419072 - perf/total_num_tokens:143227 - perf/time_per_step:27.321818605065346 - perf/throughput:655.2775735316826 (TaskRunner pid=2099163) Training Progress: 62%|██████▏ | 62/100 [51:09<28:18, 44.69s/it] (TaskRunner pid=2099163) step:63 - global_seqlen/min:15327 - global_seqlen/max:23927 - global_seqlen/minmax_diff:8600 - global_seqlen/balanced_min:19595 - global_seqlen/balanced_max:26173 - global_seqlen/mean:20418.0 - actor/entropy:0.23164616525173187 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.777930498123169 - training/rollout_probs_diff_mean:0.004647471942007542 - training/rollout_probs_diff_std:0.01323886401951313 - training/rollout_actor_probs_pearson_corr:0.998352587223053 - rollout_corr/rollout_is_mean:0.9999091625213623 - rollout_corr/rollout_is_ratio_fraction_high:2.5786488549783826e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012893244274891913 - rollout_corr/rollout_is_max:3.2854862213134766 - rollout_corr/rollout_is_min:0.04563618823885918 - rollout_corr/rollout_is_std:0.03892645984888077 - rollout_corr/rollout_is_eff_sample_size:0.9984866667412581 - rollout_corr/rollout_is_seq_mean:0.9998888373374939 - rollout_corr/rollout_is_seq_std:0.0031720264814794064 - rollout_corr/rollout_is_seq_max:1.0129690170288086 - rollout_corr/rollout_is_seq_min:0.9918916821479797 - rollout_corr/rollout_is_seq_max_deviation:0.012969017028808594 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3354551973752677) - rollout_corr/training_log_ppl:np.float64(0.2843584521106095) - rollout_corr/kl:np.float64(0.0012951181959266478) - rollout_corr/k3_kl:np.float64(0.0012268340739751693) - rollout_corr/rollout_ppl:np.float64(1.3336740545928478) - rollout_corr/rollout_log_ppl:np.float64(0.283063332484744) - rollout_corr/log_ppl_diff:np.float64(0.0012951196258654818) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027128312940476462) - rollout_corr/log_ppl_diff_max:np.float64(0.016135364770889282) - rollout_corr/log_ppl_diff_min:np.float64(-0.010828465223312378) - rollout_corr/ppl_ratio:np.float64(1.0013016369193792) - rollout_corr/chi2_token:np.float64(0.0022587431594729424) - rollout_corr/chi2_seq:np.float64(1.104415416251868) - actor/pg_loss:np.float64(-3.0042603611946106e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0019501109186421672) - actor/ppo_kl:np.float64(0.00024567433211086787) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.7029889523983002) - perf/mfu/actor:np.float64(0.0549835847859291) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.80621337890625) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:4 - critic/score/mean:0.39453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.39453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.020585674792528152 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.020585674792528152 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:302.96875 - response_length/max:7755.0 - response_length/min:91.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:302.96875 - response_length_non_aborted/max:7755.0 - response_length_non_aborted/min:91.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:335.09375 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001079123467206955 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0766320154070854) - timing_s/agent_loop/generate_sequences/max:np.float64(38.67040351592004) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.959399871499045) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(38.67040351592004) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:7755 - timing_s/gen:44.32738490961492 - timing_s/reward:0.05780384503304958 - timing_s/old_log_prob:2.531794674694538 - timing_s/adv:0.12005887180566788 - timing_s/update_actor:10.524201342836022 - timing_s/step:57.64935720153153 - timing_s/stop_profile:0.00011799484491348267 - timing_per_token_ms/gen:0.5715237868697127 - timing_per_token_ms/update_actor:0.06442967812001679 - timing_per_token_ms/adv:0.000735006316765035 - perf/total_num_tokens:163344 - perf/time_per_step:57.64935720153153 - perf/throughput:354.17567499707644 (TaskRunner pid=2099163) Training Progress: 63%|██████▎ | 63/100 [52:06<29:57, 48.59s/it] (TaskRunner pid=2099163) step:64 - global_seqlen/min:16789 - global_seqlen/max:29163 - global_seqlen/minmax_diff:12374 - global_seqlen/balanced_min:21715 - global_seqlen/balanced_max:21735 - global_seqlen/mean:21731.25 - actor/entropy:0.2434695065021515 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3808082342147827 - training/rollout_probs_diff_mean:0.004839340224862099 - training/rollout_probs_diff_std:0.012994172982871532 - training/rollout_actor_probs_pearson_corr:0.998428463935852 - rollout_corr/rollout_is_mean:1.0002532005310059 - rollout_corr/rollout_is_ratio_fraction_high:3.917164940503426e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.528608355438337e-05 - rollout_corr/rollout_is_max:3.4824414253234863 - rollout_corr/rollout_is_min:0.37827831506729126 - rollout_corr/rollout_is_std:0.03948592394590378 - rollout_corr/rollout_is_eff_sample_size:0.9984440018610561 - rollout_corr/rollout_is_seq_mean:1.000284194946289 - rollout_corr/rollout_is_seq_std:0.002908356487751007 - rollout_corr/rollout_is_seq_max:1.011995553970337 - rollout_corr/rollout_is_seq_min:0.9921226501464844 - rollout_corr/rollout_is_seq_max_deviation:0.011995553970336914 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3239814243279397) - rollout_corr/training_log_ppl:np.float64(0.27563152175571304) - rollout_corr/kl:np.float64(0.0009577943357328422) - rollout_corr/k3_kl:np.float64(0.001276045324914321) - rollout_corr/rollout_ppl:np.float64(1.3226902424357831) - rollout_corr/rollout_log_ppl:np.float64(0.27467372607497964) - rollout_corr/log_ppl_diff:np.float64(0.0009577956807333976) - rollout_corr/log_ppl_abs_diff:np.float64(0.002473829168593511) - rollout_corr/log_ppl_diff_max:np.float64(0.010276317596435547) - rollout_corr/log_ppl_diff_min:np.float64(-0.009252876043319702) - rollout_corr/ppl_ratio:np.float64(1.0009632629808038) - rollout_corr/chi2_token:np.float64(0.0031634781043976545) - rollout_corr/chi2_seq:np.float64(1.2617831665556878) - actor/pg_loss:np.float64(0.0002826264826580882) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002043358704668208) - actor/ppo_kl:np.float64(0.000321553345604908) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6392286270856857) - perf/mfu/actor:np.float64(0.06297104741900578) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.78218460083008) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:4 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009126994758844376 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009126994758844376 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:299.1640625 - response_length/max:1171.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:299.1640625 - response_length_non_aborted/max:1171.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:379.9375 - prompt_length/max:499.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013485923409461975 - timing_s/agent_loop/generate_sequences/min:np.float64(1.014983855187893) - timing_s/agent_loop/generate_sequences/max:np.float64(7.733801892027259) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1143013205582974) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.733801892027259) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1171 - timing_s/gen:13.659647185355425 - timing_s/reward:0.06105978973209858 - timing_s/old_log_prob:2.402337344363332 - timing_s/adv:0.11804847605526447 - timing_s/update_actor:9.979404045268893 - timing_s/step:26.307940509170294 - timing_s/stop_profile:0.00010655634105205536 - timing_per_token_ms/gen:0.17835697366823472 - timing_per_token_ms/update_actor:0.0574023816236347 - timing_per_token_ms/adv:0.000679024883838162 - perf/total_num_tokens:173850 - perf/time_per_step:26.307940509170294 - perf/throughput:826.0338734012655 (TaskRunner pid=2099163) Training Progress: 64%|██████▍ | 64/100 [52:33<25:09, 41.92s/it] (TaskRunner pid=2099163) step:65 - global_seqlen/min:14100 - global_seqlen/max:21693 - global_seqlen/minmax_diff:7593 - global_seqlen/balanced_min:18213 - global_seqlen/balanced_max:18230 - global_seqlen/mean:18224.5 - actor/entropy:0.25228142738342285 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.299439013004303 - training/rollout_probs_diff_mean:0.004961303435266018 - training/rollout_probs_diff_std:0.012695282697677612 - training/rollout_actor_probs_pearson_corr:0.9985328316688538 - rollout_corr/rollout_is_mean:1.0000190734863281 - rollout_corr/rollout_is_ratio_fraction_high:1.530081317469012e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.060162634938024e-05 - rollout_corr/rollout_is_max:2.15134859085083 - rollout_corr/rollout_is_min:0.4842594861984253 - rollout_corr/rollout_is_std:0.03996157646179199 - rollout_corr/rollout_is_eff_sample_size:0.9984056184691176 - rollout_corr/rollout_is_seq_mean:1.0000884532928467 - rollout_corr/rollout_is_seq_std:0.003020690055564046 - rollout_corr/rollout_is_seq_max:1.0103381872177124 - rollout_corr/rollout_is_seq_min:0.99142986536026 - rollout_corr/rollout_is_seq_max_deviation:0.010338187217712402 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.337971543893218) - rollout_corr/training_log_ppl:np.float64(0.285963032671134) - rollout_corr/kl:np.float64(0.0009719172556970079) - rollout_corr/k3_kl:np.float64(0.0011187582267666585) - rollout_corr/rollout_ppl:np.float64(1.3366436455398798) - rollout_corr/rollout_log_ppl:np.float64(0.28499111408018507) - rollout_corr/log_ppl_diff:np.float64(0.000971918590948917) - rollout_corr/log_ppl_abs_diff:np.float64(0.002662504601175897) - rollout_corr/log_ppl_diff_max:np.float64(0.010240867733955383) - rollout_corr/log_ppl_diff_min:np.float64(-0.011126220226287842) - rollout_corr/ppl_ratio:np.float64(1.0009777853265405) - rollout_corr/chi2_token:np.float64(0.0026106988079845905) - rollout_corr/chi2_seq:np.float64(0.5336100137792528) - actor/pg_loss:np.float64(-0.00011365010868757963) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015210017868412251) - actor/ppo_kl:np.float64(0.0001487335989569516) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5744654163718224) - perf/mfu/actor:np.float64(0.05192652203466894) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.78650283813477) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:4 - critic/score/mean:0.4375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003016172908246517 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003016172908246517 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:255.296875 - response_length/max:1081.0 - response_length/min:83.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:255.296875 - response_length_non_aborted/max:1081.0 - response_length_non_aborted/min:83.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.21875 - prompt_length/max:498.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000135716050863266 - timing_s/agent_loop/generate_sequences/min:np.float64(0.946566354483366) - timing_s/agent_loop/generate_sequences/max:np.float64(7.011935571208596) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.608640009377268) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.011935571208596) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1081 - timing_s/gen:12.771047074347734 - timing_s/reward:0.055413682013750076 - timing_s/old_log_prob:2.3575429376214743 - timing_s/adv:0.11941352300345898 - timing_s/update_actor:9.974061073735356 - timing_s/step:25.366581419482827 - timing_s/stop_profile:0.00011139549314975739 - timing_per_token_ms/gen:0.19540741591204686 - timing_per_token_ms/update_actor:0.06841107488364123 - timing_per_token_ms/adv:0.0008190452618964785 - perf/total_num_tokens:145796 - perf/time_per_step:25.366581419482827 - perf/throughput:718.4452527766574 (TaskRunner pid=2099163) Training Progress: 65%|██████▌ | 65/100 [52:58<21:34, 36.98s/it] (TaskRunner pid=2099163) step:66 - global_seqlen/min:13080 - global_seqlen/max:22615 - global_seqlen/minmax_diff:9535 - global_seqlen/balanced_min:17453 - global_seqlen/balanced_max:17514 - global_seqlen/mean:17466.5 - actor/entropy:0.27698102593421936 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2931666672229767 - training/rollout_probs_diff_mean:0.005428791046142578 - training/rollout_probs_diff_std:0.013127138838171959 - training/rollout_actor_probs_pearson_corr:0.998500406742096 - rollout_corr/rollout_is_mean:0.9999055862426758 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.560062446515076e-05 - rollout_corr/rollout_is_max:1.9062473773956299 - rollout_corr/rollout_is_min:0.4490605890750885 - rollout_corr/rollout_is_std:0.04169447347521782 - rollout_corr/rollout_is_eff_sample_size:0.9982642314339047 - rollout_corr/rollout_is_seq_mean:0.999854326248169 - rollout_corr/rollout_is_seq_std:0.003016769653186202 - rollout_corr/rollout_is_seq_max:1.0081275701522827 - rollout_corr/rollout_is_seq_min:0.9917121529579163 - rollout_corr/rollout_is_seq_max_deviation:0.00828784704208374 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.361108566634357) - rollout_corr/training_log_ppl:np.float64(0.3038711922999937) - rollout_corr/kl:np.float64(0.0010244684705327245) - rollout_corr/k3_kl:np.float64(0.001206662257914104) - rollout_corr/rollout_ppl:np.float64(1.3596958024427295) - rollout_corr/rollout_log_ppl:np.float64(0.3028467216063291) - rollout_corr/log_ppl_diff:np.float64(0.0010244706936646253) - rollout_corr/log_ppl_abs_diff:np.float64(0.002828612778102979) - rollout_corr/log_ppl_diff_max:np.float64(0.010544925928115845) - rollout_corr/log_ppl_diff_min:np.float64(-0.013002634048461914) - rollout_corr/ppl_ratio:np.float64(1.001031328458339) - rollout_corr/chi2_token:np.float64(0.002860499545931816) - rollout_corr/chi2_seq:np.float64(0.9732876946218312) - actor/pg_loss:np.float64(2.8914306312799454e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017755173166733584) - actor/ppo_kl:np.float64(-8.335111564150566e-05) - actor/pg_clipfrac_lower:np.float64(1.0643732821336016e-05) - actor/grad_norm:np.float64(0.679240420460701) - perf/mfu/actor:np.float64(0.05005040663095298) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.89555358886719) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:4 - critic/score/mean:0.4296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004099064040929079 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004099064040929079 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:250.390625 - response_length/max:1240.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:250.390625 - response_length_non_aborted/max:1240.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.4375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001604091376066208 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0681240409612656) - timing_s/agent_loop/generate_sequences/max:np.float64(7.348783219233155) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.5614106698412797) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.348783219233155) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:435 - timing_s/agent_loop/slowest/response_length:1240 - timing_s/gen:13.083483047783375 - timing_s/reward:0.05359955504536629 - timing_s/old_log_prob:2.406119966879487 - timing_s/adv:0.12794314324855804 - timing_s/update_actor:9.988153088837862 - timing_s/step:25.744537329301238 - timing_s/stop_profile:0.00010885298252105713 - timing_per_token_ms/gen:0.20411049996541927 - timing_per_token_ms/update_actor:0.07148078528066486 - timing_per_token_ms/adv:0.0009156323766106407 - perf/total_num_tokens:139732 - perf/time_per_step:25.744537329301238 - perf/throughput:678.4546087033555 (TaskRunner pid=2099163) Training Progress: 66%|██████▌ | 66/100 [53:24<19:03, 33.62s/it] (TaskRunner pid=2099163) step:67 - global_seqlen/min:16776 - global_seqlen/max:26215 - global_seqlen/minmax_diff:9439 - global_seqlen/balanced_min:19749 - global_seqlen/balanced_max:25482 - global_seqlen/mean:20468.5 - actor/entropy:0.2276402860879898 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21511590480804443 - training/rollout_probs_diff_mean:0.004441514611244202 - training/rollout_probs_diff_std:0.012101520784199238 - training/rollout_actor_probs_pearson_corr:0.998544454574585 - rollout_corr/rollout_is_mean:0.9999508261680603 - rollout_corr/rollout_is_ratio_fraction_high:3.911342719220556e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.911342719220556e-05 - rollout_corr/rollout_is_max:2.2540721893310547 - rollout_corr/rollout_is_min:0.3879011869430542 - rollout_corr/rollout_is_std:0.03753895312547684 - rollout_corr/rollout_is_eff_sample_size:0.9985924533404948 - rollout_corr/rollout_is_seq_mean:0.9999358654022217 - rollout_corr/rollout_is_seq_std:0.002823594259098172 - rollout_corr/rollout_is_seq_max:1.0075790882110596 - rollout_corr/rollout_is_seq_min:0.9925720691680908 - rollout_corr/rollout_is_seq_max_deviation:0.00757908821105957 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.324487020727247) - rollout_corr/training_log_ppl:np.float64(0.2760363431443693) - rollout_corr/kl:np.float64(0.0011812180125367888) - rollout_corr/k3_kl:np.float64(0.001190797103646446) - rollout_corr/rollout_ppl:np.float64(1.3228898402303457) - rollout_corr/rollout_log_ppl:np.float64(0.27485512448402005) - rollout_corr/log_ppl_diff:np.float64(0.0011812186603492592) - rollout_corr/log_ppl_abs_diff:np.float64(0.002600522890134016) - rollout_corr/log_ppl_diff_max:np.float64(0.013327181339263916) - rollout_corr/log_ppl_diff_min:np.float64(-0.00664597749710083) - rollout_corr/ppl_ratio:np.float64(1.0011871331371367) - rollout_corr/chi2_token:np.float64(0.002502224873751402) - rollout_corr/chi2_seq:np.float64(0.4073182353749871) - actor/pg_loss:np.float64(0.0001626985613256693) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002055950665180717) - actor/ppo_kl:np.float64(0.00024570608130236593) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6885803639888763) - perf/mfu/actor:np.float64(0.055952627926162825) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.84812545776367) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:4 - critic/score/mean:0.52734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.52734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.08921447396278381 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.08921447396278381 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:299.609375 - response_length/max:7123.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:299.609375 - response_length_non_aborted/max:7123.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:340.03125 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.429827332496643e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4895208682864904) - timing_s/agent_loop/generate_sequences/max:np.float64(35.69096236862242) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9385357215287513) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(35.69096236862242) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:7123 - timing_s/gen:41.56289967894554 - timing_s/reward:0.058504676446318626 - timing_s/old_log_prob:2.5877962671220303 - timing_s/adv:0.14371454156935215 - timing_s/update_actor:10.467117831110954 - timing_s/step:54.904828580096364 - timing_s/stop_profile:5.9584155678749084e-05 - timing_per_token_ms/gen:0.5418891744321452 - timing_per_token_ms/update_actor:0.06392211099439965 - timing_per_token_ms/adv:0.000877656774857416 - perf/total_num_tokens:163748 - perf/time_per_step:54.904828580096364 - perf/throughput:372.79963400924026 (TaskRunner pid=2099163) Training Progress: 67%|██████▋ | 67/100 [54:19<22:00, 40.01s/it] (TaskRunner pid=2099163) step:68 - global_seqlen/min:12125 - global_seqlen/max:24360 - global_seqlen/minmax_diff:12235 - global_seqlen/balanced_min:16093 - global_seqlen/balanced_max:18598 - global_seqlen/mean:16412.875 - actor/entropy:0.2802455723285675 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2584646940231323 - training/rollout_probs_diff_mean:0.005321393255144358 - training/rollout_probs_diff_std:0.012953302823007107 - training/rollout_actor_probs_pearson_corr:0.9985373020172119 - rollout_corr/rollout_is_mean:0.9998360276222229 - rollout_corr/rollout_is_ratio_fraction_high:3.26120643876493e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.6202330589294434 - rollout_corr/rollout_is_min:0.5360647439956665 - rollout_corr/rollout_is_std:0.0403851680457592 - rollout_corr/rollout_is_eff_sample_size:0.998371040293158 - rollout_corr/rollout_is_seq_mean:0.9997125864028931 - rollout_corr/rollout_is_seq_std:0.003077426226809621 - rollout_corr/rollout_is_seq_max:1.0127679109573364 - rollout_corr/rollout_is_seq_min:0.9891027808189392 - rollout_corr/rollout_is_seq_max_deviation:0.012767910957336426 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.368599426932633) - rollout_corr/training_log_ppl:np.float64(0.30979057060176274) - rollout_corr/kl:np.float64(0.0015277589002717917) - rollout_corr/k3_kl:np.float64(0.0013763565973192726) - rollout_corr/rollout_ppl:np.float64(1.3664680924266577) - rollout_corr/rollout_log_ppl:np.float64(0.3082628087795456) - rollout_corr/log_ppl_diff:np.float64(0.001527761822217144) - rollout_corr/log_ppl_abs_diff:np.float64(0.0030798379593761638) - rollout_corr/log_ppl_diff_max:np.float64(0.01730947196483612) - rollout_corr/log_ppl_diff_min:np.float64(-0.008380800485610962) - rollout_corr/ppl_ratio:np.float64(1.0015360116958618) - rollout_corr/chi2_token:np.float64(0.002355032367631793) - rollout_corr/chi2_seq:np.float64(0.5932061832863837) - actor/pg_loss:np.float64(0.00043787446338683367) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020217109231452923) - actor/ppo_kl:np.float64(0.00030286650112287816) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6202314719557762) - perf/mfu/actor:np.float64(0.04649614156988041) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.8221549987793) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:4 - critic/score/mean:0.47265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.47265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.05651262775063515 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.05651262775063515 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:239.55859375 - response_length/max:3687.0 - response_length/min:91.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:239.55859375 - response_length_non_aborted/max:3687.0 - response_length_non_aborted/min:91.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.34375 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.628541111946106e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.119384104385972) - timing_s/agent_loop/generate_sequences/max:np.float64(18.886547598987818) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.397038237679226) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.886547598987818) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:3687 - timing_s/gen:24.744802374392748 - timing_s/reward:0.056385381147265434 - timing_s/old_log_prob:2.4465242717415094 - timing_s/adv:0.11799081787467003 - timing_s/update_actor:10.066473811864853 - timing_s/step:37.518324200063944 - timing_s/stop_profile:5.867891013622284e-05 - timing_per_token_ms/gen:0.4034895294795563 - timing_per_token_ms/update_actor:0.0766659848736499 - timing_per_token_ms/adv:0.0008986147907867301 - perf/total_num_tokens:131303 - perf/time_per_step:37.518324200063944 - perf/throughput:437.4629024601271 (TaskRunner pid=2099163) Training Progress: 68%|██████▊ | 68/100 [54:56<20:56, 39.27s/it] (TaskRunner pid=2099163) step:69 - global_seqlen/min:14077 - global_seqlen/max:22439 - global_seqlen/minmax_diff:8362 - global_seqlen/balanced_min:18403 - global_seqlen/balanced_max:18420 - global_seqlen/mean:18416.125 - actor/entropy:0.2733437716960907 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3037760257720947 - training/rollout_probs_diff_mean:0.005474385805428028 - training/rollout_probs_diff_std:0.013468709774315357 - training/rollout_actor_probs_pearson_corr:0.9984379410743713 - rollout_corr/rollout_is_mean:0.9997246861457825 - rollout_corr/rollout_is_ratio_fraction_high:1.5642353901057504e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.692705988418311e-05 - rollout_corr/rollout_is_max:2.0041916370391846 - rollout_corr/rollout_is_min:0.16430728137493134 - rollout_corr/rollout_is_std:0.04152041673660278 - rollout_corr/rollout_is_eff_sample_size:0.9982780119380065 - rollout_corr/rollout_is_seq_mean:0.9996422529220581 - rollout_corr/rollout_is_seq_std:0.0031316319946199656 - rollout_corr/rollout_is_seq_max:1.0090826749801636 - rollout_corr/rollout_is_seq_min:0.9910612106323242 - rollout_corr/rollout_is_seq_max_deviation:0.009082674980163574 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3545742672868073) - rollout_corr/training_log_ppl:np.float64(0.2993480234290473) - rollout_corr/kl:np.float64(0.0016278082214427059) - rollout_corr/k3_kl:np.float64(0.0013303804649922313) - rollout_corr/rollout_ppl:np.float64(1.3523246650584042) - rollout_corr/rollout_log_ppl:np.float64(0.29772021461394615) - rollout_corr/log_ppl_diff:np.float64(0.0016278088151011616) - rollout_corr/log_ppl_abs_diff:np.float64(0.003158511739457026) - rollout_corr/log_ppl_diff_max:np.float64(0.016750067472457886) - rollout_corr/log_ppl_diff_min:np.float64(-0.006825670599937439) - rollout_corr/ppl_ratio:np.float64(1.0016362892929465) - rollout_corr/chi2_token:np.float64(0.002009368734434247) - rollout_corr/chi2_seq:np.float64(1.0319822325836867) - actor/pg_loss:np.float64(0.0003217571647837758) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0027308197345519147) - actor/ppo_kl:np.float64(0.00029400911517996065) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.7714866697788239) - perf/mfu/actor:np.float64(0.053972941498170224) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.87010192871094) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:4 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.012910807505249977 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.012910807505249977 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:249.72265625 - response_length/max:1039.0 - response_length/min:89.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:249.72265625 - response_length_non_aborted/max:1039.0 - response_length_non_aborted/min:89.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:325.78125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.7713518142700195e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6453578509390354) - timing_s/agent_loop/generate_sequences/max:np.float64(6.574762849137187) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3507946640675073) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.574762849137187) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:1039 - timing_s/gen:12.447308329865336 - timing_s/reward:0.05588654242455959 - timing_s/old_log_prob:2.361416006460786 - timing_s/adv:0.11917807720601559 - timing_s/update_actor:9.651425044983625 - timing_s/step:24.725356981158257 - timing_s/stop_profile:0.00011798366904258728 - timing_per_token_ms/gen:0.19470519372843836 - timing_per_token_ms/update_actor:0.0655093365527739 - timing_per_token_ms/adv:0.0008089247684163715 - perf/total_num_tokens:147329 - perf/time_per_step:24.725356981158257 - perf/throughput:744.8274665572613 (TaskRunner pid=2099163) Training Progress: 69%|██████▉ | 69/100 [55:21<18:02, 34.92s/it] (TaskRunner pid=2099163) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2099163) validation generation end (TaskRunner pid=2099163) [prompt] system (TaskRunner pid=2099163) (TaskRunner pid=2099163) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) user (TaskRunner pid=2099163) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2099163) (TaskRunner pid=2099163) A: -0.01 (TaskRunner pid=2099163) B: 1.69 (TaskRunner pid=2099163) C: 2.49 (TaskRunner pid=2099163) D: 0.45 (TaskRunner pid=2099163) Please reason step by step. (TaskRunner pid=2099163) assistant (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) [response] (TaskRunner pid=2099163) The folding stability score for a protein sequence can be determined using computational tools like I-TASSER, which predicts the structure and calculates stability scores. The provided sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR' would need to be input into such a tool to obtain the folding stability score. Based on known scores from similar sequences and computational predictions, the folding stability score for this specific sequence is approximately 1.69. (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) B (TaskRunner pid=2099163) (TaskRunner pid=2099163) [ground_truth] B (TaskRunner pid=2099163) [score] 1.0 (TaskRunner pid=2099163) [acc] 1.0 (TaskRunner pid=2099163) [pred] B (TaskRunner pid=2099163) [incorrect_format] 1 (TaskRunner pid=2099163) [feedback] (TaskRunner pid=2099163) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (WorkerDict pid=2099539) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60/actor (TaskRunner pid=2099163) step:70 - global_seqlen/min:12908 - global_seqlen/max:23884 - global_seqlen/minmax_diff:10976 - global_seqlen/balanced_min:18177 - global_seqlen/balanced_max:18200 - global_seqlen/mean:18194.5 - actor/entropy:0.27147170901298523 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2765524983406067 - training/rollout_probs_diff_mean:0.005476649384945631 - training/rollout_probs_diff_std:0.013665086589753628 - training/rollout_actor_probs_pearson_corr:0.9983801245689392 - rollout_corr/rollout_is_mean:0.9999741315841675 - rollout_corr/rollout_is_ratio_fraction_high:1.597342088643927e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.584052168065682e-05 - rollout_corr/rollout_is_max:2.8377625942230225 - rollout_corr/rollout_is_min:0.39244869351387024 - rollout_corr/rollout_is_std:0.042557571083307266 - rollout_corr/rollout_is_eff_sample_size:0.9981917712502576 - rollout_corr/rollout_is_seq_mean:0.9999601244926453 - rollout_corr/rollout_is_seq_std:0.003185849403962493 - rollout_corr/rollout_is_seq_max:1.0091570615768433 - rollout_corr/rollout_is_seq_min:0.9889717698097229 - rollout_corr/rollout_is_seq_max_deviation:0.0110282301902771 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3532036440446973) - rollout_corr/training_log_ppl:np.float64(0.297806289701839) - rollout_corr/kl:np.float64(0.001792206864962509) - rollout_corr/k3_kl:np.float64(0.0015404767690370136) - rollout_corr/rollout_ppl:np.float64(1.3506689360365272) - rollout_corr/rollout_log_ppl:np.float64(0.2960140811483143) - rollout_corr/log_ppl_diff:np.float64(0.0017922085535246879) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032309792877640575) - rollout_corr/log_ppl_diff_max:np.float64(0.019662469625473022) - rollout_corr/log_ppl_diff_min:np.float64(-0.009732484817504883) - rollout_corr/ppl_ratio:np.float64(1.0018020996358246) - rollout_corr/chi2_token:np.float64(0.0025641866959631443) - rollout_corr/chi2_seq:np.float64(0.30121213220991194) - actor/pg_loss:np.float64(0.00043956295121461153) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002650087731126405) - actor/ppo_kl:np.float64(0.0007622978668138103) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.8055207878351212) - perf/mfu/actor:np.float64(0.05315399906233151) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.96969223022461) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.395) - val-aux/sciknoweval/reward/std@16:np.float64(0.25608552674596197) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.50316) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.2167468179495711) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.28751999999999994) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.21605136415229434) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.39537999999999995) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2547211208942672) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.58856) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.15622607807957983) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.19984) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1565882935853783) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.40007999999999994) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.209600774094245) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.6486799999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.10171288109893938) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.1405) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09636893444194905) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.4068) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.15999762458359668) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.6872) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.060830405323077257) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.10672000000000001) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.050487971198454974) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.41072) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11352260117902001) - val-aux/sciknoweval/score/mean@16:np.float64(0.395) - val-aux/sciknoweval/score/std@16:np.float64(0.25608552674596197) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.50316) - val-aux/sciknoweval/score/best@2/std:np.float64(0.2167468179495711) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.28751999999999994) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.21605136415229434) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.39537999999999995) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2547211208942672) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.58856) - val-aux/sciknoweval/score/best@4/std:np.float64(0.15622607807957983) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.19984) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1565882935853783) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.40007999999999994) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.209600774094245) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.6486799999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.10171288109893938) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.1405) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09636893444194905) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.4068) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.15999762458359668) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.6872) - val-aux/sciknoweval/score/best@16/std:np.float64(0.060830405323077257) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.10672000000000001) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.050487971198454974) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.41072) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11352260117902001) - val-core/sciknoweval/acc/mean@16:np.float64(0.395) - val-aux/sciknoweval/acc/std@16:np.float64(0.25608552674596197) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.50316) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.2167468179495711) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.28751999999999994) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.21605136415229434) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.39537999999999995) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2547211208942672) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.58856) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.15622607807957983) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.19984) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1565882935853783) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.40007999999999994) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.209600774094245) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.6486799999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.10171288109893938) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.1405) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09636893444194905) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.4068) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.15999762458359668) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.6872) - val-core/sciknoweval/acc/best@16/std:np.float64(0.060830405323077257) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.10672000000000001) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.050487971198454974) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.41072) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11352260117902001) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.004841229182759271) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99994) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0010938007131100255) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.99752) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.006591631057636644) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9986799999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.004965641952456903) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9954599999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.008377851753283773) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9996600000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0025854206620973697) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9919) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.009817840903172144) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99992) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0012623787070447602) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.98744) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.00966676781556276) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999800000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0006321392251711644) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:4 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.019984744489192963 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.019984744489192963 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:244.546875 - response_length/max:1104.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:244.546875 - response_length_non_aborted/max:1104.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:324.03125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.822472929954529e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0265482608228922) - timing_s/agent_loop/generate_sequences/max:np.float64(6.8141403291374445) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.4398168486659415) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.8141403291374445) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:1104 - timing_s/gen:12.706574181094766 - timing_s/reward:0.055348560214042664 - timing_s/old_log_prob:2.4000668544322252 - timing_s/adv:0.11832345649600029 - timing_s/update_actor:9.746209550648928 - timing_s/step:25.11487785167992 - timing_s/testing:66.92163533717394 - timing_s/save_checkpoint:6.685545779764652 - timing_s/stop_profile:0.000119037926197052 - timing_per_token_ms/gen:0.20296744906227662 - timing_per_token_ms/update_actor:0.066958487115948 - timing_per_token_ms/adv:0.0008129067609442433 - perf/total_num_tokens:145556 - perf/time_per_step:25.11487785167992 - perf/throughput:724.4510647215025 (TaskRunner pid=2099163) Training Progress: 70%|███████ | 70/100 [57:00<27:02, 54.08s/it] (TaskRunner pid=2099163) step:71 - global_seqlen/min:11983 - global_seqlen/max:23722 - global_seqlen/minmax_diff:11739 - global_seqlen/balanced_min:18369 - global_seqlen/balanced_max:18382 - global_seqlen/mean:18378.375 - actor/entropy:0.26438087224960327 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.49391263723373413 - training/rollout_probs_diff_mean:0.0053352005779743195 - training/rollout_probs_diff_std:0.013760312460362911 - training/rollout_actor_probs_pearson_corr:0.9983255863189697 - rollout_corr/rollout_is_mean:1.0000841617584229 - rollout_corr/rollout_is_ratio_fraction_high:1.5517589417868294e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.655276643461548e-05 - rollout_corr/rollout_is_max:2.3463339805603027 - rollout_corr/rollout_is_min:0.04733983799815178 - rollout_corr/rollout_is_std:0.04175732657313347 - rollout_corr/rollout_is_eff_sample_size:0.9982597172135673 - rollout_corr/rollout_is_seq_mean:1.000182032585144 - rollout_corr/rollout_is_seq_std:0.0031579334754496813 - rollout_corr/rollout_is_seq_max:1.0100078582763672 - rollout_corr/rollout_is_seq_min:0.9906702637672424 - rollout_corr/rollout_is_seq_max_deviation:0.010007858276367188 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3460410116240382) - rollout_corr/training_log_ppl:np.float64(0.2919461800775025) - rollout_corr/kl:np.float64(0.0015395511177251908) - rollout_corr/k3_kl:np.float64(0.0017241420433720123) - rollout_corr/rollout_ppl:np.float64(1.3439419087953866) - rollout_corr/rollout_log_ppl:np.float64(0.29040662810439244) - rollout_corr/log_ppl_diff:np.float64(0.0015395519731100649) - rollout_corr/log_ppl_abs_diff:np.float64(0.003382161114132032) - rollout_corr/log_ppl_diff_max:np.float64(0.01569351553916931) - rollout_corr/log_ppl_diff_min:np.float64(-0.013844609260559082) - rollout_corr/ppl_ratio:np.float64(1.0015493920072913) - rollout_corr/chi2_token:np.float64(0.004469351610168815) - rollout_corr/chi2_seq:np.float64(0.9466164640616626) - actor/pg_loss:np.float64(0.0009243421955034137) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002824900551331666) - actor/ppo_kl:np.float64(0.0007459541272538672) - actor/pg_clipfrac_lower:np.float64(6.143417522253003e-05) - actor/grad_norm:np.float64(0.7537118345499039) - perf/mfu/actor:np.float64(0.05423491984814467) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(105.76616668701172) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:5 - critic/score/mean:0.44921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003462362103164196 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003462362103164196 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:251.73046875 - response_length/max:1004.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:251.73046875 - response_length_non_aborted/max:1004.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:322.59375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002311021089553833 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0665221828967333) - timing_s/agent_loop/generate_sequences/max:np.float64(6.538059942424297) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.5836986561625963) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.538059942424297) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1004 - timing_s/gen:12.774788860231638 - timing_s/reward:0.05202013999223709 - timing_s/old_log_prob:2.479814240708947 - timing_s/adv:0.12360932119190693 - timing_s/update_actor:9.744195263832808 - timing_s/step:25.277720926329494 - timing_s/stop_profile:0.00012053363025188446 - timing_per_token_ms/gen:0.19823392548813118 - timing_per_token_ms/update_actor:0.06627486967586095 - timing_per_token_ms/adv:0.0008407253170635797 - perf/total_num_tokens:147027 - perf/time_per_step:25.277720926329494 - perf/throughput:727.0582286101958 (TaskRunner pid=2099163) Training Progress: 71%|███████ | 71/100 [57:26<22:02, 45.59s/it] (TaskRunner pid=2099163) step:72 - global_seqlen/min:13664 - global_seqlen/max:22468 - global_seqlen/minmax_diff:8804 - global_seqlen/balanced_min:18001 - global_seqlen/balanced_max:18006 - global_seqlen/mean:18004.25 - actor/entropy:0.2748372554779053 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3519516587257385 - training/rollout_probs_diff_mean:0.005624288693070412 - training/rollout_probs_diff_std:0.013938440941274166 - training/rollout_actor_probs_pearson_corr:0.9983092546463013 - rollout_corr/rollout_is_mean:1.0002027750015259 - rollout_corr/rollout_is_ratio_fraction_high:1.6573303582845256e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.314660716569051e-05 - rollout_corr/rollout_is_max:3.3772945404052734 - rollout_corr/rollout_is_min:0.4679282307624817 - rollout_corr/rollout_is_std:0.04276713728904724 - rollout_corr/rollout_is_eff_sample_size:0.9981750237537251 - rollout_corr/rollout_is_seq_mean:1.0002542734146118 - rollout_corr/rollout_is_seq_std:0.0032979086972773075 - rollout_corr/rollout_is_seq_max:1.013193964958191 - rollout_corr/rollout_is_seq_min:0.9926396012306213 - rollout_corr/rollout_is_seq_max_deviation:0.013193964958190918 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3490465520881116) - rollout_corr/training_log_ppl:np.float64(0.29478535568341613) - rollout_corr/kl:np.float64(0.001085285884805387) - rollout_corr/k3_kl:np.float64(0.0015339718757445553) - rollout_corr/rollout_ppl:np.float64(1.3474991344846785) - rollout_corr/rollout_log_ppl:np.float64(0.29370006851968355) - rollout_corr/log_ppl_diff:np.float64(0.0010852871637325734) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028866918582934886) - rollout_corr/log_ppl_diff_max:np.float64(0.020227819681167603) - rollout_corr/log_ppl_diff_min:np.float64(-0.012600928544998169) - rollout_corr/ppl_ratio:np.float64(1.0010935172904283) - rollout_corr/chi2_token:np.float64(0.003948581404983997) - rollout_corr/chi2_seq:np.float64(0.5234741037711501) - actor/pg_loss:np.float64(0.00017087801825255156) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0027429519377619727) - actor/ppo_kl:np.float64(0.0003700984995536061) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.679483950138092) - perf/mfu/actor:np.float64(0.05314325335515786) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(105.71037673950195) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:5 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009772034361958504 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009772034361958504 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:235.6953125 - response_length/max:723.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:235.6953125 - response_length_non_aborted/max:723.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:326.9375 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.920448064804077e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9485215358436108) - timing_s/agent_loop/generate_sequences/max:np.float64(5.383510269224644) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.490466148345149) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.383510269224644) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:723 - timing_s/gen:11.147572057321668 - timing_s/reward:0.054647378623485565 - timing_s/old_log_prob:2.295195208862424 - timing_s/adv:0.11946884915232658 - timing_s/update_actor:9.803522318601608 - timing_s/step:23.509083898738027 - timing_s/stop_profile:0.00010769441723823547 - timing_per_token_ms/gen:0.18475209747293028 - timing_per_token_ms/update_actor:0.06806394544761382 - timing_per_token_ms/adv:0.0008294489436683463 - perf/total_num_tokens:144034 - perf/time_per_step:23.509083898738027 - perf/throughput:765.8422624016614 (TaskRunner pid=2099163) Training Progress: 72%|███████▏ | 72/100 [57:49<18:11, 38.98s/it] (TaskRunner pid=2099163) step:73 - global_seqlen/min:12492 - global_seqlen/max:19109 - global_seqlen/minmax_diff:6617 - global_seqlen/balanced_min:15295 - global_seqlen/balanced_max:16032 - global_seqlen/mean:15388.75 - actor/entropy:0.2661946415901184 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37658703327178955 - training/rollout_probs_diff_mean:0.005457909312099218 - training/rollout_probs_diff_std:0.013575312681496143 - training/rollout_actor_probs_pearson_corr:0.9983363151550293 - rollout_corr/rollout_is_mean:1.0002198219299316 - rollout_corr/rollout_is_ratio_fraction_high:1.8412814824841917e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.1717958450317383 - rollout_corr/rollout_is_min:0.5105080008506775 - rollout_corr/rollout_is_std:0.041652996093034744 - rollout_corr/rollout_is_eff_sample_size:0.9982688644919676 - rollout_corr/rollout_is_seq_mean:1.000148057937622 - rollout_corr/rollout_is_seq_std:0.0032269360963255167 - rollout_corr/rollout_is_seq_max:1.0114909410476685 - rollout_corr/rollout_is_seq_min:0.9871060252189636 - rollout_corr/rollout_is_seq_max_deviation:0.012893974781036377 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3228384959511459) - rollout_corr/training_log_ppl:np.float64(0.2764760266873054) - rollout_corr/kl:np.float64(0.0013183017936526653) - rollout_corr/k3_kl:np.float64(0.001363425177373756) - rollout_corr/rollout_ppl:np.float64(1.321010212879628) - rollout_corr/rollout_log_ppl:np.float64(0.2751577235030709) - rollout_corr/log_ppl_diff:np.float64(0.0013183031842345372) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031554124579997733) - rollout_corr/log_ppl_diff_max:np.float64(0.014645874500274658) - rollout_corr/log_ppl_diff_min:np.float64(-0.012716293334960938) - rollout_corr/ppl_ratio:np.float64(1.001326964236796) - rollout_corr/chi2_token:np.float64(0.002702318597584963) - rollout_corr/chi2_seq:np.float64(0.5265928467269987) - actor/pg_loss:np.float64(0.00030526239424943924) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001643506417167373) - actor/ppo_kl:np.float64(0.0005654427488943625) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5535854026675224) - perf/mfu/actor:np.float64(0.044940075380136224) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.07952976226807) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:5 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.012893574312329292 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.012893574312329292 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:212.1484375 - response_length/max:1646.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:212.1484375 - response_length_non_aborted/max:1646.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.75 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001221541315317154 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7583393473178148) - timing_s/agent_loop/generate_sequences/max:np.float64(9.147600388154387) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.0318894187221304) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.147600388154387) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:1646 - timing_s/gen:15.125937445089221 - timing_s/reward:0.051669539883732796 - timing_s/old_log_prob:2.325926223769784 - timing_s/adv:0.12074156478047371 - timing_s/update_actor:9.70467483624816 - timing_s/step:27.41567281074822 - timing_s/stop_profile:5.909241735935211e-05 - timing_per_token_ms/gen:0.27851109271016794 - timing_per_token_ms/update_actor:0.07882929767076728 - timing_per_token_ms/adv:0.0009807616341521706 - perf/total_num_tokens:123110 - perf/time_per_step:27.41567281074822 - perf/throughput:561.312140914043 (TaskRunner pid=2099163) Training Progress: 73%|███████▎ | 73/100 [58:17<15:59, 35.52s/it] (TaskRunner pid=2099163) step:74 - global_seqlen/min:13372 - global_seqlen/max:23732 - global_seqlen/minmax_diff:10360 - global_seqlen/balanced_min:17453 - global_seqlen/balanced_max:17463 - global_seqlen/mean:17459.25 - actor/entropy:0.2593246102333069 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3182520568370819 - training/rollout_probs_diff_mean:0.005462802015244961 - training/rollout_probs_diff_std:0.013881351798772812 - training/rollout_actor_probs_pearson_corr:0.9982698559761047 - rollout_corr/rollout_is_mean:0.9997365474700928 - rollout_corr/rollout_is_ratio_fraction_high:3.281162935309112e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.281162935309112e-05 - rollout_corr/rollout_is_max:3.5407092571258545 - rollout_corr/rollout_is_min:0.41278213262557983 - rollout_corr/rollout_is_std:0.04266597703099251 - rollout_corr/rollout_is_eff_sample_size:0.9981817939506096 - rollout_corr/rollout_is_seq_mean:0.9995617866516113 - rollout_corr/rollout_is_seq_std:0.0030475708190351725 - rollout_corr/rollout_is_seq_max:1.0164185762405396 - rollout_corr/rollout_is_seq_min:0.9908937811851501 - rollout_corr/rollout_is_seq_max_deviation:0.01641857624053955 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3252906440757215) - rollout_corr/training_log_ppl:np.float64(0.27802602780866437) - rollout_corr/kl:np.float64(0.0018082092363584934) - rollout_corr/k3_kl:np.float64(0.001329023798803064) - rollout_corr/rollout_ppl:np.float64(1.3228671373799443) - rollout_corr/rollout_log_ppl:np.float64(0.27621781785273924) - rollout_corr/log_ppl_diff:np.float64(0.001808209955925122) - rollout_corr/log_ppl_abs_diff:np.float64(0.0030505233735311776) - rollout_corr/log_ppl_diff_max:np.float64(0.0164526104927063) - rollout_corr/log_ppl_diff_min:np.float64(-0.006425350904464722) - rollout_corr/ppl_ratio:np.float64(1.0018170916009694) - rollout_corr/chi2_token:np.float64(0.001663966104388237) - rollout_corr/chi2_seq:np.float64(0.45517092291265726) - actor/pg_loss:np.float64(0.00037874444387853146) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015740662133794103) - actor/ppo_kl:np.float64(0.00035873608290160064) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5778901502490044) - perf/mfu/actor:np.float64(0.05122353731438829) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.05221939086914) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:5 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.000875660334713757 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.000875660334713757 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:238.1015625 - response_length/max:964.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:238.1015625 - response_length_non_aborted/max:964.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:307.5 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.1486695408821106e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1809429358690977) - timing_s/agent_loop/generate_sequences/max:np.float64(6.445059455931187) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.5263324798143003) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.445059455931187) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:964 - timing_s/gen:12.528105229139328 - timing_s/reward:0.05381600931286812 - timing_s/old_log_prob:2.4174425285309553 - timing_s/adv:0.12314742244780064 - timing_s/update_actor:9.780229168012738 - timing_s/step:24.991026105359197 - timing_s/stop_profile:0.00012196414172649384 - timing_per_token_ms/gen:0.20553376692488315 - timing_per_token_ms/update_actor:0.07002183060564413 - timing_per_token_ms/adv:0.0008816774950799765 - perf/total_num_tokens:139674 - perf/time_per_step:24.991026105359197 - perf/throughput:698.6207739687788 (TaskRunner pid=2099163) Training Progress: 74%|███████▍ | 74/100 [58:42<14:01, 32.37s/it] (TaskRunner pid=2099163) step:75 - global_seqlen/min:13280 - global_seqlen/max:25094 - global_seqlen/minmax_diff:11814 - global_seqlen/balanced_min:19571 - global_seqlen/balanced_max:19676 - global_seqlen/mean:19596.625 - actor/entropy:0.24914591014385223 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3969437777996063 - training/rollout_probs_diff_mean:0.005310408771038055 - training/rollout_probs_diff_std:0.013804785907268524 - training/rollout_actor_probs_pearson_corr:0.998227596282959 - rollout_corr/rollout_is_mean:1.0003116130828857 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.204922439996153e-05 - rollout_corr/rollout_is_max:1.7981332540512085 - rollout_corr/rollout_is_min:0.2928912937641144 - rollout_corr/rollout_is_std:0.042179133743047714 - rollout_corr/rollout_is_eff_sample_size:0.9982251491428059 - rollout_corr/rollout_is_seq_mean:1.000361680984497 - rollout_corr/rollout_is_seq_std:0.0032836582977324724 - rollout_corr/rollout_is_seq_max:1.011388897895813 - rollout_corr/rollout_is_seq_min:0.9914733171463013 - rollout_corr/rollout_is_seq_max_deviation:0.011388897895812988 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3274900810793042) - rollout_corr/training_log_ppl:np.float64(0.2792092165327631) - rollout_corr/kl:np.float64(0.0009840833473546695) - rollout_corr/k3_kl:np.float64(0.0012072064104131641) - rollout_corr/rollout_ppl:np.float64(1.3261237805709243) - rollout_corr/rollout_log_ppl:np.float64(0.27822513245337177) - rollout_corr/log_ppl_diff:np.float64(0.000984084079391323) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027362913970137015) - rollout_corr/log_ppl_diff_max:np.float64(0.01530107855796814) - rollout_corr/log_ppl_diff_min:np.float64(-0.010391294956207275) - rollout_corr/ppl_ratio:np.float64(1.0009908275678754) - rollout_corr/chi2_token:np.float64(0.0028621042147278786) - rollout_corr/chi2_seq:np.float64(0.6469766469672322) - actor/pg_loss:np.float64(-0.00014654418919235468) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010958636398754606) - actor/ppo_kl:np.float64(0.0003283484302762929) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5892626643180847) - perf/mfu/actor:np.float64(0.05761350722790884) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.95429992675781) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:5 - critic/score/mean:0.3828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006997780874371529 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.006997780874371529 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:271.08203125 - response_length/max:1332.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:271.08203125 - response_length_non_aborted/max:1332.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:341.3125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.216368198394775e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2000188957899809) - timing_s/agent_loop/generate_sequences/max:np.float64(8.659884678199887) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.824555200713803) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.659884678199887) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:1332 - timing_s/gen:14.400074571371078 - timing_s/reward:0.05767091363668442 - timing_s/old_log_prob:2.4206353165209293 - timing_s/adv:0.12469611130654812 - timing_s/update_actor:9.697892174124718 - timing_s/step:26.788869658485055 - timing_s/stop_profile:0.00010457262396812439 - timing_per_token_ms/gen:0.20750283976787295 - timing_per_token_ms/update_actor:0.0618594539501363 - timing_per_token_ms/adv:0.0007953927736698801 - perf/total_num_tokens:156773 - perf/time_per_step:26.788869658485055 - perf/throughput:731.5211597139188 (TaskRunner pid=2099163) Training Progress: 75%|███████▌ | 75/100 [59:09<12:47, 30.71s/it] (TaskRunner pid=2099163) step:76 - global_seqlen/min:11824 - global_seqlen/max:21898 - global_seqlen/minmax_diff:10074 - global_seqlen/balanced_min:16652 - global_seqlen/balanced_max:16677 - global_seqlen/mean:16667.125 - actor/entropy:0.25044116377830505 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.49410587549209595 - training/rollout_probs_diff_mean:0.005233317147940397 - training/rollout_probs_diff_std:0.013545987196266651 - training/rollout_actor_probs_pearson_corr:0.9983153343200684 - rollout_corr/rollout_is_mean:1.000156044960022 - rollout_corr/rollout_is_ratio_fraction_high:1.6923624571063556e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.461812103632838e-05 - rollout_corr/rollout_is_max:2.701498508453369 - rollout_corr/rollout_is_min:0.2689496576786041 - rollout_corr/rollout_is_std:0.0415240079164505 - rollout_corr/rollout_is_eff_sample_size:0.9982794375291114 - rollout_corr/rollout_is_seq_mean:1.000076413154602 - rollout_corr/rollout_is_seq_std:0.003434262704104185 - rollout_corr/rollout_is_seq_max:1.0094366073608398 - rollout_corr/rollout_is_seq_min:0.9889258742332458 - rollout_corr/rollout_is_seq_max_deviation:0.01107412576675415 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3120325771160424) - rollout_corr/training_log_ppl:np.float64(0.26778477303741965) - rollout_corr/kl:np.float64(0.001165033413258243) - rollout_corr/k3_kl:np.float64(0.0011289213275631482) - rollout_corr/rollout_ppl:np.float64(1.3104448192752898) - rollout_corr/rollout_log_ppl:np.float64(0.26661973919544835) - rollout_corr/log_ppl_diff:np.float64(0.001165033841971308) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029716190765611827) - rollout_corr/log_ppl_diff_max:np.float64(0.01145792007446289) - rollout_corr/log_ppl_diff_min:np.float64(-0.009343594312667847) - rollout_corr/ppl_ratio:np.float64(1.001172634307295) - rollout_corr/chi2_token:np.float64(0.002230182522907853) - rollout_corr/chi2_seq:np.float64(0.7144021939020604) - actor/pg_loss:np.float64(-2.578599378466606e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011448644404481456) - actor/ppo_kl:np.float64(0.00031501110731468884) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.568739764392376) - perf/mfu/actor:np.float64(0.04933886842194326) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.97966003417969) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:5 - critic/score/mean:0.43359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.43359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007423124276101589 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.007423124276101589 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:230.81640625 - response_length/max:1078.0 - response_length/min:106.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:230.81640625 - response_length_non_aborted/max:1078.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.03125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.485520422458649e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2769716065376997) - timing_s/agent_loop/generate_sequences/max:np.float64(6.788432452827692) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.423513734356675) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.788432452827692) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:496 - timing_s/agent_loop/slowest/response_length:1078 - timing_s/gen:12.49279616959393 - timing_s/reward:0.053980570286512375 - timing_s/old_log_prob:2.3466072604060173 - timing_s/adv:0.11904684640467167 - timing_s/update_actor:9.500159479677677 - timing_s/step:24.59877328388393 - timing_s/stop_profile:0.00014404766261577606 - timing_per_token_ms/gen:0.2114233811639041 - timing_per_token_ms/update_actor:0.07124923674357213 - timing_per_token_ms/adv:0.0008928267952981668 - perf/total_num_tokens:133337 - perf/time_per_step:24.59877328388393 - perf/throughput:677.5591940155647 (TaskRunner pid=2099163) Training Progress: 76%|███████▌ | 76/100 [59:33<11:33, 28.89s/it] (TaskRunner pid=2099163) step:77 - global_seqlen/min:14979 - global_seqlen/max:24145 - global_seqlen/minmax_diff:9166 - global_seqlen/balanced_min:17784 - global_seqlen/balanced_max:17827 - global_seqlen/mean:17795.375 - actor/entropy:0.2697010934352875 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3591962456703186 - training/rollout_probs_diff_mean:0.005411113146692514 - training/rollout_probs_diff_std:0.013911079615354538 - training/rollout_actor_probs_pearson_corr:0.9983001947402954 - rollout_corr/rollout_is_mean:0.9998016357421875 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00011526238813530654 - rollout_corr/rollout_is_max:1.7653801441192627 - rollout_corr/rollout_is_min:0.3611298203468323 - rollout_corr/rollout_is_std:0.04165085032582283 - rollout_corr/rollout_is_eff_sample_size:0.9982674389310602 - rollout_corr/rollout_is_seq_mean:0.999787449836731 - rollout_corr/rollout_is_seq_std:0.0032496904022991657 - rollout_corr/rollout_is_seq_max:1.0096925497055054 - rollout_corr/rollout_is_seq_min:0.989059329032898 - rollout_corr/rollout_is_seq_max_deviation:0.01094067096710205 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3272224520333111) - rollout_corr/training_log_ppl:np.float64(0.2796602731687017) - rollout_corr/kl:np.float64(0.0015744009766365252) - rollout_corr/k3_kl:np.float64(0.00129540834745967) - rollout_corr/rollout_ppl:np.float64(1.325115962419659) - rollout_corr/rollout_log_ppl:np.float64(0.2780858727055602) - rollout_corr/log_ppl_diff:np.float64(0.001574400463141501) - rollout_corr/log_ppl_abs_diff:np.float64(0.003008296713232994) - rollout_corr/log_ppl_diff_max:np.float64(0.012766122817993164) - rollout_corr/log_ppl_diff_min:np.float64(-0.008344709873199463) - rollout_corr/ppl_ratio:np.float64(1.001582418801263) - rollout_corr/chi2_token:np.float64(0.002242328831925988) - rollout_corr/chi2_seq:np.float64(0.23463519662618637) - actor/pg_loss:np.float64(0.00029490748420357704) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014422528074646834) - actor/ppo_kl:np.float64(0.0004096870738550251) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5797202438116074) - perf/mfu/actor:np.float64(0.05305596361026153) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.87368392944336) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:5 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008615863509476185 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008615863509476185 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:237.23046875 - response_length/max:1068.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:237.23046875 - response_length_non_aborted/max:1068.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:318.875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.313225746154785e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0067482870072126) - timing_s/agent_loop/generate_sequences/max:np.float64(6.926735129207373) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.472584339346213) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.926735129207373) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1068 - timing_s/gen:12.724126374348998 - timing_s/reward:0.05565854348242283 - timing_s/old_log_prob:2.3462998792529106 - timing_s/adv:0.12156063131988049 - timing_s/update_actor:9.614609019830823 - timing_s/step:24.949520375579596 - timing_s/stop_profile:0.00014005228877067566 - timing_per_token_ms/gen:0.20951616759725672 - timing_per_token_ms/update_actor:0.06753586971215009 - timing_per_token_ms/adv:0.0008538779831829934 - perf/total_num_tokens:142363 - perf/time_per_step:24.949520375579596 - perf/throughput:713.2551941727096 (TaskRunner pid=2099163) Training Progress: 77%|███████▋ | 77/100 [59:58<10:37, 27.72s/it] (TaskRunner pid=2099163) step:78 - global_seqlen/min:15123 - global_seqlen/max:23777 - global_seqlen/minmax_diff:8654 - global_seqlen/balanced_min:18759 - global_seqlen/balanced_max:18771 - global_seqlen/mean:18768.625 - actor/entropy:0.25392431020736694 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6629616618156433 - training/rollout_probs_diff_mean:0.00486655393615365 - training/rollout_probs_diff_std:0.012811429798603058 - training/rollout_actor_probs_pearson_corr:0.9985421299934387 - rollout_corr/rollout_is_mean:1.0001100301742554 - rollout_corr/rollout_is_ratio_fraction_high:1.5132027328945696e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.0528109315782785e-05 - rollout_corr/rollout_is_max:2.284006357192993 - rollout_corr/rollout_is_min:0.19336040318012238 - rollout_corr/rollout_is_std:0.03998841717839241 - rollout_corr/rollout_is_eff_sample_size:0.9984037172013397 - rollout_corr/rollout_is_seq_mean:1.000152587890625 - rollout_corr/rollout_is_seq_std:0.003139234147965908 - rollout_corr/rollout_is_seq_max:1.0102607011795044 - rollout_corr/rollout_is_seq_min:0.9885665774345398 - rollout_corr/rollout_is_seq_max_deviation:0.011433422565460205 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3306259801611304) - rollout_corr/training_log_ppl:np.float64(0.2809669480775483) - rollout_corr/kl:np.float64(0.0013480030079620064) - rollout_corr/k3_kl:np.float64(0.0014645395496017954) - rollout_corr/rollout_ppl:np.float64(1.328828570432961) - rollout_corr/rollout_log_ppl:np.float64(0.2796189447399229) - rollout_corr/log_ppl_diff:np.float64(0.0013480033376254141) - rollout_corr/log_ppl_abs_diff:np.float64(0.003176851896569133) - rollout_corr/log_ppl_diff_max:np.float64(0.02776774764060974) - rollout_corr/log_ppl_diff_min:np.float64(-0.009948790073394775) - rollout_corr/ppl_ratio:np.float64(1.0013591742608696) - rollout_corr/chi2_token:np.float64(0.0036836275830864906) - rollout_corr/chi2_seq:np.float64(2.4141830562148243) - actor/pg_loss:np.float64(0.0004103210521861911) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0027973866708634887) - actor/ppo_kl:np.float64(0.0005619901072861921) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.7025328427553177) - perf/mfu/actor:np.float64(0.054799120913851654) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.03145599365234) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:5 - critic/score/mean:0.43359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.43359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.018179239705204964 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.018179239705204964 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:258.14453125 - response_length/max:1046.0 - response_length/min:70.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:258.14453125 - response_length_non_aborted/max:1046.0 - response_length_non_aborted/min:70.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:328.375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010188482701778412 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7525202222168446) - timing_s/agent_loop/generate_sequences/max:np.float64(6.806002235040069) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3579090451748925) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.806002235040069) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:1046 - timing_s/gen:12.32015965692699 - timing_s/reward:0.056415656581521034 - timing_s/old_log_prob:2.315779550001025 - timing_s/adv:0.11760320328176022 - timing_s/update_actor:9.819602735340595 - timing_s/step:24.71673825569451 - timing_s/stop_profile:0.00011189281940460205 - timing_per_token_ms/gen:0.18642898777221745 - timing_per_token_ms/update_actor:0.065399055174131 - timing_per_token_ms/adv:0.0007832433335004577 - perf/total_num_tokens:150149 - perf/time_per_step:24.71673825569451 - perf/throughput:759.3487783800066 (TaskRunner pid=2099163) Training Progress: 78%|███████▊ | 78/100 [1:00:23<09:50, 26.84s/it] (TaskRunner pid=2099163) step:79 - global_seqlen/min:12334 - global_seqlen/max:22428 - global_seqlen/minmax_diff:10094 - global_seqlen/balanced_min:17635 - global_seqlen/balanced_max:17782 - global_seqlen/mean:17654.5 - actor/entropy:0.27384743094444275 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2592231035232544 - training/rollout_probs_diff_mean:0.005185640417039394 - training/rollout_probs_diff_std:0.012722447514533997 - training/rollout_actor_probs_pearson_corr:0.9985835552215576 - rollout_corr/rollout_is_mean:0.9998155236244202 - rollout_corr/rollout_is_ratio_fraction_high:1.6117593986564316e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.223518797312863e-05 - rollout_corr/rollout_is_max:2.173496961593628 - rollout_corr/rollout_is_min:0.37063977122306824 - rollout_corr/rollout_is_std:0.039584673941135406 - rollout_corr/rollout_is_eff_sample_size:0.9984348513727722 - rollout_corr/rollout_is_seq_mean:0.9998263120651245 - rollout_corr/rollout_is_seq_std:0.003011947264894843 - rollout_corr/rollout_is_seq_max:1.014046549797058 - rollout_corr/rollout_is_seq_min:0.9915293455123901 - rollout_corr/rollout_is_seq_max_deviation:0.014046549797058105 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.340323363430798) - rollout_corr/training_log_ppl:np.float64(0.2888445410062559) - rollout_corr/kl:np.float64(0.0010857870700355932) - rollout_corr/k3_kl:np.float64(0.0011207945151454624) - rollout_corr/rollout_ppl:np.float64(1.3388538309372962) - rollout_corr/rollout_log_ppl:np.float64(0.2877587534894701) - rollout_corr/log_ppl_diff:np.float64(0.0010857875167857856) - rollout_corr/log_ppl_abs_diff:np.float64(0.002984179329359904) - rollout_corr/log_ppl_diff_max:np.float64(0.013294041156768799) - rollout_corr/log_ppl_diff_min:np.float64(-0.009193763136863708) - rollout_corr/ppl_ratio:np.float64(1.001093138474971) - rollout_corr/chi2_token:np.float64(0.0023503343109041452) - rollout_corr/chi2_seq:np.float64(0.57042295509018) - actor/pg_loss:np.float64(-0.00013655168004333973) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013187846618620824) - actor/ppo_kl:np.float64(3.9207486743464415e-05) - actor/pg_clipfrac_lower:np.float64(2.0559211407089606e-05) - actor/grad_norm:np.float64(0.6367911696434021) - perf/mfu/actor:np.float64(0.05222786833305854) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.97899627685547) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:5 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013035103678703308 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013035103678703308 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:242.359375 - response_length/max:1251.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:242.359375 - response_length_non_aborted/max:1251.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:309.34375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014719553291797638 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1327050235122442) - timing_s/agent_loop/generate_sequences/max:np.float64(7.875947218388319) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.5553116282462724) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.875947218388319) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:1251 - timing_s/gen:13.897762881591916 - timing_s/reward:0.05496951378881931 - timing_s/old_log_prob:2.362206546589732 - timing_s/adv:0.12380427122116089 - timing_s/update_actor:9.482831593602896 - timing_s/step:26.00793389044702 - timing_s/stop_profile:0.00010822899639606476 - timing_per_token_ms/gen:0.22399849915530778 - timing_per_token_ms/update_actor:0.06714174568525656 - timing_per_token_ms/adv:0.0008765772977226832 - perf/total_num_tokens:141236 - perf/time_per_step:26.00793389044702 - perf/throughput:678.8120915089175 (TaskRunner pid=2099163) Training Progress: 79%|███████▉ | 79/100 [1:00:49<09:18, 26.60s/it] (TaskRunner pid=2099163) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2099163) validation generation end (TaskRunner pid=2099163) [prompt] system (TaskRunner pid=2099163) (TaskRunner pid=2099163) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) user (TaskRunner pid=2099163) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2099163) (TaskRunner pid=2099163) A: -0.01 (TaskRunner pid=2099163) B: 1.69 (TaskRunner pid=2099163) C: 2.49 (TaskRunner pid=2099163) D: 0.45 (TaskRunner pid=2099163) Please reason step by step. (TaskRunner pid=2099163) assistant (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) [response] (TaskRunner pid=2099163) The folding stability score for a specific protein sequence can be determined using computational methods such as those implemented in tools like I-TASSER, DSSP, or other protein structure prediction software. These tools calculate various scores based on the predicted 3D structure of the protein. The score provided (e.g., -0.01, 1.69, 2.49, 0.45) typically reflects the stability of the predicted structure relative to a reference structure. (TaskRunner pid=2099163) (TaskRunner pid=2099163) Given the sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR', it is likely that this sequence has been analyzed using a specific computational tool. Based on known results for similar sequences and the range of scores provided, the folding stability score for this sequence is most likely 0.45. (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) D (TaskRunner pid=2099163) (TaskRunner pid=2099163) [ground_truth] B (TaskRunner pid=2099163) [score] 0.0 (TaskRunner pid=2099163) [acc] 0.0 (TaskRunner pid=2099163) [pred] D (TaskRunner pid=2099163) [incorrect_format] 1 (TaskRunner pid=2099163) [feedback] (TaskRunner pid=2099163) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (WorkerDict pid=2099539) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70/actor (TaskRunner pid=2099163) step:80 - global_seqlen/min:12878 - global_seqlen/max:22644 - global_seqlen/minmax_diff:9766 - global_seqlen/balanced_min:16682 - global_seqlen/balanced_max:16689 - global_seqlen/mean:16687.125 - actor/entropy:0.31366831064224243 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3867309093475342 - training/rollout_probs_diff_mean:0.005574310198426247 - training/rollout_probs_diff_std:0.012939517386257648 - training/rollout_actor_probs_pearson_corr:0.9986551403999329 - rollout_corr/rollout_is_mean:0.9999090433120728 - rollout_corr/rollout_is_ratio_fraction_high:1.7949132598005235e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.589826519601047e-05 - rollout_corr/rollout_is_max:2.8355820178985596 - rollout_corr/rollout_is_min:0.44648870825767517 - rollout_corr/rollout_is_std:0.041322559118270874 - rollout_corr/rollout_is_eff_sample_size:0.9982948816937318 - rollout_corr/rollout_is_seq_mean:0.9998655319213867 - rollout_corr/rollout_is_seq_std:0.003264273749664426 - rollout_corr/rollout_is_seq_max:1.0161584615707397 - rollout_corr/rollout_is_seq_min:0.9922093152999878 - rollout_corr/rollout_is_seq_max_deviation:0.016158461570739746 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3791068121790886) - rollout_corr/training_log_ppl:np.float64(0.31735638427198865) - rollout_corr/kl:np.float64(0.0011907178431336263) - rollout_corr/k3_kl:np.float64(0.0012447628572545) - rollout_corr/rollout_ppl:np.float64(1.3774029430933297) - rollout_corr/rollout_log_ppl:np.float64(0.3161656655429397) - rollout_corr/log_ppl_diff:np.float64(0.0011907187290489674) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031865338096395135) - rollout_corr/log_ppl_diff_max:np.float64(0.015649110078811646) - rollout_corr/log_ppl_diff_min:np.float64(-0.012567013502120972) - rollout_corr/ppl_ratio:np.float64(1.0011996023822576) - rollout_corr/chi2_token:np.float64(0.0025439781602472067) - rollout_corr/chi2_seq:np.float64(0.696196470875293) - actor/pg_loss:np.float64(0.0002089452464133501) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0019131674780510366) - actor/ppo_kl:np.float64(0.0001621537604137302) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6633199006319046) - perf/mfu/actor:np.float64(0.049463591662902) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.01802444458008) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.41625) - val-aux/sciknoweval/reward/std@16:np.float64(0.3144664633085353) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.55652) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.28145093411213773) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.27288) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2558732573095932) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.4132) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.3137503776392298) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6777599999999998) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.19852437956426017) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.17295999999999995) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1598024868585598) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.41869999999999996) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.2850227986027261) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.74804) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.10224935564981508) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.12108) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07521408052792339) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.41981999999999997) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.2487908426641159) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7791199999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.046372263261744234) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.09526) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.035559683891627565) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.42042) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.20911651768131542) - val-aux/sciknoweval/score/mean@16:np.float64(0.41625) - val-aux/sciknoweval/score/std@16:np.float64(0.3144664633085353) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.55652) - val-aux/sciknoweval/score/best@2/std:np.float64(0.28145093411213773) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.27288) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.2558732573095932) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.4132) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.3137503776392298) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6777599999999998) - val-aux/sciknoweval/score/best@4/std:np.float64(0.19852437956426017) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.17295999999999995) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1598024868585598) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.41869999999999996) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.2850227986027261) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.74804) - val-aux/sciknoweval/score/best@8/std:np.float64(0.10224935564981508) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.12108) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07521408052792339) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.41981999999999997) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.2487908426641159) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7791199999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.046372263261744234) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.09526) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.035559683891627565) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.42042) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.20911651768131542) - val-core/sciknoweval/acc/mean@16:np.float64(0.41625) - val-aux/sciknoweval/acc/std@16:np.float64(0.3144664633085353) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.55652) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.28145093411213773) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.27288) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.2558732573095932) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.4132) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.3137503776392298) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6777599999999998) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.19852437956426017) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.17295999999999995) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1598024868585598) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.41869999999999996) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.2850227986027261) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.74804) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.10224935564981508) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.12108) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07521408052792339) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.41981999999999997) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.2487908426641159) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7791199999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.046372263261744234) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.09526) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.035559683891627565) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.42042) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.20911651768131542) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:5 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010260173119604588 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010260173119604588 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:217.62890625 - response_length/max:808.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:217.62890625 - response_length_non_aborted/max:808.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:303.84375 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013283640146255493 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0369690340012312) - timing_s/agent_loop/generate_sequences/max:np.float64(5.508786328136921) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.4494186551164603) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.508786328136921) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:808 - timing_s/gen:11.310583300888538 - timing_s/reward:0.05467207543551922 - timing_s/old_log_prob:2.336861990392208 - timing_s/adv:0.1195262223482132 - timing_s/update_actor:9.54221854172647 - timing_s/step:23.44945344515145 - timing_s/testing:31.679148972034454 - timing_s/save_checkpoint:7.0816802475601435 - timing_s/stop_profile:0.0001507047563791275 - timing_per_token_ms/gen:0.20301515446823074 - timing_per_token_ms/update_actor:0.07147889871477614 - timing_per_token_ms/adv:0.0008953476283977407 - perf/total_num_tokens:133497 - perf/time_per_step:23.44945344515145 - perf/throughput:711.6210635369982 (TaskRunner pid=2099163) Training Progress: 80%|████████ | 80/100 [1:01:51<12:26, 37.30s/it] (TaskRunner pid=2099163) step:81 - global_seqlen/min:15559 - global_seqlen/max:22877 - global_seqlen/minmax_diff:7318 - global_seqlen/balanced_min:19228 - global_seqlen/balanced_max:19441 - global_seqlen/mean:19257.0 - actor/entropy:0.2643565535545349 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2313688099384308 - training/rollout_probs_diff_mean:0.0049353111535310745 - training/rollout_probs_diff_std:0.012572631239891052 - training/rollout_actor_probs_pearson_corr:0.9985886812210083 - rollout_corr/rollout_is_mean:1.0000520944595337 - rollout_corr/rollout_is_ratio_fraction_high:1.5707464626757428e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.8178207874298096 - rollout_corr/rollout_is_min:0.5387734174728394 - rollout_corr/rollout_is_std:0.03955530375242233 - rollout_corr/rollout_is_eff_sample_size:0.9984379411292814 - rollout_corr/rollout_is_seq_mean:1.000102162361145 - rollout_corr/rollout_is_seq_std:0.0028133587911725044 - rollout_corr/rollout_is_seq_max:1.0072697401046753 - rollout_corr/rollout_is_seq_min:0.9939631819725037 - rollout_corr/rollout_is_seq_max_deviation:0.007269740104675293 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.344506647437811) - rollout_corr/training_log_ppl:np.float64(0.29159667881322093) - rollout_corr/kl:np.float64(0.0010141099861371572) - rollout_corr/k3_kl:np.float64(0.0013735205002944895) - rollout_corr/rollout_ppl:np.float64(1.3431055014953017) - rollout_corr/rollout_log_ppl:np.float64(0.2905825663037831) - rollout_corr/log_ppl_diff:np.float64(0.0010141125094378367) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028742786817019805) - rollout_corr/log_ppl_diff_max:np.float64(0.01934957504272461) - rollout_corr/log_ppl_diff_min:np.float64(-0.010468631982803345) - rollout_corr/ppl_ratio:np.float64(1.0010220303665847) - rollout_corr/chi2_token:np.float64(0.003484314540401101) - rollout_corr/chi2_seq:np.float64(0.5838833905290812) - actor/pg_loss:np.float64(0.00048545573372393847) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0023849409749345796) - actor/ppo_kl:np.float64(0.00024174882969418832) - actor/pg_clipfrac_lower:np.float64(3.98596930608619e-05) - actor/grad_norm:np.float64(0.7806772142648697) - perf/mfu/actor:np.float64(0.05648842110660632) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.96889114379883) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:5 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:3.9268659747904167e-05 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:3.9268659747904167e-05 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:248.6875 - response_length/max:1421.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:248.6875 - response_length_non_aborted/max:1421.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:353.09375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015094690024852753 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7632148675620556) - timing_s/agent_loop/generate_sequences/max:np.float64(8.421508487313986) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.538855082115333) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.421508487313986) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1421 - timing_s/gen:13.950085857883096 - timing_s/reward:0.053744787350296974 - timing_s/old_log_prob:2.4113160967826843 - timing_s/adv:0.11996482498943806 - timing_s/update_actor:9.743194399401546 - timing_s/step:26.376118445768952 - timing_s/stop_profile:4.84175980091095e-05 - timing_per_token_ms/gen:0.21912047401801796 - timing_per_token_ms/update_actor:0.0632444981007007 - timing_per_token_ms/adv:0.0007787092030783485 - perf/total_num_tokens:154056 - perf/time_per_step:26.376118445768952 - perf/throughput:730.0922628018095 (TaskRunner pid=2099163) Training Progress: 81%|████████ | 81/100 [1:02:18<10:46, 34.04s/it] (TaskRunner pid=2099163) step:82 - global_seqlen/min:13327 - global_seqlen/max:21973 - global_seqlen/minmax_diff:8646 - global_seqlen/balanced_min:17016 - global_seqlen/balanced_max:17032 - global_seqlen/mean:17027.5 - actor/entropy:0.2781171500682831 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25190842151641846 - training/rollout_probs_diff_mean:0.005356854293495417 - training/rollout_probs_diff_std:0.013084305450320244 - training/rollout_actor_probs_pearson_corr:0.9985019564628601 - rollout_corr/rollout_is_mean:0.9998621344566345 - rollout_corr/rollout_is_ratio_fraction_high:1.7285487047047354e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.642743341624737e-05 - rollout_corr/rollout_is_max:2.251864433288574 - rollout_corr/rollout_is_min:0.29135599732398987 - rollout_corr/rollout_is_std:0.0413888581097126 - rollout_corr/rollout_is_eff_sample_size:0.99828953558267 - rollout_corr/rollout_is_seq_mean:0.999849796295166 - rollout_corr/rollout_is_seq_std:0.003499638754874468 - rollout_corr/rollout_is_seq_max:1.008987307548523 - rollout_corr/rollout_is_seq_min:0.9893508553504944 - rollout_corr/rollout_is_seq_max_deviation:0.010649144649505615 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3479181840084493) - rollout_corr/training_log_ppl:np.float64(0.293532015173696) - rollout_corr/kl:np.float64(0.0012917956987497092) - rollout_corr/k3_kl:np.float64(0.0012352343883321737) - rollout_corr/rollout_ppl:np.float64(1.346091446466744) - rollout_corr/rollout_log_ppl:np.float64(0.29224021729896776) - rollout_corr/log_ppl_diff:np.float64(0.0012917978747282177) - rollout_corr/log_ppl_abs_diff:np.float64(0.0030594941636081785) - rollout_corr/log_ppl_diff_max:np.float64(0.012573987245559692) - rollout_corr/log_ppl_diff_min:np.float64(-0.011092990636825562) - rollout_corr/ppl_ratio:np.float64(1.0012995367869735) - rollout_corr/chi2_token:np.float64(0.0022806792985647917) - rollout_corr/chi2_seq:np.float64(1.2513767953496426) - actor/pg_loss:np.float64(-1.996220089495182e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011945247019866656) - actor/ppo_kl:np.float64(0.00018004048818909268) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6068511605262756) - perf/mfu/actor:np.float64(0.04981626783172659) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.0265121459961) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:5 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005667479243129492 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005667479243129492 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:225.984375 - response_length/max:982.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:225.984375 - response_length_non_aborted/max:982.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:306.125 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.120729863643646e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0436728745698929) - timing_s/agent_loop/generate_sequences/max:np.float64(6.355573097243905) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3314835890123504) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.355573097243905) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:425 - timing_s/agent_loop/slowest/response_length:982 - timing_s/gen:12.609378339722753 - timing_s/reward:0.05375535041093826 - timing_s/old_log_prob:2.3262925799936056 - timing_s/adv:0.12487000413239002 - timing_s/update_actor:9.59987291507423 - timing_s/step:24.80237107910216 - timing_s/stop_profile:0.00011940300464630127 - timing_per_token_ms/gen:0.2179592466936796 - timing_per_token_ms/update_actor:0.0704732999197932 - timing_per_token_ms/adv:0.0009166789321126855 - perf/total_num_tokens:136220 - perf/time_per_step:24.80237107910216 - perf/throughput:686.5271044326457 (TaskRunner pid=2099163) Training Progress: 82%|████████▏ | 82/100 [1:02:43<09:23, 31.28s/it] (TaskRunner pid=2099163) step:83 - global_seqlen/min:12925 - global_seqlen/max:26465 - global_seqlen/minmax_diff:13540 - global_seqlen/balanced_min:17539 - global_seqlen/balanced_max:17863 - global_seqlen/mean:17587.5 - actor/entropy:0.2657328248023987 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2070435881614685 - training/rollout_probs_diff_mean:0.005034568719565868 - training/rollout_probs_diff_std:0.012614690698683262 - training/rollout_actor_probs_pearson_corr:0.9985789060592651 - rollout_corr/rollout_is_mean:1.000186562538147 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.767050268128514e-05 - rollout_corr/rollout_is_max:1.6093506813049316 - rollout_corr/rollout_is_min:0.31631869077682495 - rollout_corr/rollout_is_std:0.03947233781218529 - rollout_corr/rollout_is_eff_sample_size:0.9984449525707558 - rollout_corr/rollout_is_seq_mean:1.0001424551010132 - rollout_corr/rollout_is_seq_std:0.0030042307917028666 - rollout_corr/rollout_is_seq_max:1.0098767280578613 - rollout_corr/rollout_is_seq_min:0.9898255467414856 - rollout_corr/rollout_is_seq_max_deviation:0.010174453258514404 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3396281930617988) - rollout_corr/training_log_ppl:np.float64(0.28824585146503523) - rollout_corr/kl:np.float64(0.0013105916907552029) - rollout_corr/k3_kl:np.float64(0.0012636836211754598) - rollout_corr/rollout_ppl:np.float64(1.3378654848784208) - rollout_corr/rollout_log_ppl:np.float64(0.2869352596462704) - rollout_corr/log_ppl_diff:np.float64(0.0013105918187648058) - rollout_corr/log_ppl_abs_diff:np.float64(0.002559717104304582) - rollout_corr/log_ppl_diff_max:np.float64(0.012187853455543518) - rollout_corr/log_ppl_diff_min:np.float64(-0.008456051349639893) - rollout_corr/ppl_ratio:np.float64(1.0013164200354367) - rollout_corr/chi2_token:np.float64(0.0023376578465104103) - rollout_corr/chi2_seq:np.float64(0.27040194789879024) - actor/pg_loss:np.float64(0.00010856811422854662) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011537321538526157) - actor/ppo_kl:np.float64(0.0005853857873887591) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6243516802787781) - perf/mfu/actor:np.float64(0.05163460793549127) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.9116096496582) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:5 - critic/score/mean:0.5234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0019763396121561527 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0019763396121561527 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:245.828125 - response_length/max:1487.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:245.828125 - response_length_non_aborted/max:1487.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:303.78125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.453855454921722e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1134148389101028) - timing_s/agent_loop/generate_sequences/max:np.float64(8.951366694644094) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.531380183805595) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.951366694644094) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:415 - timing_s/agent_loop/slowest/response_length:1487 - timing_s/gen:14.601487068459392 - timing_s/reward:0.0553099662065506 - timing_s/old_log_prob:2.3057028613984585 - timing_s/adv:0.11853219196200371 - timing_s/update_actor:9.658092549070716 - timing_s/step:26.824403259903193 - timing_s/stop_profile:4.142150282859802e-05 - timing_per_token_ms/gen:0.23202007036896002 - timing_per_token_ms/update_actor:0.0686431595527414 - timing_per_token_ms/adv:0.0008424462826013057 - perf/total_num_tokens:140700 - perf/time_per_step:26.824403259903193 - perf/throughput:655.6529824575666 (TaskRunner pid=2099163) Training Progress: 83%|████████▎ | 83/100 [1:03:09<08:29, 29.95s/it] (TaskRunner pid=2099163) step:84 - global_seqlen/min:14425 - global_seqlen/max:25507 - global_seqlen/minmax_diff:11082 - global_seqlen/balanced_min:19094 - global_seqlen/balanced_max:19099 - global_seqlen/mean:19098.0 - actor/entropy:0.29119229316711426 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3042685389518738 - training/rollout_probs_diff_mean:0.005367741920053959 - training/rollout_probs_diff_std:0.012942570261657238 - training/rollout_actor_probs_pearson_corr:0.9985758066177368 - rollout_corr/rollout_is_mean:1.0001381635665894 - rollout_corr/rollout_is_ratio_fraction_high:1.4721469597134273e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.9442939194268547e-05 - rollout_corr/rollout_is_max:2.259265661239624 - rollout_corr/rollout_is_min:0.32297244668006897 - rollout_corr/rollout_is_std:0.04044489935040474 - rollout_corr/rollout_is_eff_sample_size:0.9983673568485679 - rollout_corr/rollout_is_seq_mean:1.0002832412719727 - rollout_corr/rollout_is_seq_std:0.0029685506597161293 - rollout_corr/rollout_is_seq_max:1.0090413093566895 - rollout_corr/rollout_is_seq_min:0.9883840680122375 - rollout_corr/rollout_is_seq_max_deviation:0.011615931987762451 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3692081668414176) - rollout_corr/training_log_ppl:np.float64(0.31009084297693335) - rollout_corr/kl:np.float64(0.0009806591620620253) - rollout_corr/k3_kl:np.float64(0.0011354790972291084) - rollout_corr/rollout_ppl:np.float64(1.367792998906225) - rollout_corr/rollout_log_ppl:np.float64(0.30911018152255565) - rollout_corr/log_ppl_diff:np.float64(0.000980661454377696) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027504815079737455) - rollout_corr/log_ppl_diff_max:np.float64(0.018712282180786133) - rollout_corr/log_ppl_diff_min:np.float64(-0.009203359484672546) - rollout_corr/ppl_ratio:np.float64(1.0009879535064101) - rollout_corr/chi2_token:np.float64(0.0026025581173598766) - rollout_corr/chi2_seq:np.float64(1.0249379503075033) - actor/pg_loss:np.float64(0.0003060917370021343) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015142915781325428) - actor/ppo_kl:np.float64(0.00034652300968218697) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5972811579704285) - perf/mfu/actor:np.float64(0.05583602661339712) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.96282958984375) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:5 - critic/score/mean:0.4296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0036454040091484785 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0036454040091484785 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:265.34375 - response_length/max:926.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:265.34375 - response_length_non_aborted/max:926.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:331.46875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.8295442461967468e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1363158952444792) - timing_s/agent_loop/generate_sequences/max:np.float64(6.200030520558357) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.6375616744408035) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.200030520558357) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:926 - timing_s/gen:12.048710951581597 - timing_s/reward:0.057196732610464096 - timing_s/old_log_prob:2.329210080206394 - timing_s/adv:0.11962089873850346 - timing_s/update_actor:9.724592242389917 - timing_s/step:24.364509677514434 - timing_s/stop_profile:3.784149885177612e-05 - timing_per_token_ms/gen:0.1773747343007537 - timing_per_token_ms/update_actor:0.0636492842338852 - timing_per_token_ms/adv:0.0007829412683167311 - perf/total_num_tokens:152784 - perf/time_per_step:24.364509677514434 - perf/throughput:783.8450374244633 (TaskRunner pid=2099163) Training Progress: 84%|████████▍ | 84/100 [1:03:34<07:32, 28.28s/it] (TaskRunner pid=2099163) step:85 - global_seqlen/min:13201 - global_seqlen/max:23013 - global_seqlen/minmax_diff:9812 - global_seqlen/balanced_min:17098 - global_seqlen/balanced_max:17114 - global_seqlen/mean:17108.75 - actor/entropy:0.3020118772983551 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4086613655090332 - training/rollout_probs_diff_mean:0.005688806995749474 - training/rollout_probs_diff_std:0.01348093617707491 - training/rollout_actor_probs_pearson_corr:0.9984976053237915 - rollout_corr/rollout_is_mean:0.9999252557754517 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.976279888069257e-05 - rollout_corr/rollout_is_max:1.660659909248352 - rollout_corr/rollout_is_min:0.12563924491405487 - rollout_corr/rollout_is_std:0.04169733077287674 - rollout_corr/rollout_is_eff_sample_size:0.9982643502297281 - rollout_corr/rollout_is_seq_mean:0.9999344944953918 - rollout_corr/rollout_is_seq_std:0.0032068761065602303 - rollout_corr/rollout_is_seq_max:1.0129774808883667 - rollout_corr/rollout_is_seq_min:0.99162358045578 - rollout_corr/rollout_is_seq_max_deviation:0.0129774808883667 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3707473212853074) - rollout_corr/training_log_ppl:np.float64(0.3106351788446773) - rollout_corr/kl:np.float64(0.0010191974550188831) - rollout_corr/k3_kl:np.float64(0.0012126699965619991) - rollout_corr/rollout_ppl:np.float64(1.3693385659717023) - rollout_corr/rollout_log_ppl:np.float64(0.3096159797569271) - rollout_corr/log_ppl_diff:np.float64(0.0010191990877501667) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027312866295687854) - rollout_corr/log_ppl_diff_max:np.float64(0.01148831844329834) - rollout_corr/log_ppl_diff_min:np.float64(-0.009180232882499695) - rollout_corr/ppl_ratio:np.float64(1.001025541452691) - rollout_corr/chi2_token:np.float64(0.002835497260093689) - rollout_corr/chi2_seq:np.float64(0.9057672906201333) - actor/pg_loss:np.float64(-5.922233685851097e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001044121142513177) - actor/ppo_kl:np.float64(4.945502578124206e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4638519212603569) - perf/mfu/actor:np.float64(0.050201444400562734) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.93396759033203) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:6 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006879706867039204 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006879706867039204 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:235.4921875 - response_length/max:1005.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:235.4921875 - response_length_non_aborted/max:1005.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.15625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0004599113017320633 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2714347038418055) - timing_s/agent_loop/generate_sequences/max:np.float64(6.474359966814518) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.499616449684254) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.474359966814518) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:1005 - timing_s/gen:12.510914074257016 - timing_s/reward:0.04957201145589352 - timing_s/old_log_prob:2.413747824728489 - timing_s/adv:0.13123666122555733 - timing_s/update_actor:9.761451695114374 - timing_s/step:24.98761263117194 - timing_s/stop_profile:0.0001248195767402649 - timing_per_token_ms/gen:0.20752602717475063 - timing_per_token_ms/update_actor:0.07131914733041846 - timing_per_token_ms/adv:0.0009588416835358904 - perf/total_num_tokens:136870 - perf/time_per_step:24.98761263117194 - perf/throughput:684.6892599358174 (TaskRunner pid=2099163) Training Progress: 85%|████████▌ | 85/100 [1:03:59<06:51, 27.44s/it] (TaskRunner pid=2099163) step:86 - global_seqlen/min:12509 - global_seqlen/max:25462 - global_seqlen/minmax_diff:12953 - global_seqlen/balanced_min:18292 - global_seqlen/balanced_max:18296 - global_seqlen/mean:18294.125 - actor/entropy:0.2849932610988617 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35150495171546936 - training/rollout_probs_diff_mean:0.0054772403091192245 - training/rollout_probs_diff_std:0.013425457291305065 - training/rollout_actor_probs_pearson_corr:0.9984551668167114 - rollout_corr/rollout_is_mean:1.0001648664474487 - rollout_corr/rollout_is_ratio_fraction_high:4.6187244151951745e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.0791496101301163e-05 - rollout_corr/rollout_is_max:2.974616289138794 - rollout_corr/rollout_is_min:0.459360271692276 - rollout_corr/rollout_is_std:0.04225114360451698 - rollout_corr/rollout_is_eff_sample_size:0.9982186159276442 - rollout_corr/rollout_is_seq_mean:1.0002799034118652 - rollout_corr/rollout_is_seq_std:0.003540112404152751 - rollout_corr/rollout_is_seq_max:1.0164246559143066 - rollout_corr/rollout_is_seq_min:0.9909144639968872 - rollout_corr/rollout_is_seq_max_deviation:0.01642465591430664 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3678325335495174) - rollout_corr/training_log_ppl:np.float64(0.30802994256373495) - rollout_corr/kl:np.float64(0.0011918487436304304) - rollout_corr/k3_kl:np.float64(0.0014298067977733808) - rollout_corr/rollout_ppl:np.float64(1.366152125876397) - rollout_corr/rollout_log_ppl:np.float64(0.30683809251058847) - rollout_corr/log_ppl_diff:np.float64(0.0011918500531464815) - rollout_corr/log_ppl_abs_diff:np.float64(0.003217801859136671) - rollout_corr/log_ppl_diff_max:np.float64(0.010454237461090088) - rollout_corr/log_ppl_diff_min:np.float64(-0.01674443483352661) - rollout_corr/ppl_ratio:np.float64(1.0012002452276647) - rollout_corr/chi2_token:np.float64(0.003339178394526243) - rollout_corr/chi2_seq:np.float64(1.9388157108332962) - actor/pg_loss:np.float64(-0.0001976813655346632) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001634979141272197) - actor/ppo_kl:np.float64(0.0004851831923424754) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5677638277411461) - perf/mfu/actor:np.float64(0.05395100018655371) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.07014083862305) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:6 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019225439755246043 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0019225439755246043 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:253.72265625 - response_length/max:792.0 - response_length/min:116.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:253.72265625 - response_length_non_aborted/max:792.0 - response_length_non_aborted/min:116.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:317.96875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015383213758468628 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4205127842724323) - timing_s/agent_loop/generate_sequences/max:np.float64(5.750876966863871) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.6700041760486783) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.750876966863871) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:432 - timing_s/agent_loop/slowest/response_length:787 - timing_s/gen:11.532334372401237 - timing_s/reward:0.0561569519340992 - timing_s/old_log_prob:2.331783862784505 - timing_s/adv:0.12503085657954216 - timing_s/update_actor:9.721932409331203 - timing_s/step:23.852430075407028 - timing_s/stop_profile:0.00010659173130989075 - timing_per_token_ms/gen:0.17754891032594702 - timing_per_token_ms/update_actor:0.0664279680589479 - timing_per_token_ms/adv:0.0008543101718416579 - perf/total_num_tokens:146353 - perf/time_per_step:23.852430075407028 - perf/throughput:766.9711195951518 (TaskRunner pid=2099163) Training Progress: 86%|████████▌ | 86/100 [1:04:23<06:09, 26.38s/it] (TaskRunner pid=2099163) step:87 - global_seqlen/min:13933 - global_seqlen/max:20602 - global_seqlen/minmax_diff:6669 - global_seqlen/balanced_min:16687 - global_seqlen/balanced_max:16691 - global_seqlen/mean:16689.75 - actor/entropy:0.28263986110687256 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105812072753906 - training/rollout_probs_diff_mean:0.0056616635993123055 - training/rollout_probs_diff_std:0.013550084084272385 - training/rollout_actor_probs_pearson_corr:0.9984059929847717 - rollout_corr/rollout_is_mean:1.0001320838928223 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.4052106861490756e-05 - rollout_corr/rollout_is_max:1.8633671998977661 - rollout_corr/rollout_is_min:0.20320795476436615 - rollout_corr/rollout_is_std:0.04268762469291687 - rollout_corr/rollout_is_eff_sample_size:0.998181556398287 - rollout_corr/rollout_is_seq_mean:1.000098466873169 - rollout_corr/rollout_is_seq_std:0.00287253619171679 - rollout_corr/rollout_is_seq_max:1.0077333450317383 - rollout_corr/rollout_is_seq_min:0.9905746579170227 - rollout_corr/rollout_is_seq_max_deviation:0.009425342082977295 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3366769887506962) - rollout_corr/training_log_ppl:np.float64(0.2858038642152678) - rollout_corr/kl:np.float64(0.0010864866192696887) - rollout_corr/k3_kl:np.float64(0.001194243301597453) - rollout_corr/rollout_ppl:np.float64(1.3351997244171798) - rollout_corr/rollout_log_ppl:np.float64(0.28471737628569826) - rollout_corr/log_ppl_diff:np.float64(0.0010864879295695573) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029874266765546054) - rollout_corr/log_ppl_diff_max:np.float64(0.014816895127296448) - rollout_corr/log_ppl_diff_min:np.float64(-0.01041361689567566) - rollout_corr/ppl_ratio:np.float64(1.0010941252112389) - rollout_corr/chi2_token:np.float64(0.002591905649751425) - rollout_corr/chi2_seq:np.float64(0.8676178343594074) - actor/pg_loss:np.float64(2.4948501959443092e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008350495745617081) - actor/ppo_kl:np.float64(0.00021582131192943166) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.45856113731861115) - perf/mfu/actor:np.float64(0.04922829341564032) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.0233154296875) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:6 - critic/score/mean:0.5234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.013735991902649403 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.013735991902649403 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:216.8046875 - response_length/max:808.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:216.8046875 - response_length_non_aborted/max:808.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:304.75 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.88928771018982e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.328505938872695) - timing_s/agent_loop/generate_sequences/max:np.float64(5.489037185907364) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3260599871064187) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.489037185907364) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:435 - timing_s/agent_loop/slowest/response_length:808 - timing_s/gen:11.125011002644897 - timing_s/reward:0.04664120823144913 - timing_s/old_log_prob:2.281039910390973 - timing_s/adv:0.12088093161582947 - timing_s/update_actor:9.694726234301925 - timing_s/step:23.32410356402397 - timing_s/stop_profile:0.00012058950960636139 - timing_per_token_ms/gen:0.20044342550979957 - timing_per_token_ms/update_actor:0.07260988207059665 - timing_per_token_ms/adv:0.0009053530731124602 - perf/total_num_tokens:133518 - perf/time_per_step:23.32410356402397 - perf/throughput:715.5580472444367 (TaskRunner pid=2099163) Training Progress: 87%|████████▋ | 87/100 [1:04:47<05:31, 25.48s/it] (TaskRunner pid=2099163) step:88 - global_seqlen/min:13016 - global_seqlen/max:21866 - global_seqlen/minmax_diff:8850 - global_seqlen/balanced_min:18267 - global_seqlen/balanced_max:18275 - global_seqlen/mean:18273.375 - actor/entropy:0.2799515724182129 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5366853475570679 - training/rollout_probs_diff_mean:0.005581203382462263 - training/rollout_probs_diff_std:0.013881382532417774 - training/rollout_actor_probs_pearson_corr:0.9983412027359009 - rollout_corr/rollout_is_mean:1.0000699758529663 - rollout_corr/rollout_is_ratio_fraction_high:1.5643821825506166e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.128764365101233e-05 - rollout_corr/rollout_is_max:2.0318520069122314 - rollout_corr/rollout_is_min:0.3905743956565857 - rollout_corr/rollout_is_std:0.042155105620622635 - rollout_corr/rollout_is_eff_sample_size:0.998226337009296 - rollout_corr/rollout_is_seq_mean:1.0000312328338623 - rollout_corr/rollout_is_seq_std:0.0031614850740879774 - rollout_corr/rollout_is_seq_max:1.0118353366851807 - rollout_corr/rollout_is_seq_min:0.9903686046600342 - rollout_corr/rollout_is_seq_max_deviation:0.011835336685180664 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.347297566011548) - rollout_corr/training_log_ppl:np.float64(0.29373508717981167) - rollout_corr/kl:np.float64(0.0011751465276006456) - rollout_corr/k3_kl:np.float64(0.001120794104167544) - rollout_corr/rollout_ppl:np.float64(1.3456657594069839) - rollout_corr/rollout_log_ppl:np.float64(0.29255994062987156) - rollout_corr/log_ppl_diff:np.float64(0.0011751465499401093) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026922697434201837) - rollout_corr/log_ppl_diff_max:np.float64(0.011649519205093384) - rollout_corr/log_ppl_diff_min:np.float64(-0.010374203324317932) - rollout_corr/ppl_ratio:np.float64(1.001181423664093) - rollout_corr/chi2_token:np.float64(0.002159958705306053) - rollout_corr/chi2_seq:np.float64(1.1241237167268991) - actor/pg_loss:np.float64(0.00016208365559577942) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000806990730779944) - actor/ppo_kl:np.float64(0.00024305590492046392) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4049631729722023) - perf/mfu/actor:np.float64(0.05369724660460473) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.09536361694336) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:6 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01436884980648756 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01436884980648756 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:249.69921875 - response_length/max:838.0 - response_length/min:109.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:249.69921875 - response_length_non_aborted/max:838.0 - response_length_non_aborted/min:109.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:321.34375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014184601604938507 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1800287831574678) - timing_s/agent_loop/generate_sequences/max:np.float64(5.7370752189308405) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.6392609360191273) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.7370752189308405) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:838 - timing_s/gen:11.481335692107677 - timing_s/reward:0.0557560957968235 - timing_s/old_log_prob:2.273548835888505 - timing_s/adv:0.11617436446249485 - timing_s/update_actor:9.607393434271216 - timing_s/step:23.618814580142498 - timing_s/stop_profile:5.055032670497894e-05 - timing_per_token_ms/gen:0.1796119658355784 - timing_per_token_ms/update_actor:0.06571988914384463 - timing_per_token_ms/adv:0.0007946969598014519 - perf/total_num_tokens:146187 - perf/time_per_step:23.618814580142498 - perf/throughput:773.6787525045108 (TaskRunner pid=2099163) Training Progress: 88%|████████▊ | 88/100 [1:05:10<04:59, 24.94s/it] (TaskRunner pid=2099163) step:89 - global_seqlen/min:15884 - global_seqlen/max:25708 - global_seqlen/minmax_diff:9824 - global_seqlen/balanced_min:19388 - global_seqlen/balanced_max:19400 - global_seqlen/mean:19395.0 - actor/entropy:0.2626587748527527 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9052478075027466 - training/rollout_probs_diff_mean:0.00535238441079855 - training/rollout_probs_diff_std:0.013822822831571102 - training/rollout_actor_probs_pearson_corr:0.9982951879501343 - rollout_corr/rollout_is_mean:0.9997128248214722 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.095330900279805e-05 - rollout_corr/rollout_is_max:1.9251736402511597 - rollout_corr/rollout_is_min:0.07634969055652618 - rollout_corr/rollout_is_std:0.04105702042579651 - rollout_corr/rollout_is_eff_sample_size:0.9983161479024087 - rollout_corr/rollout_is_seq_mean:0.9995173215866089 - rollout_corr/rollout_is_seq_std:0.0029820753261446953 - rollout_corr/rollout_is_seq_max:1.0083783864974976 - rollout_corr/rollout_is_seq_min:0.9902425408363342 - rollout_corr/rollout_is_seq_max_deviation:0.009757459163665771 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3277254402637482) - rollout_corr/training_log_ppl:np.float64(0.2786243225564249) - rollout_corr/kl:np.float64(0.0015448006739369191) - rollout_corr/k3_kl:np.float64(0.0011937761133822278) - rollout_corr/rollout_ppl:np.float64(1.3255531983450055) - rollout_corr/rollout_log_ppl:np.float64(0.2770795207761694) - rollout_corr/log_ppl_diff:np.float64(0.0015448017802555114) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028917556919623166) - rollout_corr/log_ppl_diff_max:np.float64(0.015002906322479248) - rollout_corr/log_ppl_diff_min:np.float64(-0.009606748819351196) - rollout_corr/ppl_ratio:np.float64(1.0015520232263952) - rollout_corr/chi2_token:np.float64(0.001597132533788681) - rollout_corr/chi2_seq:np.float64(0.5050114756450057) - actor/pg_loss:np.float64(0.0002184249460697174) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010780754209918086) - actor/ppo_kl:np.float64(0.00011659164282207257) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.43044082075357437) - perf/mfu/actor:np.float64(0.056737380618191155) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.99333953857422) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:6 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013847829774022102 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013847829774022102 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:256.34375 - response_length/max:944.0 - response_length/min:121.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:256.34375 - response_length_non_aborted/max:944.0 - response_length_non_aborted/min:121.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:349.75 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.797747194766998e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5112595465034246) - timing_s/agent_loop/generate_sequences/max:np.float64(6.177097799256444) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.6719028287552646) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.177097799256444) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:944 - timing_s/gen:11.939295399934053 - timing_s/reward:0.054529689252376556 - timing_s/old_log_prob:2.3607167918235064 - timing_s/adv:0.12182499840855598 - timing_s/update_actor:9.700234856456518 - timing_s/step:24.254260797053576 - timing_s/stop_profile:0.00011785514652729034 - timing_per_token_ms/gen:0.1819348927211699 - timing_per_token_ms/update_actor:0.06251762604058081 - timing_per_token_ms/adv:0.0007851572467682134 - perf/total_num_tokens:155160 - perf/time_per_step:24.254260797053576 - perf/throughput:799.6533129699058 (TaskRunner pid=2099163) Training Progress: 89%|████████▉ | 89/100 [1:05:35<04:32, 24.74s/it] (TaskRunner pid=2099163) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2099163) validation generation end (TaskRunner pid=2099163) [prompt] system (TaskRunner pid=2099163) (TaskRunner pid=2099163) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) user (TaskRunner pid=2099163) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2099163) (TaskRunner pid=2099163) A: -0.01 (TaskRunner pid=2099163) B: 1.69 (TaskRunner pid=2099163) C: 2.49 (TaskRunner pid=2099163) D: 0.45 (TaskRunner pid=2099163) Please reason step by step. (TaskRunner pid=2099163) assistant (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) [response] (TaskRunner pid=2099163) The folding stability score for a protein sequence can be determined using computational tools such as I-TAFOR, DSSP, or other protein structure prediction software. These tools calculate the stability of a protein's folded structure based on the sequence and predicted three-dimensional structure. The score provided in the question, "-0.01", is likely derived from such computational analysis. Scores close to zero indicate that the protein sequence has a stable fold, while positive scores may indicate higher stability. Based on the given sequence and typical computational results, the folding stability score is most likely -0.01. (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) [ground_truth] B (TaskRunner pid=2099163) [score] 0.0 (TaskRunner pid=2099163) [acc] 0.0 (TaskRunner pid=2099163) [pred] A (TaskRunner pid=2099163) [incorrect_format] 1 (TaskRunner pid=2099163) [feedback] (TaskRunner pid=2099163) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (WorkerDict pid=2099539) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80/actor (TaskRunner pid=2099163) step:90 - global_seqlen/min:15076 - global_seqlen/max:22705 - global_seqlen/minmax_diff:7629 - global_seqlen/balanced_min:18281 - global_seqlen/balanced_max:18304 - global_seqlen/mean:18298.25 - actor/entropy:0.25007060170173645 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3519518971443176 - training/rollout_probs_diff_mean:0.005134246312081814 - training/rollout_probs_diff_std:0.013177147135138512 - training/rollout_actor_probs_pearson_corr:0.998388409614563 - rollout_corr/rollout_is_mean:0.9999549388885498 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.610581815242767e-05 - rollout_corr/rollout_is_max:1.6880807876586914 - rollout_corr/rollout_is_min:0.44652095437049866 - rollout_corr/rollout_is_std:0.04015500843524933 - rollout_corr/rollout_is_eff_sample_size:0.9983898144007888 - rollout_corr/rollout_is_seq_mean:0.9998966455459595 - rollout_corr/rollout_is_seq_std:0.0029496627394109964 - rollout_corr/rollout_is_seq_max:1.0089014768600464 - rollout_corr/rollout_is_seq_min:0.9883391261100769 - rollout_corr/rollout_is_seq_max_deviation:0.011660873889923096 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.315980046056211) - rollout_corr/training_log_ppl:np.float64(0.2705623500805814) - rollout_corr/kl:np.float64(0.0011479333818513737) - rollout_corr/k3_kl:np.float64(0.0010572865134577114) - rollout_corr/rollout_ppl:np.float64(1.314408981706947) - rollout_corr/rollout_log_ppl:np.float64(0.26941441488452256) - rollout_corr/log_ppl_diff:np.float64(0.0011479351960588247) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027164410275872797) - rollout_corr/log_ppl_diff_max:np.float64(0.013363391160964966) - rollout_corr/log_ppl_diff_min:np.float64(-0.008001953363418579) - rollout_corr/ppl_ratio:np.float64(1.0011545014567673) - rollout_corr/chi2_token:np.float64(0.0019355297554284334) - rollout_corr/chi2_seq:np.float64(0.5621733826119453) - actor/pg_loss:np.float64(-3.921461757272482e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012568379429467313) - actor/ppo_kl:np.float64(0.00014396155199847271) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4547141119837761) - perf/mfu/actor:np.float64(0.05313581446106291) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.05733489990234) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.4675) - val-aux/sciknoweval/reward/std@16:np.float64(0.28097887193097565) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.59294) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.22754928387157947) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.34464) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2509343043440332) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.4650000000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.28133902959180757) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6816800000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.14720983664825582) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.23678000000000002) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.17523408016078515) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.48656) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.24240246066242588) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.73372) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08196201028513199) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.17182000000000003) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09563785531224381) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.5071399999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.19410742505337317) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7635599999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04110937154135006) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.14181999999999997) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04166172799618433) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.52198) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.15012735298954785) - val-aux/sciknoweval/score/mean@16:np.float64(0.4675) - val-aux/sciknoweval/score/std@16:np.float64(0.28097887193097565) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.59294) - val-aux/sciknoweval/score/best@2/std:np.float64(0.22754928387157947) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.34464) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.2509343043440332) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.4650000000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.28133902959180757) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6816800000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.14720983664825582) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.23678000000000002) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.17523408016078515) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.48656) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.24240246066242588) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.73372) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08196201028513199) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.17182000000000003) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09563785531224381) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.5071399999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.19410742505337317) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7635599999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04110937154135006) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.14181999999999997) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04166172799618433) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.52198) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.15012735298954785) - val-core/sciknoweval/acc/mean@16:np.float64(0.4675) - val-aux/sciknoweval/acc/std@16:np.float64(0.28097887193097565) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.59294) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.22754928387157947) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.34464) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.2509343043440332) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.4650000000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.28133902959180757) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6816800000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.14720983664825582) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.23678000000000002) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.17523408016078515) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.48656) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.24240246066242588) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.73372) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08196201028513199) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.17182000000000003) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09563785531224381) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.5071399999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.19410742505337317) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7635599999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04110937154135006) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.14181999999999997) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04166172799618433) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.52198) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.15012735298954785) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:6 - critic/score/mean:0.51953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007901686243712902 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007901686243712902 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:256.6328125 - response_length/max:1166.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:256.6328125 - response_length_non_aborted/max:1166.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.1875 - prompt_length/max:499.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.221715688705444e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1986623331904411) - timing_s/agent_loop/generate_sequences/max:np.float64(7.395749419927597) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.6543824151231092) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.395749419927597) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:1166 - timing_s/gen:13.268566366285086 - timing_s/reward:0.05747130326926708 - timing_s/old_log_prob:2.363747773692012 - timing_s/adv:0.1281336061656475 - timing_s/update_actor:9.728703785687685 - timing_s/step:25.604432402178645 - timing_s/testing:25.468937627971172 - timing_s/save_checkpoint:6.738213421776891 - timing_s/stop_profile:3.814697265625e-05 - timing_per_token_ms/gen:0.20196301814796624 - timing_per_token_ms/update_actor:0.06645925010375094 - timing_per_token_ms/adv:0.0008753132551312797 - perf/total_num_tokens:146386 - perf/time_per_step:25.604432402178645 - perf/throughput:714.6516553299197 (TaskRunner pid=2099163) Training Progress: 90%|█████████ | 90/100 [1:06:32<05:46, 34.67s/it] (TaskRunner pid=2099163) step:91 - global_seqlen/min:13664 - global_seqlen/max:28551 - global_seqlen/minmax_diff:14887 - global_seqlen/balanced_min:19769 - global_seqlen/balanced_max:19785 - global_seqlen/mean:19781.5 - actor/entropy:0.25078386068344116 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43638211488723755 - training/rollout_probs_diff_mean:0.004951483104377985 - training/rollout_probs_diff_std:0.013005034998059273 - training/rollout_actor_probs_pearson_corr:0.9984056949615479 - rollout_corr/rollout_is_mean:0.9997699856758118 - rollout_corr/rollout_is_ratio_fraction_high:2.716062772378791e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.432125544757582e-05 - rollout_corr/rollout_is_max:2.0405869483947754 - rollout_corr/rollout_is_min:0.43364647030830383 - rollout_corr/rollout_is_std:0.0399608314037323 - rollout_corr/rollout_is_eff_sample_size:0.9984049054928523 - rollout_corr/rollout_is_seq_mean:0.9997459053993225 - rollout_corr/rollout_is_seq_std:0.002915602643042803 - rollout_corr/rollout_is_seq_max:1.009341835975647 - rollout_corr/rollout_is_seq_min:0.9896574020385742 - rollout_corr/rollout_is_seq_max_deviation:0.010342597961425781 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3241292075254023) - rollout_corr/training_log_ppl:np.float64(0.2761981814110186) - rollout_corr/kl:np.float64(0.0012420743942662682) - rollout_corr/k3_kl:np.float64(0.0012305704847221932) - rollout_corr/rollout_ppl:np.float64(1.3224302087910473) - rollout_corr/rollout_log_ppl:np.float64(0.274956106179161) - rollout_corr/log_ppl_diff:np.float64(0.0012420752318575978) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025912187120411545) - rollout_corr/log_ppl_diff_max:np.float64(0.014826744794845581) - rollout_corr/log_ppl_diff_min:np.float64(-0.007786452770233154) - rollout_corr/ppl_ratio:np.float64(1.0012476784177125) - rollout_corr/chi2_token:np.float64(0.0025545565877109766) - rollout_corr/chi2_seq:np.float64(0.8443398529198021) - actor/pg_loss:np.float64(-0.00023542286362498999) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013123745520715602) - actor/ppo_kl:np.float64(7.698053936522342e-05) - actor/pg_clipfrac_lower:np.float64(1.1660447853500955e-05) - actor/grad_norm:np.float64(0.4663517475128174) - perf/mfu/actor:np.float64(0.05740788145887977) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.15924453735352) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:6 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0005907436716370285 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0005907436716370285 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:287.640625 - response_length/max:1198.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:287.640625 - response_length_non_aborted/max:1198.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.53125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.8155744075775146e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3114315513521433) - timing_s/agent_loop/generate_sequences/max:np.float64(7.799869127571583) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8764019390873727) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.799869127571583) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:1198 - timing_s/gen:13.566020984202623 - timing_s/reward:0.05244571715593338 - timing_s/old_log_prob:2.490916972979903 - timing_s/adv:0.13142306357622147 - timing_s/update_actor:9.768587896600366 - timing_s/step:26.121974842622876 - timing_s/stop_profile:0.00011107511818408966 - timing_per_token_ms/gen:0.18423082438213134 - timing_per_token_ms/update_actor:0.06172805333645304 - timing_per_token_ms/adv:0.0008304669993189436 - perf/total_num_tokens:158252 - perf/time_per_step:26.121974842622876 - perf/throughput:757.2742918243223 (TaskRunner pid=2099163) Training Progress: 91%|█████████ | 91/100 [1:06:59<04:49, 32.12s/it] (TaskRunner pid=2099163) step:92 - global_seqlen/min:15474 - global_seqlen/max:25697 - global_seqlen/minmax_diff:10223 - global_seqlen/balanced_min:20472 - global_seqlen/balanced_max:20489 - global_seqlen/mean:20485.25 - actor/entropy:0.25056755542755127 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5584663152694702 - training/rollout_probs_diff_mean:0.004964837804436684 - training/rollout_probs_diff_std:0.013341370038688183 - training/rollout_actor_probs_pearson_corr:0.9983838200569153 - rollout_corr/rollout_is_mean:0.9999274015426636 - rollout_corr/rollout_is_ratio_fraction_high:4.238126348354854e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.650835009873845e-05 - rollout_corr/rollout_is_max:3.1624722480773926 - rollout_corr/rollout_is_min:0.20737510919570923 - rollout_corr/rollout_is_std:0.0406433641910553 - rollout_corr/rollout_is_eff_sample_size:0.9983503658269968 - rollout_corr/rollout_is_seq_mean:0.9998391270637512 - rollout_corr/rollout_is_seq_std:0.0029326851945370436 - rollout_corr/rollout_is_seq_max:1.0081546306610107 - rollout_corr/rollout_is_seq_min:0.9876515865325928 - rollout_corr/rollout_is_seq_max_deviation:0.012348413467407227 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3244820055551827) - rollout_corr/training_log_ppl:np.float64(0.27609299635514617) - rollout_corr/kl:np.float64(0.0011841020946801173) - rollout_corr/k3_kl:np.float64(0.0013603362282310627) - rollout_corr/rollout_ppl:np.float64(1.3228559507988393) - rollout_corr/rollout_log_ppl:np.float64(0.2749088934651809) - rollout_corr/log_ppl_diff:np.float64(0.0011841028899652883) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026790284464368597) - rollout_corr/log_ppl_diff_max:np.float64(0.01207616925239563) - rollout_corr/log_ppl_diff_min:np.float64(-0.0074624717235565186) - rollout_corr/ppl_ratio:np.float64(1.0011904705315828) - rollout_corr/chi2_token:np.float64(0.0030524577014148235) - rollout_corr/chi2_seq:np.float64(0.5289049397688359) - actor/pg_loss:np.float64(0.00013589137233793736) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002009198718951666) - actor/ppo_kl:np.float64(3.7602182644036475e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.613745704293251) - perf/mfu/actor:np.float64(0.05991881909984166) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.0025405883789) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:6 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012523663230240345 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012523663230240345 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:276.5078125 - response_length/max:1210.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:276.5078125 - response_length_non_aborted/max:1210.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:363.65625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001337248831987381 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3419308010488749) - timing_s/agent_loop/generate_sequences/max:np.float64(7.376888804137707) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.7873674428483355) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.376888804137707) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:425 - timing_s/agent_loop/slowest/response_length:1210 - timing_s/gen:13.59444515965879 - timing_s/reward:0.05653318390250206 - timing_s/old_log_prob:2.394727673381567 - timing_s/adv:0.11553080752491951 - timing_s/update_actor:9.629633232951164 - timing_s/step:25.88193485327065 - timing_s/stop_profile:0.00010392814874649048 - timing_per_token_ms/gen:0.1920499132548638 - timing_per_token_ms/update_actor:0.05875955402637974 - timing_per_token_ms/adv:0.0007049633731887548 - perf/total_num_tokens:163882 - perf/time_per_step:25.88193485327065 - perf/throughput:791.4883534069061 (TaskRunner pid=2099163) Training Progress: 92%|█████████▏| 92/100 [1:07:25<04:02, 30.27s/it] (TaskRunner pid=2099163) step:93 - global_seqlen/min:13545 - global_seqlen/max:25655 - global_seqlen/minmax_diff:12110 - global_seqlen/balanced_min:19112 - global_seqlen/balanced_max:19881 - global_seqlen/mean:19214.125 - actor/entropy:0.23102349042892456 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3046841025352478 - training/rollout_probs_diff_mean:0.004829892423003912 - training/rollout_probs_diff_std:0.013052316382527351 - training/rollout_actor_probs_pearson_corr:0.9983353018760681 - rollout_corr/rollout_is_mean:0.9999238848686218 - rollout_corr/rollout_is_ratio_fraction_high:1.537870048196055e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.689350604778156e-05 - rollout_corr/rollout_is_max:2.5575833320617676 - rollout_corr/rollout_is_min:0.3091418743133545 - rollout_corr/rollout_is_std:0.03926188126206398 - rollout_corr/rollout_is_eff_sample_size:0.9984606395421983 - rollout_corr/rollout_is_seq_mean:0.9999052882194519 - rollout_corr/rollout_is_seq_std:0.003070805687457323 - rollout_corr/rollout_is_seq_max:1.0085487365722656 - rollout_corr/rollout_is_seq_min:0.9925494194030762 - rollout_corr/rollout_is_seq_max_deviation:0.008548736572265625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.298731068149209) - rollout_corr/training_log_ppl:np.float64(0.2568639310920844) - rollout_corr/kl:np.float64(0.001255048227367439) - rollout_corr/k3_kl:np.float64(0.001259192493506589) - rollout_corr/rollout_ppl:np.float64(1.297051596455276) - rollout_corr/rollout_log_ppl:np.float64(0.25560888355539646) - rollout_corr/log_ppl_diff:np.float64(0.0012550475366879255) - rollout_corr/log_ppl_abs_diff:np.float64(0.002867596718715504) - rollout_corr/log_ppl_diff_max:np.float64(0.02118036150932312) - rollout_corr/log_ppl_diff_min:np.float64(-0.007840782403945923) - rollout_corr/ppl_ratio:np.float64(1.0012623802758753) - rollout_corr/chi2_token:np.float64(0.002499846974387765) - rollout_corr/chi2_seq:np.float64(0.36207995703443885) - actor/pg_loss:np.float64(-4.324305336922407e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0018963009379149298) - actor/ppo_kl:np.float64(0.0002913880196118157) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6442021429538727) - perf/mfu/actor:np.float64(0.05641402480565709) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.02770233154297) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:6 - critic/score/mean:0.484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.014271434396505356 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.014271434396505356 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:254.00390625 - response_length/max:2020.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:254.00390625 - response_length_non_aborted/max:2020.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:346.4375 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012409500777721405 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9811534341424704) - timing_s/agent_loop/generate_sequences/max:np.float64(11.214596100151539) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.4594217599369586) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.214596100151539) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:2020 - timing_s/gen:17.094698660075665 - timing_s/reward:0.04627825319766998 - timing_s/old_log_prob:2.2830312568694353 - timing_s/adv:0.11555880308151245 - timing_s/update_actor:9.736344682052732 - timing_s/step:29.311943497508764 - timing_s/stop_profile:0.00011210516095161438 - timing_per_token_ms/gen:0.2628942508277688 - timing_per_token_ms/update_actor:0.06334106212260987 - timing_per_token_ms/adv:0.0007517828881194984 - perf/total_num_tokens:153713 - perf/time_per_step:29.311943497508764 - perf/throughput:655.5049821801484 (TaskRunner pid=2099163) Training Progress: 93%|█████████▎| 93/100 [1:07:54<03:29, 29.99s/it] (TaskRunner pid=2099163) step:94 - global_seqlen/min:11634 - global_seqlen/max:21223 - global_seqlen/minmax_diff:9589 - global_seqlen/balanced_min:15316 - global_seqlen/balanced_max:15322 - global_seqlen/mean:15319.375 - actor/entropy:0.2614341378211975 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2891600728034973 - training/rollout_probs_diff_mean:0.005587474443018436 - training/rollout_probs_diff_std:0.013904578052461147 - training/rollout_actor_probs_pearson_corr:0.9982290267944336 - rollout_corr/rollout_is_mean:1.0003408193588257 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.0101715563214384e-05 - rollout_corr/rollout_is_max:1.7715976238250732 - rollout_corr/rollout_is_min:0.2889171540737152 - rollout_corr/rollout_is_std:0.04165585711598396 - rollout_corr/rollout_is_eff_sample_size:0.9982688644919676 - rollout_corr/rollout_is_seq_mean:1.0004125833511353 - rollout_corr/rollout_is_seq_std:0.0032136309891939163 - rollout_corr/rollout_is_seq_max:1.0084282159805298 - rollout_corr/rollout_is_seq_min:0.9903004765510559 - rollout_corr/rollout_is_seq_max_deviation:0.009699523448944092 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2973201628774405) - rollout_corr/training_log_ppl:np.float64(0.25612859573448077) - rollout_corr/kl:np.float64(0.0017466846469749164) - rollout_corr/k3_kl:np.float64(0.0015775575718635082) - rollout_corr/rollout_ppl:np.float64(1.2949957121163607) - rollout_corr/rollout_log_ppl:np.float64(0.2543819086276926) - rollout_corr/log_ppl_diff:np.float64(0.0017466871067881584) - rollout_corr/log_ppl_abs_diff:np.float64(0.0035796407610177994) - rollout_corr/log_ppl_diff_max:np.float64(0.01710490882396698) - rollout_corr/log_ppl_diff_min:np.float64(-0.010113626718521118) - rollout_corr/ppl_ratio:np.float64(1.0017579961568117) - rollout_corr/chi2_token:np.float64(0.0027468986809253693) - rollout_corr/chi2_seq:np.float64(0.5400724075734615) - actor/pg_loss:np.float64(-0.0003080737078562379) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0026132785301342665) - actor/ppo_kl:np.float64(0.0012474513576563595) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6366036757826805) - perf/mfu/actor:np.float64(0.044733689603173685) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.00236892700195) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:6 - critic/score/mean:0.48046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008616097271442413 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008616097271442413 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:194.32421875 - response_length/max:710.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:194.32421875 - response_length_non_aborted/max:710.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:284.40625 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011759810149669647 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1009019035845995) - timing_s/agent_loop/generate_sequences/max:np.float64(4.770813453942537) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.1518594626468257) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.770813453942537) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:670 - timing_s/gen:10.57761050015688 - timing_s/reward:0.05101897194981575 - timing_s/old_log_prob:2.3127746544778347 - timing_s/adv:0.1194564662873745 - timing_s/update_actor:9.648187587037683 - timing_s/step:22.79769565165043 - timing_s/stop_profile:5.702488124370575e-05 - timing_per_token_ms/gen:0.21262810823078537 - timing_per_token_ms/update_actor:0.07872536891222458 - timing_per_token_ms/adv:0.000974717198705679 - perf/total_num_tokens:122555 - perf/time_per_step:22.79769565165043 - perf/throughput:671.9703269172716 (TaskRunner pid=2099163) Training Progress: 94%|█████████▍| 94/100 [1:08:17<02:47, 27.84s/it] (TaskRunner pid=2099163) step:95 - global_seqlen/min:11894 - global_seqlen/max:19955 - global_seqlen/minmax_diff:8061 - global_seqlen/balanced_min:16319 - global_seqlen/balanced_max:16327 - global_seqlen/mean:16323.0 - actor/entropy:0.2705520987510681 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6408352255821228 - training/rollout_probs_diff_mean:0.005575786344707012 - training/rollout_probs_diff_std:0.013862286694347858 - training/rollout_actor_probs_pearson_corr:0.9983189105987549 - rollout_corr/rollout_is_mean:1.0001085996627808 - rollout_corr/rollout_is_ratio_fraction_high:5.360972136259079e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.360972136259079e-05 - rollout_corr/rollout_is_max:4.217565536499023 - rollout_corr/rollout_is_min:0.15521328151226044 - rollout_corr/rollout_is_std:0.04240885004401207 - rollout_corr/rollout_is_eff_sample_size:0.9982049558450077 - rollout_corr/rollout_is_seq_mean:1.0000663995742798 - rollout_corr/rollout_is_seq_std:0.0035495145712047815 - rollout_corr/rollout_is_seq_max:1.0250022411346436 - rollout_corr/rollout_is_seq_min:0.989392101764679 - rollout_corr/rollout_is_seq_max_deviation:0.025002241134643555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3268689936958253) - rollout_corr/training_log_ppl:np.float64(0.2789607130398508) - rollout_corr/kl:np.float64(0.001470564366549354) - rollout_corr/k3_kl:np.float64(0.001639172450438764) - rollout_corr/rollout_ppl:np.float64(1.324940457008779) - rollout_corr/rollout_log_ppl:np.float64(0.27749014867004007) - rollout_corr/log_ppl_diff:np.float64(0.0014705643698107451) - rollout_corr/log_ppl_abs_diff:np.float64(0.003353676962433383) - rollout_corr/log_ppl_diff_max:np.float64(0.019685059785842896) - rollout_corr/log_ppl_diff_min:np.float64(-0.014085322618484497) - rollout_corr/ppl_ratio:np.float64(1.0014813262969255) - rollout_corr/chi2_token:np.float64(0.0037556993775069714) - rollout_corr/chi2_seq:np.float64(0.9716611602343619) - actor/pg_loss:np.float64(0.0004548488650470972) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002927706459104229) - actor/ppo_kl:np.float64(0.0005539002601722132) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.7531055063009262) - perf/mfu/actor:np.float64(0.048065567875728424) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.10460662841797) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:6 - critic/score/mean:0.53125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0015368120511993766 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0015368120511993766 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:218.59375 - response_length/max:779.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:218.59375 - response_length_non_aborted/max:779.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.5 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.889948129653931e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0940031111240387) - timing_s/agent_loop/generate_sequences/max:np.float64(5.365459702908993) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3305309581846814) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.365459702908993) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:779 - timing_s/gen:11.15835091844201 - timing_s/reward:0.05341525003314018 - timing_s/old_log_prob:2.3560058772563934 - timing_s/adv:0.13407864421606064 - timing_s/update_actor:9.62122217938304 - timing_s/step:23.406750781461596 - timing_s/stop_profile:0.00011238828301429749 - timing_per_token_ms/gen:0.19939869403934973 - timing_per_token_ms/update_actor:0.07367841526820315 - timing_per_token_ms/adv:0.0010267616569875379 - perf/total_num_tokens:130584 - perf/time_per_step:23.406750781461596 - perf/throughput:697.3629168952401 (TaskRunner pid=2099163) Training Progress: 95%|█████████▌| 95/100 [1:08:40<02:12, 26.53s/it] (TaskRunner pid=2099163) step:96 - global_seqlen/min:11717 - global_seqlen/max:21275 - global_seqlen/minmax_diff:9558 - global_seqlen/balanced_min:16664 - global_seqlen/balanced_max:16674 - global_seqlen/mean:16669.875 - actor/entropy:0.25728729367256165 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2606017589569092 - training/rollout_probs_diff_mean:0.005339381285011768 - training/rollout_probs_diff_std:0.013366999104619026 - training/rollout_actor_probs_pearson_corr:0.9983529448509216 - rollout_corr/rollout_is_mean:0.9999350309371948 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6451245546340942 - rollout_corr/rollout_is_min:0.5036647319793701 - rollout_corr/rollout_is_std:0.04145504906773567 - rollout_corr/rollout_is_eff_sample_size:0.9982843083294409 - rollout_corr/rollout_is_seq_mean:0.9999916553497314 - rollout_corr/rollout_is_seq_std:0.0029035070911049843 - rollout_corr/rollout_is_seq_max:1.008498191833496 - rollout_corr/rollout_is_seq_min:0.9916707277297974 - rollout_corr/rollout_is_seq_max_deviation:0.008498191833496094 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3080988936126232) - rollout_corr/training_log_ppl:np.float64(0.2644939655147027) - rollout_corr/kl:np.float64(0.0013578184077687183) - rollout_corr/k3_kl:np.float64(0.001467021100893362) - rollout_corr/rollout_ppl:np.float64(1.3062553596682847) - rollout_corr/rollout_log_ppl:np.float64(0.2631361448438838) - rollout_corr/log_ppl_diff:np.float64(0.00135782067081891) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032226947078015655) - rollout_corr/log_ppl_diff_max:np.float64(0.01374003291130066) - rollout_corr/log_ppl_diff_min:np.float64(-0.008670985698699951) - rollout_corr/ppl_ratio:np.float64(1.0013662767596543) - rollout_corr/chi2_token:np.float64(0.003215978853404522) - rollout_corr/chi2_seq:np.float64(0.44253751053474844) - actor/pg_loss:np.float64(0.00011486734729260206) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020753747680828383) - actor/ppo_kl:np.float64(0.00043077189779694436) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6561485677957535) - perf/mfu/actor:np.float64(0.04976022063857087) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.02475357055664) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:6 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014713210053741932 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014713210053741932 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:219.49609375 - response_length/max:856.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:219.49609375 - response_length_non_aborted/max:856.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:301.4375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013059936463832855 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2289475314319134) - timing_s/agent_loop/generate_sequences/max:np.float64(5.740047561004758) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.310526862871484) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.740047561004758) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:856 - timing_s/gen:11.425192607566714 - timing_s/reward:0.051955023780465126 - timing_s/old_log_prob:2.3286557272076607 - timing_s/adv:0.12474869564175606 - timing_s/update_actor:9.438106391578913 - timing_s/step:23.46123932301998 - timing_s/stop_profile:3.103911876678467e-05 - timing_per_token_ms/gen:0.2033278035195443 - timing_per_token_ms/update_actor:0.07077217429329039 - timing_per_token_ms/adv:0.0009354351460475563 - perf/total_num_tokens:133359 - perf/time_per_step:23.46123932301998 - perf/throughput:710.5283216493875 (TaskRunner pid=2099163) Training Progress: 96%|█████████▌| 96/100 [1:09:04<01:42, 25.61s/it] (TaskRunner pid=2099163) step:97 - global_seqlen/min:10859 - global_seqlen/max:28193 - global_seqlen/minmax_diff:17334 - global_seqlen/balanced_min:17081 - global_seqlen/balanced_max:23683 - global_seqlen/mean:18237.375 - actor/entropy:0.22356657683849335 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35336679220199585 - training/rollout_probs_diff_mean:0.004439533222466707 - training/rollout_probs_diff_std:0.01241324283182621 - training/rollout_actor_probs_pearson_corr:0.998500406742096 - rollout_corr/rollout_is_mean:1.0001307725906372 - rollout_corr/rollout_is_ratio_fraction_high:2.887044320232235e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.4435221601161174e-05 - rollout_corr/rollout_is_max:3.0604419708251953 - rollout_corr/rollout_is_min:0.4321064054965973 - rollout_corr/rollout_is_std:0.03731440007686615 - rollout_corr/rollout_is_eff_sample_size:0.9986098092374474 - rollout_corr/rollout_is_seq_mean:1.000051498413086 - rollout_corr/rollout_is_seq_std:0.0030096296686679125 - rollout_corr/rollout_is_seq_max:1.0088967084884644 - rollout_corr/rollout_is_seq_min:0.9923568964004517 - rollout_corr/rollout_is_seq_max_deviation:0.008896708488464355 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.32025858014822) - rollout_corr/training_log_ppl:np.float64(0.2736287588450068) - rollout_corr/kl:np.float64(0.001147672094734986) - rollout_corr/k3_kl:np.float64(0.0012670807614085788) - rollout_corr/rollout_ppl:np.float64(1.318727934267372) - rollout_corr/rollout_log_ppl:np.float64(0.27248108579806285) - rollout_corr/log_ppl_diff:np.float64(0.0011476730469439644) - rollout_corr/log_ppl_abs_diff:np.float64(0.002580658172519179) - rollout_corr/log_ppl_diff_max:np.float64(0.010686814785003662) - rollout_corr/log_ppl_diff_min:np.float64(-0.00808987021446228) - rollout_corr/ppl_ratio:np.float64(1.0011534958612174) - rollout_corr/chi2_token:np.float64(0.002894336823374033) - rollout_corr/chi2_seq:np.float64(0.502729723462835) - actor/pg_loss:np.float64(1.6757636331021786e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014571902823945493) - actor/ppo_kl:np.float64(0.0002972853384548557) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5494845286011696) - perf/mfu/actor:np.float64(0.050740963915102175) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.98857498168945) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:6 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.03918260708451271 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.03918260708451271 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:270.60546875 - response_length/max:8192.0 - response_length/min:88.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:270.60546875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:299.3125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.5369226932525635e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.058408448472619) - timing_s/agent_loop/generate_sequences/max:np.float64(49.16501975432038) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.604979175426706) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.16501975432038) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:179 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.1183186173439 - timing_s/reward:0.05663715489208698 - timing_s/old_log_prob:2.494466742500663 - timing_s/adv:0.13287332467734814 - timing_s/update_actor:10.396746782585979 - timing_s/step:68.28716198541224 - timing_s/stop_profile:0.00018381327390670776 - timing_per_token_ms/gen:0.7956451622857293 - timing_per_token_ms/update_actor:0.07125989062698153 - timing_per_token_ms/adv:0.0009107212844320259 - perf/total_num_tokens:145899 - perf/time_per_step:68.28716198541224 - perf/throughput:267.06886726228186 (TaskRunner pid=2099163) Training Progress: 97%|█████████▋| 97/100 [1:10:12<01:55, 38.43s/it] (TaskRunner pid=2099163) step:98 - global_seqlen/min:10881 - global_seqlen/max:21856 - global_seqlen/minmax_diff:10975 - global_seqlen/balanced_min:16595 - global_seqlen/balanced_max:16744 - global_seqlen/mean:16625.0 - actor/entropy:0.27939262986183167 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2851577401161194 - training/rollout_probs_diff_mean:0.00533874798566103 - training/rollout_probs_diff_std:0.01307785976678133 - training/rollout_actor_probs_pearson_corr:0.9984779357910156 - rollout_corr/rollout_is_mean:0.9997860789299011 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.518648736644536e-05 - rollout_corr/rollout_is_max:1.6622998714447021 - rollout_corr/rollout_is_min:0.3280748724937439 - rollout_corr/rollout_is_std:0.040275804698467255 - rollout_corr/rollout_is_eff_sample_size:0.9983798331417372 - rollout_corr/rollout_is_seq_mean:0.999704897403717 - rollout_corr/rollout_is_seq_std:0.0033357173670083284 - rollout_corr/rollout_is_seq_max:1.0101433992385864 - rollout_corr/rollout_is_seq_min:0.9884958267211914 - rollout_corr/rollout_is_seq_max_deviation:0.011504173278808594 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3345653787255287) - rollout_corr/training_log_ppl:np.float64(0.2840118541062111) - rollout_corr/kl:np.float64(0.0016829417983466044) - rollout_corr/k3_kl:np.float64(0.0013737696041289382) - rollout_corr/rollout_ppl:np.float64(1.3323034564964473) - rollout_corr/rollout_log_ppl:np.float64(0.282328910601791) - rollout_corr/log_ppl_diff:np.float64(0.001682943504420109) - rollout_corr/log_ppl_abs_diff:np.float64(0.003258420663769357) - rollout_corr/log_ppl_diff_max:np.float64(0.02178385853767395) - rollout_corr/log_ppl_diff_min:np.float64(-0.009423941373825073) - rollout_corr/ppl_ratio:np.float64(1.0016928501427174) - rollout_corr/chi2_token:np.float64(0.002250493736937642) - rollout_corr/chi2_seq:np.float64(0.3969768069218844) - actor/pg_loss:np.float64(0.00017058837693184614) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0019098859092991916) - actor/ppo_kl:np.float64(0.0005005859824507297) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6195161938667297) - perf/mfu/actor:np.float64(0.048713654755540986) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.02078247070312) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:6 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0020672061946243048 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0020672061946243048 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:222.03125 - response_length/max:1144.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:222.03125 - response_length_non_aborted/max:1144.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:297.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013743340969085693 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1269784551113844) - timing_s/agent_loop/generate_sequences/max:np.float64(7.022609854117036) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.2723799158920883) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.022609854117036) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1144 - timing_s/gen:12.626804063096642 - timing_s/reward:0.05744520016014576 - timing_s/old_log_prob:2.334137598052621 - timing_s/adv:0.12326470203697681 - timing_s/update_actor:9.706466360017657 - timing_s/step:24.93613464012742 - timing_s/stop_profile:0.00017854012548923492 - timing_per_token_ms/gen:0.22214644727474742 - timing_per_token_ms/update_actor:0.07298095007532072 - timing_per_token_ms/adv:0.0009268022709547129 - perf/total_num_tokens:133000 - perf/time_per_step:24.93613464012742 - perf/throughput:666.703169513968 (TaskRunner pid=2099163) Training Progress: 98%|█████████▊| 98/100 [1:10:37<01:08, 34.39s/it] (TaskRunner pid=2099163) step:99 - global_seqlen/min:10950 - global_seqlen/max:22756 - global_seqlen/minmax_diff:11806 - global_seqlen/balanced_min:15802 - global_seqlen/balanced_max:15820 - global_seqlen/mean:15813.5 - actor/entropy:0.2819356322288513 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7740305066108704 - training/rollout_probs_diff_mean:0.005276948679238558 - training/rollout_probs_diff_std:0.013337536714971066 - training/rollout_actor_probs_pearson_corr:0.9984665513038635 - rollout_corr/rollout_is_mean:1.0000571012496948 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.5403706937795505e-05 - rollout_corr/rollout_is_max:1.6278910636901855 - rollout_corr/rollout_is_min:0.07193134725093842 - rollout_corr/rollout_is_std:0.040468472987413406 - rollout_corr/rollout_is_eff_sample_size:0.9983652180867625 - rollout_corr/rollout_is_seq_mean:1.0001047849655151 - rollout_corr/rollout_is_seq_std:0.003330203937366605 - rollout_corr/rollout_is_seq_max:1.0106587409973145 - rollout_corr/rollout_is_seq_min:0.9881437420845032 - rollout_corr/rollout_is_seq_max_deviation:0.011856257915496826 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3568300306797028) - rollout_corr/training_log_ppl:np.float64(0.3001139621483162) - rollout_corr/kl:np.float64(0.0011941435956686064) - rollout_corr/k3_kl:np.float64(0.0011522117289359812) - rollout_corr/rollout_ppl:np.float64(1.3551504230126739) - rollout_corr/rollout_log_ppl:np.float64(0.29891981728724204) - rollout_corr/log_ppl_diff:np.float64(0.0011941448610741645) - rollout_corr/log_ppl_abs_diff:np.float64(0.0030851933697704226) - rollout_corr/log_ppl_diff_max:np.float64(0.027537941932678223) - rollout_corr/log_ppl_diff_min:np.float64(-0.007955223321914673) - rollout_corr/ppl_ratio:np.float64(1.0012031164951622) - rollout_corr/chi2_token:np.float64(0.0022564970422536135) - rollout_corr/chi2_seq:np.float64(0.5483720011543483) - actor/pg_loss:np.float64(0.0006328735034912825) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009942931160367152) - actor/ppo_kl:np.float64(0.0003664185717591195) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5217531993985176) - perf/mfu/actor:np.float64(0.046482210549823294) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.91424560546875) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:7 - critic/score/mean:0.578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011117677204310894 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011117677204310894 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:211.515625 - response_length/max:1026.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:211.515625 - response_length_non_aborted/max:1026.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:282.65625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0004795752465724945 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0314996894448996) - timing_s/agent_loop/generate_sequences/max:np.float64(6.317301785573363) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.143986992428836) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.317301785573363) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:1026 - timing_s/gen:12.40069780871272 - timing_s/reward:0.04780052416026592 - timing_s/old_log_prob:2.5466749370098114 - timing_s/adv:0.11402323842048645 - timing_s/update_actor:9.808757556602359 - timing_s/step:25.033864453434944 - timing_s/stop_profile:0.00011504441499710083 - timing_per_token_ms/gen:0.2290148815969698 - timing_per_token_ms/update_actor:0.07753468204858474 - timing_per_token_ms/adv:0.0009013124736813992 - perf/total_num_tokens:126508 - perf/time_per_step:25.033864453434944 - perf/throughput:631.6843342111409 (TaskRunner pid=2099163) Training Progress: 99%|█████████▉| 99/100 [1:11:03<00:31, 31.74s/it] (TaskRunner pid=2099163) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2099163) validation generation end (TaskRunner pid=2099163) [prompt] system (TaskRunner pid=2099163) (TaskRunner pid=2099163) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) user (TaskRunner pid=2099163) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2099163) (TaskRunner pid=2099163) A: -0.01 (TaskRunner pid=2099163) B: 1.69 (TaskRunner pid=2099163) C: 2.49 (TaskRunner pid=2099163) D: 0.45 (TaskRunner pid=2099163) Please reason step by step. (TaskRunner pid=2099163) assistant (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) [response] (TaskRunner pid=2099163) The folding stability score for a given protein sequence can be determined using computational tools such as I-TAFOR, which predicts the stability of a protein's folded structure based on its sequence. The specific sequence provided, 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR', would be input into such a tool to calculate the folding stability score. Based on computational predictions, the folding stability score for this sequence is known to be 0.45. (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) D (TaskRunner pid=2099163) (TaskRunner pid=2099163) [ground_truth] B (TaskRunner pid=2099163) [score] 0.0 (TaskRunner pid=2099163) [acc] 0.0 (TaskRunner pid=2099163) [pred] D (TaskRunner pid=2099163) [incorrect_format] 1 (TaskRunner pid=2099163) [feedback] (TaskRunner pid=2099163) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (TaskRunner pid=2099163) (WorkerDict pid=2099539) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90/actor (TaskRunner pid=2099163) step:100 - global_seqlen/min:13194 - global_seqlen/max:24171 - global_seqlen/minmax_diff:10977 - global_seqlen/balanced_min:19146 - global_seqlen/balanced_max:19154 - global_seqlen/mean:19150.75 - actor/entropy:0.2835403382778168 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3625403642654419 - training/rollout_probs_diff_mean:0.005042023025453091 - training/rollout_probs_diff_std:0.012832351960241795 - training/rollout_actor_probs_pearson_corr:0.9986383318901062 - rollout_corr/rollout_is_mean:1.000214695930481 - rollout_corr/rollout_is_ratio_fraction_high:3.143171488773078e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.7147572331596166e-05 - rollout_corr/rollout_is_max:4.838179588317871 - rollout_corr/rollout_is_min:0.209492027759552 - rollout_corr/rollout_is_std:0.040638964623212814 - rollout_corr/rollout_is_eff_sample_size:0.9983516728081235 - rollout_corr/rollout_is_seq_mean:1.000152587890625 - rollout_corr/rollout_is_seq_std:0.002947622211650014 - rollout_corr/rollout_is_seq_max:1.0129314661026 - rollout_corr/rollout_is_seq_min:0.9921380877494812 - rollout_corr/rollout_is_seq_max_deviation:0.012931466102600098 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3729312745854259) - rollout_corr/training_log_ppl:np.float64(0.3114306655479595) - rollout_corr/kl:np.float64(0.0015098723376403278) - rollout_corr/k3_kl:np.float64(0.001307048618286899) - rollout_corr/rollout_ppl:np.float64(1.3707687840797007) - rollout_corr/rollout_log_ppl:np.float64(0.30992078845156357) - rollout_corr/log_ppl_diff:np.float64(0.0015098770963959396) - rollout_corr/log_ppl_abs_diff:np.float64(0.003156350867357105) - rollout_corr/log_ppl_diff_max:np.float64(0.014475628733634949) - rollout_corr/log_ppl_diff_min:np.float64(-0.009900987148284912) - rollout_corr/ppl_ratio:np.float64(1.0015187887474895) - rollout_corr/chi2_token:np.float64(0.0023139831610023975) - rollout_corr/chi2_seq:np.float64(0.866482462733984) - actor/pg_loss:np.float64(5.068804603070021e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010578534675005358) - actor/ppo_kl:np.float64(0.0007291194974783366) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.496917188167572) - perf/mfu/actor:np.float64(0.056309282462256305) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.95842933654785) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.42625) - val-aux/sciknoweval/reward/std@16:np.float64(0.20340466932397192) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.51122) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1717994688879882) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.34018) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.17528803531142753) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.42506) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2031877981200387) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.5764) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.12262009307652777) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.26536) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12852696069401104) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.43562000000000006) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.16364220520548403) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.62372) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08028605088755732) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.21248) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07648178927728648) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.4453000000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1155248617088842) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.6543800000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.0472167585635409) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.18634) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.032643145410378605) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.44676000000000005) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07340249583082686) - val-aux/sciknoweval/score/mean@16:np.float64(0.42625) - val-aux/sciknoweval/score/std@16:np.float64(0.20340466932397192) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.51122) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1717994688879882) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.34018) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.17528803531142753) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.42506) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2031877981200387) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.5764) - val-aux/sciknoweval/score/best@4/std:np.float64(0.12262009307652777) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.26536) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12852696069401104) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.43562000000000006) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.16364220520548403) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.62372) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08028605088755732) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.21248) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07648178927728648) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.4453000000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.1155248617088842) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.6543800000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.0472167585635409) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.18634) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.032643145410378605) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.44676000000000005) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07340249583082686) - val-core/sciknoweval/acc/mean@16:np.float64(0.42625) - val-aux/sciknoweval/acc/std@16:np.float64(0.20340466932397192) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.51122) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1717994688879882) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.34018) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.17528803531142753) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.42506) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2031877981200387) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.5764) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.12262009307652777) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.26536) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12852696069401104) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.43562000000000006) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.16364220520548403) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.62372) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08028605088755732) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.21248) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07648178927728648) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.4453000000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.1155248617088842) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.6543800000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.0472167585635409) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.18634) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.032643145410378605) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.44676000000000005) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07340249583082686) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9975) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.009682458365518542) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9997800000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0029552123640712146) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9955799999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.012525805206779563) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.99798) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.008749306906094262) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9915) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.01634315511370097) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9993599999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.004842664831259046) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.98444) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.019500890160644296) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99984) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.002504474311076614) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9751599999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.019403379135729548) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:7 - critic/score/mean:0.5234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003972182981669903 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003972182981669903 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:248.5546875 - response_length/max:977.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:248.5546875 - response_length_non_aborted/max:977.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:349.90625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.883567690849304e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7739909812808037) - timing_s/agent_loop/generate_sequences/max:np.float64(6.653986681252718) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.1721716843530885) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.653986681252718) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:977 - timing_s/gen:12.76038683950901 - timing_s/reward:0.05503258667886257 - timing_s/old_log_prob:2.375061484053731 - timing_s/adv:0.11991865560412407 - timing_s/update_actor:9.685975370928645 - timing_s/step:25.090152885764837 - timing_s/testing:66.87851952016354 - timing_s/save_checkpoint:6.8637774139642715 - timing_s/stop_profile:0.00011190585792064667 - timing_per_token_ms/gen:0.20054041866272215 - timing_per_token_ms/update_actor:0.06322190626299652 - timing_per_token_ms/adv:0.000782728193439709 - perf/total_num_tokens:153206 - perf/time_per_step:25.090152885764837 - perf/throughput:763.2775331100266 (TaskRunner pid=2099163) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2099163) "np.float64(0.42625), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2099163) "np.float64(0.20340466932397192), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2099163) "np.float64(0.51122), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2099163) "np.float64(0.1717994688879882), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2099163) "np.float64(0.34018), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2099163) "np.float64(0.17528803531142753), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2099163) "np.float64(0.42506), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2099163) "np.float64(0.2031877981200387), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2099163) "np.float64(0.5764), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2099163) "np.float64(0.12262009307652777), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2099163) "np.float64(0.26536), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2099163) "np.float64(0.12852696069401104), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2099163) "np.float64(0.43562000000000006), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2099163) "np.float64(0.16364220520548403), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2099163) "np.float64(0.62372), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2099163) "np.float64(0.08028605088755732), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2099163) "np.float64(0.21248), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2099163) "np.float64(0.07648178927728648), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2099163) "np.float64(0.4453000000000001), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2099163) "np.float64(0.1155248617088842), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2099163) "np.float64(0.6543800000000001), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2099163) "np.float64(0.0472167585635409), 'val-aux/sciknoweval/reward/worst@16/mean': " (TaskRunner pid=2099163) "np.float64(0.18634), 'val-aux/sciknoweval/reward/worst@16/std': " (TaskRunner pid=2099163) "np.float64(0.032643145410378605), 'val-aux/sciknoweval/reward/maj@16/mean': " (TaskRunner pid=2099163) "np.float64(0.44676000000000005), 'val-aux/sciknoweval/reward/maj@16/std': " (TaskRunner pid=2099163) "np.float64(0.07340249583082686), 'val-aux/sciknoweval/score/mean@16': " (TaskRunner pid=2099163) "np.float64(0.42625), 'val-aux/sciknoweval/score/std@16': " (TaskRunner pid=2099163) "np.float64(0.20340466932397192), 'val-aux/sciknoweval/score/best@2/mean': " (TaskRunner pid=2099163) "np.float64(0.51122), 'val-aux/sciknoweval/score/best@2/std': " (TaskRunner pid=2099163) "np.float64(0.1717994688879882), 'val-aux/sciknoweval/score/worst@2/mean': " (TaskRunner pid=2099163) "np.float64(0.34018), 'val-aux/sciknoweval/score/worst@2/std': " (TaskRunner pid=2099163) "np.float64(0.17528803531142753), 'val-aux/sciknoweval/score/maj@2/mean': " (TaskRunner pid=2099163) "np.float64(0.42506), 'val-aux/sciknoweval/score/maj@2/std': " (TaskRunner pid=2099163) "np.float64(0.2031877981200387), 'val-aux/sciknoweval/score/best@4/mean': " (TaskRunner pid=2099163) "np.float64(0.5764), 'val-aux/sciknoweval/score/best@4/std': " (TaskRunner pid=2099163) "np.float64(0.12262009307652777), 'val-aux/sciknoweval/score/worst@4/mean': " (TaskRunner pid=2099163) "np.float64(0.26536), 'val-aux/sciknoweval/score/worst@4/std': " (TaskRunner pid=2099163) "np.float64(0.12852696069401104), 'val-aux/sciknoweval/score/maj@4/mean': " (TaskRunner pid=2099163) "np.float64(0.43562000000000006), 'val-aux/sciknoweval/score/maj@4/std': " (TaskRunner pid=2099163) "np.float64(0.16364220520548403), 'val-aux/sciknoweval/score/best@8/mean': " (TaskRunner pid=2099163) "np.float64(0.62372), 'val-aux/sciknoweval/score/best@8/std': " (TaskRunner pid=2099163) "np.float64(0.08028605088755732), 'val-aux/sciknoweval/score/worst@8/mean': " (TaskRunner pid=2099163) "np.float64(0.21248), 'val-aux/sciknoweval/score/worst@8/std': " (TaskRunner pid=2099163) "np.float64(0.07648178927728648), 'val-aux/sciknoweval/score/maj@8/mean': " (TaskRunner pid=2099163) "np.float64(0.4453000000000001), 'val-aux/sciknoweval/score/maj@8/std': " (TaskRunner pid=2099163) "np.float64(0.1155248617088842), 'val-aux/sciknoweval/score/best@16/mean': " (TaskRunner pid=2099163) "np.float64(0.6543800000000001), 'val-aux/sciknoweval/score/best@16/std': " (TaskRunner pid=2099163) "np.float64(0.0472167585635409), 'val-aux/sciknoweval/score/worst@16/mean': " (TaskRunner pid=2099163) "np.float64(0.18634), 'val-aux/sciknoweval/score/worst@16/std': " (TaskRunner pid=2099163) "np.float64(0.032643145410378605), 'val-aux/sciknoweval/score/maj@16/mean': " (TaskRunner pid=2099163) "np.float64(0.44676000000000005), 'val-aux/sciknoweval/score/maj@16/std': " (TaskRunner pid=2099163) "np.float64(0.07340249583082686), 'val-core/sciknoweval/acc/mean@16': " (TaskRunner pid=2099163) "np.float64(0.42625), 'val-aux/sciknoweval/acc/std@16': " (TaskRunner pid=2099163) "np.float64(0.20340466932397192), 'val-aux/sciknoweval/acc/best@2/mean': " (TaskRunner pid=2099163) "np.float64(0.51122), 'val-aux/sciknoweval/acc/best@2/std': " (TaskRunner pid=2099163) "np.float64(0.1717994688879882), 'val-aux/sciknoweval/acc/worst@2/mean': " (TaskRunner pid=2099163) "np.float64(0.34018), 'val-aux/sciknoweval/acc/worst@2/std': " (TaskRunner pid=2099163) "np.float64(0.17528803531142753), 'val-aux/sciknoweval/acc/maj@2/mean': " (TaskRunner pid=2099163) "np.float64(0.42506), 'val-aux/sciknoweval/acc/maj@2/std': " (TaskRunner pid=2099163) "np.float64(0.2031877981200387), 'val-aux/sciknoweval/acc/best@4/mean': " (TaskRunner pid=2099163) "np.float64(0.5764), 'val-aux/sciknoweval/acc/best@4/std': " (TaskRunner pid=2099163) "np.float64(0.12262009307652777), 'val-aux/sciknoweval/acc/worst@4/mean': " (TaskRunner pid=2099163) "np.float64(0.26536), 'val-aux/sciknoweval/acc/worst@4/std': " (TaskRunner pid=2099163) "np.float64(0.12852696069401104), 'val-aux/sciknoweval/acc/maj@4/mean': " (TaskRunner pid=2099163) "np.float64(0.43562000000000006), 'val-aux/sciknoweval/acc/maj@4/std': " (TaskRunner pid=2099163) "np.float64(0.16364220520548403), 'val-aux/sciknoweval/acc/best@8/mean': " (TaskRunner pid=2099163) "np.float64(0.62372), 'val-aux/sciknoweval/acc/best@8/std': " (TaskRunner pid=2099163) "np.float64(0.08028605088755732), 'val-aux/sciknoweval/acc/worst@8/mean': " (TaskRunner pid=2099163) "np.float64(0.21248), 'val-aux/sciknoweval/acc/worst@8/std': " (TaskRunner pid=2099163) "np.float64(0.07648178927728648), 'val-aux/sciknoweval/acc/maj@8/mean': " (TaskRunner pid=2099163) "np.float64(0.4453000000000001), 'val-aux/sciknoweval/acc/maj@8/std': " (TaskRunner pid=2099163) "np.float64(0.1155248617088842), 'val-core/sciknoweval/acc/best@16/mean': " (TaskRunner pid=2099163) "np.float64(0.6543800000000001), 'val-core/sciknoweval/acc/best@16/std': " (TaskRunner pid=2099163) "np.float64(0.0472167585635409), 'val-aux/sciknoweval/acc/worst@16/mean': " (TaskRunner pid=2099163) "np.float64(0.18634), 'val-aux/sciknoweval/acc/worst@16/std': " (TaskRunner pid=2099163) "np.float64(0.032643145410378605), 'val-core/sciknoweval/acc/maj@16/mean': " (TaskRunner pid=2099163) "np.float64(0.44676000000000005), 'val-core/sciknoweval/acc/maj@16/std': " (TaskRunner pid=2099163) 'np.float64(0.07340249583082686), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/mean@16': np.float64(0.9975), " (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2099163) 'np.float64(0.009682458365518542), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/best@2/mean': " (TaskRunner pid=2099163) 'np.float64(0.9997800000000001), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2099163) 'np.float64(0.0029552123640712146), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': " (TaskRunner pid=2099163) 'np.float64(0.9955799999999999), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2099163) 'np.float64(0.012525805206779563), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(0.99798), " (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2099163) 'np.float64(0.008749306906094262), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/best@4/mean': np.float64(1.0), " (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/best@4/std': np.float64(0.0), " (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': np.float64(0.9915), " (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2099163) 'np.float64(0.01634315511370097), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': " (TaskRunner pid=2099163) 'np.float64(0.9993599999999999), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2099163) 'np.float64(0.004842664831259046), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2099163) (TaskRunner pid=2099163) Training Progress: 100%|██████████| 100/100 [1:12:41<00:00, 51.89s/it] (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': np.float64(0.98444), " (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2099163) 'np.float64(0.019500890160644296), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': np.float64(0.99984), " (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/maj@8/std': " (TaskRunner pid=2099163) 'np.float64(0.002504474311076614), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': " (TaskRunner pid=2099163) 'np.float64(0.9751599999999999), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2099163) 'np.float64(0.019403379135729548), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': np.float64(1.0), " (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/maj@16/std': np.float64(0.0), " (TaskRunner pid=2099163) "'val-aux/num_turns/min': np.int32(2), 'val-aux/num_turns/max': np.int32(2), " (TaskRunner pid=2099163) "'val-aux/num_turns/mean': np.float64(2.0)}") (TaskRunner pid=2099163) Training Progress: 100%|██████████| 100/100 [1:12:41<00:00, 43.62s/it] (TaskRunner pid=2099163) wandb: updating run metadata (TaskRunner pid=2099163) wandb: uploading output.log; uploading wandb-summary.json (TaskRunner pid=2099163) wandb: uploading output.log (TaskRunner pid=2099163) wandb: uploading history steps 98-99, summary, console lines 546-667 (TaskRunner pid=2099163) wandb: (TaskRunner pid=2099163) wandb: Run history: (TaskRunner pid=2099163) wandb: actor/entropy ▂▄▂▁▃▁▃▃▄▄▄▃▄▄▄▄▃▄▄▄▄▄▄▅▅▇▆▆▆▆▆█▆▇▆█▇▆▆▇ (TaskRunner pid=2099163) wandb: actor/grad_norm █▃▄▃▄▄▄▂▃▃▃▂▂▂▃▂▂▂▂▂▂▂▂▂▂▂▃▃▂▄▄▂▂▁▁▂▃▃▄▂ (TaskRunner pid=2099163) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2099163) wandb: actor/lr ▁▂▄▅▆███████████████████████████████████ (TaskRunner pid=2099163) wandb: actor/pg_clipfrac ▄▄▄▄▆▄▂▃▆▃▁▂▂▂▃▃▅▃▂▄▄▂▄▃▅▂▅▅▃▄▇▄▂▂█▅▅▇█▂ (TaskRunner pid=2099163) wandb: actor/pg_clipfrac_lower ▁▁▁▁▆▁▁▁▁▁▁▁▁▁█▁▄▄▃▁▁▁▁▁▁▁▁▁▁▁▁▁█▁▁▁▅▁▁▁ (TaskRunner pid=2099163) wandb: actor/pg_loss ▂▅▆▆█▁▆▂▇▅▃▄▃█▄▄▅▅▄▄▄▆▄▄▄▅▃▄▆▅▆▂▅▇▃▄▆▄▃█ (TaskRunner pid=2099163) wandb: actor/ppo_kl ▂▂▁▂▂▃▃▃▂▃▂▁▂▄▁▄▂▂▄█▃▂▅▂▃▃▆▆▃▄▂▃▅▄▂▃▃▂▃▃ (TaskRunner pid=2099163) wandb: critic/advantages/max ████████▁███████████████████████████████ (TaskRunner pid=2099163) wandb: critic/advantages/mean ▃▂▁▁▅▁▄▅▅▃▄▃▃▄▄▂▄▂▄▁▄▄▄▅▄▅▄▃▄█▄▄▃▄▄▄▅▃▄▄ (TaskRunner pid=2099163) wandb: +204 ... (TaskRunner pid=2099163) wandb: (TaskRunner pid=2099163) wandb: Run summary: (TaskRunner pid=2099163) wandb: actor/entropy 0.28354 (TaskRunner pid=2099163) wandb: actor/grad_norm 0.49692 (TaskRunner pid=2099163) wandb: actor/kl_loss 0 (TaskRunner pid=2099163) wandb: actor/lr 0.0 (TaskRunner pid=2099163) wandb: actor/pg_clipfrac 0.00106 (TaskRunner pid=2099163) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2099163) wandb: actor/pg_loss 5e-05 (TaskRunner pid=2099163) wandb: actor/ppo_kl 0.00073 (TaskRunner pid=2099163) wandb: critic/advantages/max 0.875 (TaskRunner pid=2099163) wandb: critic/advantages/mean -0.00397 (TaskRunner pid=2099163) wandb: +204 ... (TaskRunner pid=2099163) wandb: (TaskRunner pid=2099163) wandb: 🚀 View run qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/94h2xs6b (TaskRunner pid=2099163) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2099163) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2099163) wandb: Find logs at: ./wandb/run-20260702_102415-94h2xs6b/logs (TaskRunner pid=2099163) Exception ignored in atexit callback: .teardown_atexit at 0x7d8258333880> (TaskRunner pid=2099163) Traceback (most recent call last): (TaskRunner pid=2099163) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 65, in teardown_atexit (TaskRunner pid=2099163) conn.teardown(hooks.exit_code) (TaskRunner pid=2099163) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 299, in teardown (TaskRunner pid=2099163) self._asyncer.run(publish_teardown_and_close) (TaskRunner pid=2099163) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run (TaskRunner pid=2099163) return future.result() (TaskRunner pid=2099163) ^^^^^^^^^^^^^^^ (TaskRunner pid=2099163) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result (TaskRunner pid=2099163) return self.__get_result() (TaskRunner pid=2099163) ^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2099163) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (TaskRunner pid=2099163) raise self._exception (TaskRunner pid=2099163) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap (TaskRunner pid=2099163) return await fn() (TaskRunner pid=2099163) ^^^^^^^^^^ (TaskRunner pid=2099163) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 297, in publish_teardown_and_close (TaskRunner pid=2099163) await self._client.close() (TaskRunner pid=2099163) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 69, in close (TaskRunner pid=2099163) await self._writer.wait_closed() (TaskRunner pid=2099163) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 364, in wait_closed (TaskRunner pid=2099163) await self._protocol._get_close_waiter(self) (TaskRunner pid=2099163) BrokenPipeError: [Errno 32] Broken pipe main_ppo exit code: 0 [2026-07-02 11:37:05] Finished biology grpo [2026-07-02 11:37:05] Starting biology rlsd_tr Experiment: qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Dataset: biology Method: rlsd_tr Config: rlsd Model: Qwen/Qwen3-4B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet Ray tmp: /tmp/ray_q3g_biology_rlsd_tr_Qwen3_4B 2026-07-02 11:37:06,851 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Config: rlsd Task: datasets/sciknoweval/biology Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-RLSD-TR-GRPO-matched-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_biology_rlsd_tr_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/biology +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.model.use_fused_kernels=False actor_rollout_ref.actor.policy_loss.loss_mode=rlsd actor_rollout_ref.actor.self_distillation.token_reweight_lambda=0.5 actor_rollout_ref.actor.self_distillation.token_reweight_eps_w=0.2 actor_rollout_ref.actor.self_distillation.token_reweight_decay_steps=0 actor_rollout_ref.actor.self_distillation.teacher_regularization=trust-region actor_rollout_ref.actor.self_distillation.teacher_update_rate=0.1 actor_rollout_ref.actor.self_distillation.max_reprompt_len=10240 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_biology_rlsd_tr_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/biology', 'EXPERIMENT': 'qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 11:37:16,648 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2122883) TaskRunner hostname: mole-workspace-rw, PID: 2122883 (TaskRunner pid=2122883) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2122883) 'calculate_entropy': False, (TaskRunner pid=2122883) 'calculate_sum_pi_squared': False, (TaskRunner pid=2122883) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2122883) 'async_save': False, (TaskRunner pid=2122883) 'load_contents': ['model', (TaskRunner pid=2122883) 'optimizer', (TaskRunner pid=2122883) 'extra'], (TaskRunner pid=2122883) 'save_contents': ['model', (TaskRunner pid=2122883) 'optimizer', (TaskRunner pid=2122883) 'extra']}, (TaskRunner pid=2122883) 'clip_ratio': 0.2, (TaskRunner pid=2122883) 'clip_ratio_c': 3.0, (TaskRunner pid=2122883) 'clip_ratio_high': 0.28, (TaskRunner pid=2122883) 'clip_ratio_low': 0.2, (TaskRunner pid=2122883) 'data_loader_seed': 42, (TaskRunner pid=2122883) 'entropy_checkpointing': False, (TaskRunner pid=2122883) 'entropy_coeff': 0, (TaskRunner pid=2122883) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2122883) 'freeze_vision_tower': False, (TaskRunner pid=2122883) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2122883) 'dtype': 'bfloat16', (TaskRunner pid=2122883) 'entropy_checkpointing': False, (TaskRunner pid=2122883) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2122883) 'forward_only': False, (TaskRunner pid=2122883) 'forward_prefetch': False, (TaskRunner pid=2122883) 'fsdp_size': -1, (TaskRunner pid=2122883) 'full_determinism': False, (TaskRunner pid=2122883) 'model_dtype': 'fp32', (TaskRunner pid=2122883) 'offload_policy': False, (TaskRunner pid=2122883) 'optimizer_offload': False, (TaskRunner pid=2122883) 'param_offload': False, (TaskRunner pid=2122883) 'reshard_after_forward': True, (TaskRunner pid=2122883) 'seed': 42, (TaskRunner pid=2122883) 'strategy': 'fsdp', (TaskRunner pid=2122883) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2122883) 'use_orig_params': False, (TaskRunner pid=2122883) 'use_torch_compile': True, (TaskRunner pid=2122883) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2122883) 'grad_clip': 1.0, (TaskRunner pid=2122883) 'kl_loss_coef': 0.001, (TaskRunner pid=2122883) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2122883) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2122883) 'loss_scale_factor': None, (TaskRunner pid=2122883) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2122883) 'betas': [0.9, 0.999], (TaskRunner pid=2122883) 'clip_grad': 1.0, (TaskRunner pid=2122883) 'lr': 1e-06, (TaskRunner pid=2122883) 'lr_scheduler_type': 'constant', (TaskRunner pid=2122883) 'lr_warmup_steps': 10, (TaskRunner pid=2122883) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2122883) 'min_lr_ratio': 0.0, (TaskRunner pid=2122883) 'num_cycles': 0.5, (TaskRunner pid=2122883) 'optimizer': 'AdamW', (TaskRunner pid=2122883) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2122883) 'override_optimizer_config': None, (TaskRunner pid=2122883) 'total_training_steps': -1, (TaskRunner pid=2122883) 'warmup_style': None, (TaskRunner pid=2122883) 'weight_decay': 0.01}, (TaskRunner pid=2122883) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2122883) 'clip_cov_lb': 1.0, (TaskRunner pid=2122883) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2122883) 'clip_cov_ub': 5.0, (TaskRunner pid=2122883) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2122883) 'loss_mode': 'rlsd', (TaskRunner pid=2122883) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2122883) 'ppo_epochs': 1, (TaskRunner pid=2122883) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2122883) 'ppo_micro_batch_size': None, (TaskRunner pid=2122883) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2122883) 'ppo_mini_batch_size': 8, (TaskRunner pid=2122883) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2122883) 'all_ranks': False, (TaskRunner pid=2122883) 'enable': False, (TaskRunner pid=2122883) 'ranks': [], (TaskRunner pid=2122883) 'save_path': 'outputs/profile', (TaskRunner pid=2122883) 'tool': None, (TaskRunner pid=2122883) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2122883) 'analysis': True, (TaskRunner pid=2122883) 'contents': [], (TaskRunner pid=2122883) 'discrete': False, (TaskRunner pid=2122883) 'level': 'level0'}, (TaskRunner pid=2122883) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2122883) 'discrete': False}, (TaskRunner pid=2122883) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2122883) 'step_end': None, (TaskRunner pid=2122883) 'step_start': 0}, (TaskRunner pid=2122883) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2122883) 'stack_depth': 32, (TaskRunner pid=2122883) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2122883) 'rollout_n': 8, (TaskRunner pid=2122883) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2122883) 'mode': 'disabled', (TaskRunner pid=2122883) 'record_file': None, (TaskRunner pid=2122883) 'replay_file': None}, (TaskRunner pid=2122883) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2122883) 'alpha': 0.5, (TaskRunner pid=2122883) 'distillation_add_tail': True, (TaskRunner pid=2122883) 'distillation_topk': 100, (TaskRunner pid=2122883) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2122883) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2122883) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2122883) 'enable': False, (TaskRunner pid=2122883) 'loss_weight': 0.0, (TaskRunner pid=2122883) 'mask': 'all'}, (TaskRunner pid=2122883) 'feedback_template': '\n' (TaskRunner pid=2122883) 'The ' (TaskRunner pid=2122883) 'following ' (TaskRunner pid=2122883) 'is ' (TaskRunner pid=2122883) 'feedback ' (TaskRunner pid=2122883) 'from ' (TaskRunner pid=2122883) 'your ' (TaskRunner pid=2122883) 'unsuccessful ' (TaskRunner pid=2122883) 'earlier ' (TaskRunner pid=2122883) 'attempt:\n' (TaskRunner pid=2122883) '\n' (TaskRunner pid=2122883) '{feedback_raw}', (TaskRunner pid=2122883) 'full_logit_distillation': True, (TaskRunner pid=2122883) 'include_environment_feedback': True, (TaskRunner pid=2122883) 'is_clip': 2, (TaskRunner pid=2122883) 'max_reprompt_len': 10240, (TaskRunner pid=2122883) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2122883) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2122883) '\n' (TaskRunner pid=2122883) 'Correctly ' (TaskRunner pid=2122883) 'solve ' (TaskRunner pid=2122883) 'the ' (TaskRunner pid=2122883) 'original ' (TaskRunner pid=2122883) 'question.', (TaskRunner pid=2122883) 'reprompt_truncation': 'right', (TaskRunner pid=2122883) 'solution_template': '\n' (TaskRunner pid=2122883) 'Correct ' (TaskRunner pid=2122883) 'solution:\n' (TaskRunner pid=2122883) '\n' (TaskRunner pid=2122883) '{successful_previous_attempt}', (TaskRunner pid=2122883) 'srpo_dynamic_weighting': False, (TaskRunner pid=2122883) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2122883) 'success_reward_threshold': 0.5, (TaskRunner pid=2122883) 'teacher_regularization': 'trust-region', (TaskRunner pid=2122883) 'teacher_update_rate': 0.1, (TaskRunner pid=2122883) 'token_reweight_decay_steps': 0, (TaskRunner pid=2122883) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2122883) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2122883) 'shuffle': False, (TaskRunner pid=2122883) 'strategy': 'fsdp', (TaskRunner pid=2122883) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2122883) 'tau_neg': 1.05, (TaskRunner pid=2122883) 'tau_pos': 1.0, (TaskRunner pid=2122883) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2122883) 'use_dynamic_bsz': False, (TaskRunner pid=2122883) 'use_fused_kernels': False, (TaskRunner pid=2122883) 'use_kl_loss': False, (TaskRunner pid=2122883) 'use_prefix_grouper': False, (TaskRunner pid=2122883) 'use_remove_padding': True, (TaskRunner pid=2122883) 'use_torch_compile': True}, (TaskRunner pid=2122883) 'hybrid_engine': True, (TaskRunner pid=2122883) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2122883) 'custom_chat_template': None, (TaskRunner pid=2122883) 'enable_activation_offload': False, (TaskRunner pid=2122883) 'enable_gradient_checkpointing': True, (TaskRunner pid=2122883) 'exclude_modules': None, (TaskRunner pid=2122883) 'external_lib': None, (TaskRunner pid=2122883) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2122883) 'hf_config_path': None, (TaskRunner pid=2122883) 'lora_adapter_path': None, (TaskRunner pid=2122883) 'lora_alpha': 16, (TaskRunner pid=2122883) 'lora_rank': 0, (TaskRunner pid=2122883) 'override_config': {}, (TaskRunner pid=2122883) 'path': 'Qwen/Qwen3-4B', (TaskRunner pid=2122883) 'target_modules': 'all-linear', (TaskRunner pid=2122883) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2122883) 'num_shards': 4}, (TaskRunner pid=2122883) 'tokenizer_path': None, (TaskRunner pid=2122883) 'trust_remote_code': True, (TaskRunner pid=2122883) 'use_fused_kernels': False, (TaskRunner pid=2122883) 'use_liger': False, (TaskRunner pid=2122883) 'use_remove_padding': True, (TaskRunner pid=2122883) 'use_shm': False}, (TaskRunner pid=2122883) 'nccl_timeout': 600, (TaskRunner pid=2122883) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2122883) 'entropy_checkpointing': False, (TaskRunner pid=2122883) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2122883) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2122883) 'dtype': 'bfloat16', (TaskRunner pid=2122883) 'entropy_checkpointing': False, (TaskRunner pid=2122883) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2122883) 'forward_only': True, (TaskRunner pid=2122883) 'forward_prefetch': False, (TaskRunner pid=2122883) 'fsdp_size': -1, (TaskRunner pid=2122883) 'full_determinism': False, (TaskRunner pid=2122883) 'model_dtype': 'fp32', (TaskRunner pid=2122883) 'offload_policy': False, (TaskRunner pid=2122883) 'optimizer_offload': False, (TaskRunner pid=2122883) 'param_offload': False, (TaskRunner pid=2122883) 'reshard_after_forward': True, (TaskRunner pid=2122883) 'seed': 42, (TaskRunner pid=2122883) 'strategy': 'fsdp', (TaskRunner pid=2122883) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2122883) 'use_orig_params': False, (TaskRunner pid=2122883) 'use_torch_compile': True, (TaskRunner pid=2122883) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2122883) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2122883) 'log_prob_micro_batch_size': None, (TaskRunner pid=2122883) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2122883) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2122883) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2122883) 'all_ranks': False, (TaskRunner pid=2122883) 'enable': False, (TaskRunner pid=2122883) 'ranks': [], (TaskRunner pid=2122883) 'save_path': 'outputs/profile', (TaskRunner pid=2122883) 'tool': None, (TaskRunner pid=2122883) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2122883) 'analysis': True, (TaskRunner pid=2122883) 'contents': [], (TaskRunner pid=2122883) 'discrete': False, (TaskRunner pid=2122883) 'level': 'level0'}, (TaskRunner pid=2122883) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2122883) 'discrete': False}, (TaskRunner pid=2122883) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2122883) 'step_end': None, (TaskRunner pid=2122883) 'step_start': 0}, (TaskRunner pid=2122883) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2122883) 'stack_depth': 32, (TaskRunner pid=2122883) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2122883) 'rollout_n': 8, (TaskRunner pid=2122883) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2122883) 'mode': 'disabled', (TaskRunner pid=2122883) 'record_file': None, (TaskRunner pid=2122883) 'replay_file': None}, (TaskRunner pid=2122883) 'strategy': 'fsdp', (TaskRunner pid=2122883) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2122883) 'use_torch_compile': True}, (TaskRunner pid=2122883) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2122883) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2122883) 'agent_loop_config_path': None, (TaskRunner pid=2122883) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2122883) 'name': None, (TaskRunner pid=2122883) 'path': None}, (TaskRunner pid=2122883) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2122883) 'num_workers': 8}, (TaskRunner pid=2122883) 'calculate_log_probs': True, (TaskRunner pid=2122883) 'cudagraph_capture_sizes': None, (TaskRunner pid=2122883) 'data_parallel_size': 1, (TaskRunner pid=2122883) 'disable_log_stats': True, (TaskRunner pid=2122883) 'do_sample': True, (TaskRunner pid=2122883) 'dtype': 'bfloat16', (TaskRunner pid=2122883) 'enable_chunked_prefill': True, (TaskRunner pid=2122883) 'enable_prefix_caching': True, (TaskRunner pid=2122883) 'enable_rollout_routing_replay': False, (TaskRunner pid=2122883) 'enforce_eager': False, (TaskRunner pid=2122883) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2122883) 'expert_parallel_size': 1, (TaskRunner pid=2122883) 'free_cache_engine': True, (TaskRunner pid=2122883) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2122883) 'ignore_eos': False, (TaskRunner pid=2122883) 'layered_summon': False, (TaskRunner pid=2122883) 'load_format': 'dummy', (TaskRunner pid=2122883) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2122883) 'log_prob_micro_batch_size': None, (TaskRunner pid=2122883) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2122883) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2122883) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2122883) 'max_model_len': 10240, (TaskRunner pid=2122883) 'max_num_batched_tokens': 10240, (TaskRunner pid=2122883) 'max_num_seqs': 1024, (TaskRunner pid=2122883) 'mode': 'async', (TaskRunner pid=2122883) 'multi_stage_wake_up': False, (TaskRunner pid=2122883) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2122883) 'enable': False, (TaskRunner pid=2122883) 'format': 'hermes', (TaskRunner pid=2122883) 'interaction_config_path': None, (TaskRunner pid=2122883) 'max_assistant_turns': None, (TaskRunner pid=2122883) 'max_parallel_calls': 1, (TaskRunner pid=2122883) 'max_tool_response_length': 256, (TaskRunner pid=2122883) 'max_user_turns': None, (TaskRunner pid=2122883) 'num_repeat_rollouts': None, (TaskRunner pid=2122883) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2122883) 'tool_config_path': None, (TaskRunner pid=2122883) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2122883) 'use_inference_chat_template': False}, (TaskRunner pid=2122883) 'n': 8, (TaskRunner pid=2122883) 'name': 'vllm', (TaskRunner pid=2122883) 'over_sample_rate': 0, (TaskRunner pid=2122883) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2122883) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2122883) 'all_ranks': False, (TaskRunner pid=2122883) 'enable': False, (TaskRunner pid=2122883) 'ranks': [], (TaskRunner pid=2122883) 'save_path': 'outputs/profile', (TaskRunner pid=2122883) 'tool': None, (TaskRunner pid=2122883) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2122883) 'analysis': True, (TaskRunner pid=2122883) 'contents': [], (TaskRunner pid=2122883) 'discrete': False, (TaskRunner pid=2122883) 'level': 'level0'}, (TaskRunner pid=2122883) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2122883) 'discrete': False}, (TaskRunner pid=2122883) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2122883) 'step_end': None, (TaskRunner pid=2122883) 'step_start': 0}, (TaskRunner pid=2122883) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2122883) 'stack_depth': 32, (TaskRunner pid=2122883) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2122883) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2122883) 'enable': False, (TaskRunner pid=2122883) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2122883) 'port': 9090, (TaskRunner pid=2122883) 'served_model_name': 'Qwen/Qwen3-4B'}, (TaskRunner pid=2122883) 'prompt_length': 2048, (TaskRunner pid=2122883) 'quantization': None, (TaskRunner pid=2122883) 'quantization_config_file': None, (TaskRunner pid=2122883) 'response_length': 8192, (TaskRunner pid=2122883) 'scheduling_policy': 'fcfs', (TaskRunner pid=2122883) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2122883) 'skip_rollout': False, (TaskRunner pid=2122883) 'skip_tokenizer_init': True, (TaskRunner pid=2122883) 'temperature': 1.0, (TaskRunner pid=2122883) 'tensor_model_parallel_size': 2, (TaskRunner pid=2122883) 'top_k': -1, (TaskRunner pid=2122883) 'top_p': 1.0, (TaskRunner pid=2122883) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2122883) 'backend': None, (TaskRunner pid=2122883) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2122883) 'token2text': False}, (TaskRunner pid=2122883) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2122883) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2122883) 'do_sample': True, (TaskRunner pid=2122883) 'n': 16, (TaskRunner pid=2122883) 'temperature': 0.6, (TaskRunner pid=2122883) 'top_k': -1, (TaskRunner pid=2122883) 'top_p': 0.95}}}, (TaskRunner pid=2122883) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2122883) 'adv_estimator': 'grpo', (TaskRunner pid=2122883) 'gamma': 1.0, (TaskRunner pid=2122883) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2122883) 'horizon': 10000, (TaskRunner pid=2122883) 'kl_coef': 0.001, (TaskRunner pid=2122883) 'target_kl': 0.1, (TaskRunner pid=2122883) 'type': 'fixed'}, (TaskRunner pid=2122883) 'kl_penalty': 'kl', (TaskRunner pid=2122883) 'lam': 1.0, (TaskRunner pid=2122883) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2122883) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2122883) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2122883) 'loss_type': 'ppo_clip', (TaskRunner pid=2122883) 'rollout_is': 'token', (TaskRunner pid=2122883) 'rollout_is_batch_normalize': False, (TaskRunner pid=2122883) 'rollout_is_threshold': 2.0, (TaskRunner pid=2122883) 'rollout_rs': None, (TaskRunner pid=2122883) 'rollout_rs_threshold': None}, (TaskRunner pid=2122883) 'use_kl_in_reward': False, (TaskRunner pid=2122883) 'use_pf_ppo': False}, (TaskRunner pid=2122883) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2122883) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2122883) 'async_save': False, (TaskRunner pid=2122883) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2122883) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2122883) 'cliprange_value': 0.5, (TaskRunner pid=2122883) 'data_loader_seed': 42, (TaskRunner pid=2122883) 'enable': None, (TaskRunner pid=2122883) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2122883) 'forward_micro_batch_size': None, (TaskRunner pid=2122883) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2122883) 'grad_clip': 1.0, (TaskRunner pid=2122883) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2122883) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2122883) 'enable_activation_offload': False, (TaskRunner pid=2122883) 'enable_gradient_checkpointing': True, (TaskRunner pid=2122883) 'external_lib': None, (TaskRunner pid=2122883) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2122883) 'dtype': 'bfloat16', (TaskRunner pid=2122883) 'entropy_checkpointing': False, (TaskRunner pid=2122883) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2122883) 'forward_only': False, (TaskRunner pid=2122883) 'forward_prefetch': False, (TaskRunner pid=2122883) 'fsdp_size': -1, (TaskRunner pid=2122883) 'full_determinism': False, (TaskRunner pid=2122883) 'model_dtype': 'fp32', (TaskRunner pid=2122883) 'offload_policy': False, (TaskRunner pid=2122883) 'optimizer_offload': False, (TaskRunner pid=2122883) 'param_offload': False, (TaskRunner pid=2122883) 'reshard_after_forward': True, (TaskRunner pid=2122883) 'seed': 42, (TaskRunner pid=2122883) 'strategy': 'fsdp', (TaskRunner pid=2122883) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2122883) 'use_orig_params': False, (TaskRunner pid=2122883) 'use_torch_compile': True, (TaskRunner pid=2122883) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2122883) 'lora_alpha': 16, (TaskRunner pid=2122883) 'lora_rank': 0, (TaskRunner pid=2122883) 'override_config': {}, (TaskRunner pid=2122883) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2122883) 'target_modules': 'all-linear', (TaskRunner pid=2122883) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2122883) 'tokenizer_path': 'Qwen/Qwen3-4B', (TaskRunner pid=2122883) 'trust_remote_code': True, (TaskRunner pid=2122883) 'use_remove_padding': False, (TaskRunner pid=2122883) 'use_shm': False}, (TaskRunner pid=2122883) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2122883) 'betas': [0.9, 0.999], (TaskRunner pid=2122883) 'clip_grad': 1.0, (TaskRunner pid=2122883) 'lr': 1e-05, (TaskRunner pid=2122883) 'lr_scheduler_type': 'constant', (TaskRunner pid=2122883) 'lr_warmup_steps': -1, (TaskRunner pid=2122883) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2122883) 'min_lr_ratio': 0.0, (TaskRunner pid=2122883) 'num_cycles': 0.5, (TaskRunner pid=2122883) 'optimizer': 'AdamW', (TaskRunner pid=2122883) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2122883) 'override_optimizer_config': None, (TaskRunner pid=2122883) 'total_training_steps': -1, (TaskRunner pid=2122883) 'warmup_style': None, (TaskRunner pid=2122883) 'weight_decay': 0.01}, (TaskRunner pid=2122883) 'ppo_epochs': 1, (TaskRunner pid=2122883) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2122883) 'ppo_micro_batch_size': None, (TaskRunner pid=2122883) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2122883) 'ppo_mini_batch_size': 8, (TaskRunner pid=2122883) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2122883) 'all_ranks': False, (TaskRunner pid=2122883) 'enable': False, (TaskRunner pid=2122883) 'ranks': [], (TaskRunner pid=2122883) 'save_path': 'outputs/profile', (TaskRunner pid=2122883) 'tool': None, (TaskRunner pid=2122883) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2122883) 'analysis': True, (TaskRunner pid=2122883) 'contents': [], (TaskRunner pid=2122883) 'discrete': False, (TaskRunner pid=2122883) 'level': 'level0'}, (TaskRunner pid=2122883) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2122883) 'discrete': False}, (TaskRunner pid=2122883) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2122883) 'step_end': None, (TaskRunner pid=2122883) 'step_start': 0}, (TaskRunner pid=2122883) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2122883) 'stack_depth': 32, (TaskRunner pid=2122883) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2122883) 'rollout_n': 8, (TaskRunner pid=2122883) 'shuffle': False, (TaskRunner pid=2122883) 'strategy': 'fsdp', (TaskRunner pid=2122883) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2122883) 'use_dynamic_bsz': False}, (TaskRunner pid=2122883) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2122883) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2122883) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2122883) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2122883) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2122883) 'dataloader_num_workers': 8, (TaskRunner pid=2122883) 'filter_overlong_prompts': True, (TaskRunner pid=2122883) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2122883) 'image_key': 'images', (TaskRunner pid=2122883) 'image_patch_size': 14, (TaskRunner pid=2122883) 'max_prompt_length': 2048, (TaskRunner pid=2122883) 'max_response_length': 8192, (TaskRunner pid=2122883) 'prompt_key': 'prompt', (TaskRunner pid=2122883) 'return_full_prompt': False, (TaskRunner pid=2122883) 'return_multi_modal_inputs': True, (TaskRunner pid=2122883) 'return_raw_chat': True, (TaskRunner pid=2122883) 'return_raw_input_ids': False, (TaskRunner pid=2122883) 'reward_fn_key': 'data_source', (TaskRunner pid=2122883) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2122883) 'seed': None, (TaskRunner pid=2122883) 'shuffle': True, (TaskRunner pid=2122883) 'tokenizer': None, (TaskRunner pid=2122883) 'tool_config_path': None, (TaskRunner pid=2122883) 'train_batch_size': 32, (TaskRunner pid=2122883) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet'], (TaskRunner pid=2122883) 'train_max_samples': 3200, (TaskRunner pid=2122883) 'truncation': 'error', (TaskRunner pid=2122883) 'trust_remote_code': True, (TaskRunner pid=2122883) 'use_shm': False, (TaskRunner pid=2122883) 'val_batch_size': None, (TaskRunner pid=2122883) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet'], (TaskRunner pid=2122883) 'val_max_samples': -1, (TaskRunner pid=2122883) 'validation_shuffle': False, (TaskRunner pid=2122883) 'video_key': 'videos'}, (TaskRunner pid=2122883) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2122883) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2122883) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2122883) 'cuda-memory-usage': 'true', (TaskRunner pid=2122883) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2122883) 'discrete': False, (TaskRunner pid=2122883) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2122883) 'capture-range-end': None, (TaskRunner pid=2122883) 'cuda-graph-trace': 'graph', (TaskRunner pid=2122883) 'cuda-memory-usage': 'true', (TaskRunner pid=2122883) 'kill': 'none', (TaskRunner pid=2122883) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2122883) 'torch_memory': {'context': 'all', (TaskRunner pid=2122883) 'kw_args': {}, (TaskRunner pid=2122883) 'stack_depth': 32, (TaskRunner pid=2122883) 'stacks': 'all', (TaskRunner pid=2122883) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2122883) 'profile_continuous_steps': False, (TaskRunner pid=2122883) 'save_path': 'outputs/profile', (TaskRunner pid=2122883) 'steps': None, (TaskRunner pid=2122883) 'tool': None}, (TaskRunner pid=2122883) 'max_model_len': 10240, (TaskRunner pid=2122883) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_biology_rlsd_tr_Qwen3_4B', (TaskRunner pid=2122883) 'include_dashboard': False, (TaskRunner pid=2122883) 'num_cpus': None, (TaskRunner pid=2122883) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2122883) 'TASK': 'datasets/sciknoweval/biology', (TaskRunner pid=2122883) 'USER': 'root'}}}, (TaskRunner pid=2122883) 'timeline_json_file': None}, (TaskRunner pid=2122883) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2122883) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2122883) 'name': 'custom_reward_manager', (TaskRunner pid=2122883) 'path': None}, (TaskRunner pid=2122883) 'name': 'naive', (TaskRunner pid=2122883) 'source': 'register'}, (TaskRunner pid=2122883) 'reward_model': {'enable': False, (TaskRunner pid=2122883) 'enable_resource_pool': False, (TaskRunner pid=2122883) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2122883) 'launch_reward_fn_async': False, (TaskRunner pid=2122883) 'max_length': None, (TaskRunner pid=2122883) 'micro_batch_size': None, (TaskRunner pid=2122883) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2122883) 'model': {'external_lib': None, (TaskRunner pid=2122883) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2122883) 'forward_prefetch': False, (TaskRunner pid=2122883) 'fsdp_size': -1, (TaskRunner pid=2122883) 'param_offload': False, (TaskRunner pid=2122883) 'reshard_after_forward': True, (TaskRunner pid=2122883) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2122883) 'input_tokenizer': 'Qwen/Qwen3-4B', (TaskRunner pid=2122883) 'override_config': {}, (TaskRunner pid=2122883) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2122883) 'trust_remote_code': False, (TaskRunner pid=2122883) 'use_fused_kernels': False, (TaskRunner pid=2122883) 'use_remove_padding': False, (TaskRunner pid=2122883) 'use_shm': False}, (TaskRunner pid=2122883) 'n_gpus_per_node': 8, (TaskRunner pid=2122883) 'nnodes': 0, (TaskRunner pid=2122883) 'num_workers': 1, (TaskRunner pid=2122883) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2122883) 'all_ranks': False, (TaskRunner pid=2122883) 'enable': False, (TaskRunner pid=2122883) 'ranks': [], (TaskRunner pid=2122883) 'save_path': 'outputs/profile', (TaskRunner pid=2122883) 'tool': None, (TaskRunner pid=2122883) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2122883) 'analysis': True, (TaskRunner pid=2122883) 'contents': [], (TaskRunner pid=2122883) 'discrete': False, (TaskRunner pid=2122883) 'level': 'level0'}, (TaskRunner pid=2122883) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2122883) 'discrete': False}, (TaskRunner pid=2122883) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2122883) 'step_end': None, (TaskRunner pid=2122883) 'step_start': 0}, (TaskRunner pid=2122883) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2122883) 'stack_depth': 32, (TaskRunner pid=2122883) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2122883) 'reward_loop_class_name': None, (TaskRunner pid=2122883) 'reward_loop_module_path': None, (TaskRunner pid=2122883) 'reward_loop_source': 'register', (TaskRunner pid=2122883) 'reward_manager': 'naive', (TaskRunner pid=2122883) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2122883) 'cudagraph_capture_sizes': None, (TaskRunner pid=2122883) 'data_parallel_size': 1, (TaskRunner pid=2122883) 'disable_log_stats': True, (TaskRunner pid=2122883) 'dtype': 'bfloat16', (TaskRunner pid=2122883) 'enable_chunked_prefill': True, (TaskRunner pid=2122883) 'enable_prefix_caching': True, (TaskRunner pid=2122883) 'enforce_eager': True, (TaskRunner pid=2122883) 'engine_kwargs': {}, (TaskRunner pid=2122883) 'expert_parallel_size': 1, (TaskRunner pid=2122883) 'free_cache_engine': True, (TaskRunner pid=2122883) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2122883) 'limit_images': None, (TaskRunner pid=2122883) 'load_format': 'auto', (TaskRunner pid=2122883) 'max_model_len': None, (TaskRunner pid=2122883) 'max_num_batched_tokens': 8192, (TaskRunner pid=2122883) 'max_num_seqs': 1024, (TaskRunner pid=2122883) 'name': '???', (TaskRunner pid=2122883) 'prompt_length': 2048, (TaskRunner pid=2122883) 'response_length': 2048, (TaskRunner pid=2122883) 'skip_tokenizer_init': False, (TaskRunner pid=2122883) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2122883) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2122883) 'memory_limit_mb': 1024, (TaskRunner pid=2122883) 'url': None}, (TaskRunner pid=2122883) 'strategy': 'fsdp', (TaskRunner pid=2122883) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2122883) 'use_dynamic_bsz': False, (TaskRunner pid=2122883) 'use_reward_loop': False}, (TaskRunner pid=2122883) 'trainer': {'balance_batch': True, (TaskRunner pid=2122883) 'critic_warmup': 0, (TaskRunner pid=2122883) 'default_hdfs_dir': None, (TaskRunner pid=2122883) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2122883) 'del_local_ckpt_after_load': False, (TaskRunner pid=2122883) 'device': 'cuda', (TaskRunner pid=2122883) 'esi_redundant_time': 0, (TaskRunner pid=2122883) 'experiment_name': 'qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2122883) 'group_name': 'QWEN3-RLSD-TR-GRPO-matched-generalization', (TaskRunner pid=2122883) 'log_val_generations': 0, (TaskRunner pid=2122883) 'logger': ['console', 'wandb'], (TaskRunner pid=2122883) 'max_actor_ckpt_to_keep': 1, (TaskRunner pid=2122883) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2122883) 'n_gpus_per_node': 8, (TaskRunner pid=2122883) 'nnodes': 1, (TaskRunner pid=2122883) 'project_name': 'SDPO-root', (TaskRunner pid=2122883) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2122883) 'resume_from_path': None, (TaskRunner pid=2122883) 'resume_mode': 'auto', (TaskRunner pid=2122883) 'rollout_data_dir': None, (TaskRunner pid=2122883) 'save_freq': 10, (TaskRunner pid=2122883) 'test_freq': 10, (TaskRunner pid=2122883) 'total_epochs': 30, (TaskRunner pid=2122883) 'total_training_steps': 100, (TaskRunner pid=2122883) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2122883) 'val_before_train': False, (TaskRunner pid=2122883) 'val_only': False, (TaskRunner pid=2122883) 'validation_data_dir': None}, (TaskRunner pid=2122883) 'transfer_queue': {'enable': False}, (TaskRunner pid=2122883) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/biology', (TaskRunner pid=2122883) 'dir': '/users/root/SDPO', (TaskRunner pid=2122883) 'log_dir': '/users/root/output', (TaskRunner pid=2122883) 'task': 'datasets/sciknoweval/biology'}} (TaskRunner pid=2122883) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2122883) use_critic=need_critic(config), (TaskRunner pid=2122883) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2122883) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2122883) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2122883) Using dataset class: RLHFDataset (TaskRunner pid=2122883) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2122883) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2122883) dataset len: 450 (TaskRunner pid=2122883) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2122883) WARNING:2026-07-02 11:37:25,419:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2122883) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/450 [00:00 (TaskRunner pid=2122883) bind role actor_rollout_ref method chat_completion to class .WorkerDict'> (TaskRunner pid=2122883) bind role actor_rollout_ref method generate to class .WorkerDict'> (TaskRunner pid=2122883) bind role actor_rollout_ref method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2122883) bind role actor_rollout_ref method sleep to class .WorkerDict'> (TaskRunner pid=2122883) bind role actor_rollout_ref method wake_up to class .WorkerDict'> (TaskRunner pid=2122883) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 50/50 [00:00<00:00, 75.24 examples/s] (TaskRunner pid=2122883) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2122883) self.use_critic = need_critic(self.config) (WorkerDict pid=2123259) [W702 11:37:33.058108451 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:53649 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2123259) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2123259) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2123259) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2123266) [W702 11:37:34.147181241 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:53649 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2123259) Model config after override: Qwen3Config { (WorkerDict pid=2123259) "architectures": [ (WorkerDict pid=2123259) "Qwen3ForCausalLM" (WorkerDict pid=2123259) ], (WorkerDict pid=2123259) "attention_bias": false, (WorkerDict pid=2123259) "attention_dropout": 0.0, (WorkerDict pid=2123259) "dtype": "bfloat16", (WorkerDict pid=2123259) "eos_token_id": 151645, (WorkerDict pid=2123259) "head_dim": 128, (WorkerDict pid=2123259) "hidden_act": "silu", (WorkerDict pid=2123259) "hidden_size": 2560, (WorkerDict pid=2123259) "initializer_range": 0.02, (WorkerDict pid=2123259) "intermediate_size": 9728, (WorkerDict pid=2123259) "layer_types": [ (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention" (WorkerDict pid=2123259) ], (WorkerDict pid=2123259) "max_position_embeddings": 40960, (WorkerDict pid=2123259) "max_window_layers": 36, (WorkerDict pid=2123259) "model_type": "qwen3", (WorkerDict pid=2123259) "num_attention_heads": 32, (WorkerDict pid=2123259) "num_hidden_layers": 36, (WorkerDict pid=2123259) "num_key_value_heads": 8, (WorkerDict pid=2123259) "pad_token_id": 151643, (WorkerDict pid=2123259) "rms_norm_eps": 1e-06, (WorkerDict pid=2123259) "rope_scaling": null, (WorkerDict pid=2123259) "rope_theta": 1000000, (WorkerDict pid=2123259) "sliding_window": null, (WorkerDict pid=2123259) "tie_word_embeddings": true, (WorkerDict pid=2123259) "transformers_version": "4.57.1", (WorkerDict pid=2123259) "use_cache": true, (WorkerDict pid=2123259) "use_sliding_window": false, (WorkerDict pid=2123259) "vocab_size": 151936 (WorkerDict pid=2123259) } (WorkerDict pid=2123259) (WorkerDict pid=2123264) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2123264) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2123264) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2123264) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2123259) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2123259) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2123259) Actor use_remove_padding=True (WorkerDict pid=2123259) Actor use_fused_kernels=False (WorkerDict pid=2123259) Actor use_prefix_grouper=False (WorkerDict pid=2123265) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2123265) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2123259) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2123259) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2123265) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.95s/it] [repeated 13x across cluster] (WorkerDict pid=2123265) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.99s/it] [repeated 5x across cluster] (WorkerDict pid=2123263) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.95s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.98s/it] (WorkerDict pid=2123260) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2123260) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2123260) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2123266) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2123266) warnings.warn( (WorkerDict pid=2123265) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2123265) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2123259) reference model: Qwen/Qwen3-4B (WorkerDict pid=2123263) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 15x across cluster] (WorkerDict pid=2123263) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 15x across cluster] (WorkerDict pid=2123259) Model config after override: Qwen3Config { (WorkerDict pid=2123259) "architectures": [ (WorkerDict pid=2123259) "Qwen3ForCausalLM" (WorkerDict pid=2123259) ], (WorkerDict pid=2123259) "attention_bias": false, (WorkerDict pid=2123259) "attention_dropout": 0.0, (WorkerDict pid=2123259) "dtype": "bfloat16", (WorkerDict pid=2123259) "eos_token_id": 151645, (WorkerDict pid=2123259) "head_dim": 128, (WorkerDict pid=2123259) "hidden_act": "silu", (WorkerDict pid=2123259) "hidden_size": 2560, (WorkerDict pid=2123259) "initializer_range": 0.02, (WorkerDict pid=2123259) "intermediate_size": 9728, (WorkerDict pid=2123259) "layer_types": [ (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention" (WorkerDict pid=2123259) ], (WorkerDict pid=2123259) "max_position_embeddings": 40960, (WorkerDict pid=2123259) "max_window_layers": 36, (WorkerDict pid=2123259) "model_type": "qwen3", (WorkerDict pid=2123259) "num_attention_heads": 32, (WorkerDict pid=2123259) "num_hidden_layers": 36, (WorkerDict pid=2123259) "num_key_value_heads": 8, (WorkerDict pid=2123259) "pad_token_id": 151643, (WorkerDict pid=2123259) "rms_norm_eps": 1e-06, (WorkerDict pid=2123259) "rope_scaling": null, (WorkerDict pid=2123259) "rope_theta": 1000000, (WorkerDict pid=2123259) "sliding_window": null, (WorkerDict pid=2123259) "tie_word_embeddings": true, (WorkerDict pid=2123259) "transformers_version": "4.57.1", (WorkerDict pid=2123259) "use_cache": true, (WorkerDict pid=2123259) "use_sliding_window": false, (WorkerDict pid=2123259) "vocab_size": 151936 (WorkerDict pid=2123259) } (WorkerDict pid=2123259) (WorkerDict pid=2123259) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2123259) Ref use_remove_padding=True (WorkerDict pid=2123259) Ref use_fused_kernels=False (WorkerDict pid=2123259) Ref use_prefix_grouper=False (WorkerDict pid=2123266) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2123266) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (TaskRunner pid=2122883) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2122883) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2123266) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.86s/it] [repeated 15x across cluster] (WorkerDict pid=2123265) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.92s/it] [repeated 6x across cluster] (vLLMHttpServer pid=2124149) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2124149) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2124149) ['serve', (vLLMHttpServer pid=2124149) 'Qwen/Qwen3-4B', (vLLMHttpServer pid=2124149) '--dtype', (vLLMHttpServer pid=2124149) 'bfloat16', (vLLMHttpServer pid=2124149) '--load_format', (vLLMHttpServer pid=2124149) 'dummy', (vLLMHttpServer pid=2124149) '--trust_remote_code', (vLLMHttpServer pid=2124149) '--max_model_len', (vLLMHttpServer pid=2124149) '40960', (vLLMHttpServer pid=2124149) '--max_num_seqs', (vLLMHttpServer pid=2124149) '1024', (vLLMHttpServer pid=2124149) '--enable_chunked_prefill', (vLLMHttpServer pid=2124149) '--max_num_batched_tokens', (vLLMHttpServer pid=2124149) '10240', (vLLMHttpServer pid=2124149) '--enable_prefix_caching', (vLLMHttpServer pid=2124149) '--enable_sleep_mode', (vLLMHttpServer pid=2124149) '--logprobs_mode', (vLLMHttpServer pid=2124149) 'processed_logprobs', (vLLMHttpServer pid=2124149) '--disable_custom_all_reduce', (vLLMHttpServer pid=2124149) '--gpu_memory_utilization', (vLLMHttpServer pid=2124149) '0.8', (vLLMHttpServer pid=2124149) '--disable_log_stats', (vLLMHttpServer pid=2124149) '--tensor_parallel_size', (vLLMHttpServer pid=2124149) '2', (vLLMHttpServer pid=2124149) '--seed', (vLLMHttpServer pid=2124149) '0', (vLLMHttpServer pid=2124149) '--override_generation_config', (vLLMHttpServer pid=2124149) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2124149) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2124149) '--hf_overrides', (vLLMHttpServer pid=2124149) '{}', (vLLMHttpServer pid=2124149) '--scheduling_policy', (vLLMHttpServer pid=2124149) 'fcfs'] (vLLMHttpServer pid=2124149) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2124149) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2124149) WARNING 07-02 11:38:30 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2123259) WARNING 07-02 11:38:39 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2124150) WARNING 07-02 11:38:30 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2123261) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2123261) [rank2]:W0702 11:38:49.111000 2123261 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph (WorkerDict pid=2123261) [rank2]:W0702 11:38:49.111000 2123261 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last): (WorkerDict pid=2123261) [rank2]:W0702 11:38:49.111000 2123261 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates (WorkerDict pid=2123261) [rank2]:W0702 11:38:49.111000 2123261 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f: (WorkerDict pid=2123261) [rank2]:W0702 11:38:49.111000 2123261 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (WorkerDict pid=2123261) [rank2]:W0702 11:38:49.111000 2123261 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/fxgraph/vs/fvscv5dg7y3y3dcup4ddfpgbmpglg73u5zi54vpf2q4orgekbtq7/.2123259.129147915196096.tmp' (vLLMHttpServer pid=2124150) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 3x across cluster] (vLLMHttpServer pid=2124150) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 3x across cluster] (vLLMHttpServer pid=2124150) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2124150) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2123261) 2026-07-02 11:38:53,118 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2123261) 2026-07-02 11:38:53,134 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2123259) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2122883) (TaskRunner pid=2122883) A (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) user (TaskRunner pid=2122883) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2122883) (TaskRunner pid=2122883) A: -0.01 (TaskRunner pid=2122883) B: 1.69 (TaskRunner pid=2122883) C: 2.49 (TaskRunner pid=2122883) D: 0.45 (TaskRunner pid=2122883) Please reason step by step. (TaskRunner pid=2122883) assistant (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) [response] (TaskRunner pid=2122883) The folding stability score of a protein sequence is typically determined through computational methods or experimental techniques such as circular dichroism, NMR, or molecular dynamics simulations. However, without specific computational tools or experimental data provided in the question, it is not possible to calculate or determine the folding stability score for the given protein sequence. Therefore, the correct answer cannot be determined from the information provided. (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) A (TaskRunner pid=2122883) (TaskRunner pid=2122883) [ground_truth] B (TaskRunner pid=2122883) [score] 0.0 (TaskRunner pid=2122883) [acc] 0.0 (TaskRunner pid=2122883) [pred] A (TaskRunner pid=2122883) [incorrect_format] 1 (TaskRunner pid=2122883) [feedback] (TaskRunner pid=2122883) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2122883) step:10 - global_seqlen/min:11726 - global_seqlen/max:24851 - global_seqlen/minmax_diff:13125 - global_seqlen/balanced_min:17477 - global_seqlen/balanced_max:17949 - global_seqlen/mean:17600.125 - actor/entropy:0.20009024441242218 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8149087429046631 - training/rollout_probs_diff_mean:0.005511445924639702 - training/rollout_probs_diff_std:0.016095276921987534 - training/rollout_actor_probs_pearson_corr:0.9971664547920227 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.60546875 - self_distillation/correct_sample_fraction:0.2734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.60546875 - rollout_corr/rollout_is_mean:1.0001614093780518 - rollout_corr/rollout_is_ratio_fraction_high:4.249713310855441e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012749139568768442 - rollout_corr/rollout_is_max:3.9662325382232666 - rollout_corr/rollout_is_min:0.003977110609412193 - rollout_corr/rollout_is_std:0.045301079750061035 - rollout_corr/rollout_is_eff_sample_size:0.9979527272601818 - rollout_corr/rollout_is_seq_mean:1.0003551244735718 - rollout_corr/rollout_is_seq_std:0.0033447842579334974 - rollout_corr/rollout_is_seq_max:1.0103979110717773 - rollout_corr/rollout_is_seq_min:0.991142749786377 - rollout_corr/rollout_is_seq_max_deviation:0.010397911071777344 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2529795560985804) - rollout_corr/training_log_ppl:np.float64(0.2225916018069256) - rollout_corr/kl:np.float64(0.001012606808284744) - rollout_corr/k3_kl:np.float64(0.0014178051028466143) - rollout_corr/rollout_ppl:np.float64(1.2516679712571204) - rollout_corr/rollout_log_ppl:np.float64(0.22157899395097047) - rollout_corr/log_ppl_diff:np.float64(0.0010126078559551388) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031039943860378116) - rollout_corr/log_ppl_diff_max:np.float64(0.011974595487117767) - rollout_corr/log_ppl_diff_min:np.float64(-0.011962682008743286) - rollout_corr/ppl_ratio:np.float64(1.0010205947328359) - rollout_corr/chi2_token:np.float64(0.004173249937593937) - rollout_corr/chi2_seq:np.float64(1.0239088295493275) - actor/pg_loss:np.float64(0.007663122727535665) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016782767970653367) - actor/ppo_kl:np.float64(0.0002169081007590279) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.60546875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.60546875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.2734375) - self_distillation/token_reweight_w_mean:np.float64(510711.6452796182) - self_distillation/token_reweight_w_std:np.float64(6248755.756001078) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0036613668780774) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11369272705633193) - self_distillation/empty_target_batch:np.float64(0.39453125) - actor/grad_norm:np.float64(0.7940817475318909) - perf/mfu/actor:np.float64(0.02968673911541976) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(141.75575494766235) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.325) - val-aux/sciknoweval/reward/std@16:np.float64(0.2554249056239752) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.42513999999999996) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.24136437801418256) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.22138000000000002) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.19069531655682617) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.32382) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2551337969977601) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.5265) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.20460662938883328) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.15156000000000003) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12392120350057902) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.32012) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.20443076961937032) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.61274) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.1543338362640398) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.10611999999999998) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0743746346022705) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.31182) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.14713308978809259) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.67476) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.09348939416746192) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.07942) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04233228578656238) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.3041) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.1033212819785522) - val-aux/sciknoweval/score/mean@16:np.float64(0.325) - val-aux/sciknoweval/score/std@16:np.float64(0.2554249056239752) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.42513999999999996) - val-aux/sciknoweval/score/best@2/std:np.float64(0.24136437801418256) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.22138000000000002) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.19069531655682617) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.32382) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2551337969977601) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.5265) - val-aux/sciknoweval/score/best@4/std:np.float64(0.20460662938883328) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.15156000000000003) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12392120350057902) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.32012) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.20443076961937032) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.61274) - val-aux/sciknoweval/score/best@8/std:np.float64(0.1543338362640398) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.10611999999999998) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0743746346022705) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.31182) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.14713308978809259) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.67476) - val-aux/sciknoweval/score/best@16/std:np.float64(0.09348939416746192) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.07942) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04233228578656238) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.3041) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.1033212819785522) - val-core/sciknoweval/acc/mean@16:np.float64(0.325) - val-aux/sciknoweval/acc/std@16:np.float64(0.2554249056239752) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.42513999999999996) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.24136437801418256) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.22138000000000002) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.19069531655682617) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.32382) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2551337969977601) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.5265) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.20460662938883328) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.15156000000000003) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12392120350057902) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.32012) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.20443076961937032) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.61274) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.1543338362640398) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.10611999999999998) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0743746346022705) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.31182) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.14713308978809259) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.67476) - val-core/sciknoweval/acc/best@16/std:np.float64(0.09348939416746192) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.07942) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04233228578656238) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.3041) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.1033212819785522) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.03743490246911931) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9988800000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.011805957445723348) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9780599999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.049950316494815884) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9881399999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.038377557849678695) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9605799999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.06172430689859443) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9969) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.019827817850231286) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.93312) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.06831160926205504) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99944) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.008418479524687841) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.90012) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.06084132628023402) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999800000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0006321392251711646) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.2734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.2734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0029588628094643354 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0029588628094643354 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:275.75390625 - response_length/max:1820.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:275.75390625 - response_length_non_aborted/max:1820.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.25 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016302801668643951 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0913536846637726) - timing_s/agent_loop/generate_sequences/max:np.float64(10.443698544055223) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.7668629826584947) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.443698544055223) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1820 - timing_s/gen:16.08851490728557 - timing_s/reward:0.05504823662340641 - timing_s/old_log_prob:2.380241384729743 - timing_s/adv:0.6068263426423073 - timing_s/update_actor:16.999784832820296 - timing_s/step:36.216833928599954 - timing_s/testing:81.34854041971266 - timing_s/save_checkpoint:7.180700551718473 - timing_s/stop_profile:0.0001176241785287857 - timing_per_token_ms/gen:0.22790524424922542 - timing_per_token_ms/update_actor:0.12073625068586372 - timing_per_token_ms/adv:0.004309815574053503 - perf/total_num_tokens:140801 - perf/time_per_step:36.216833928599954 - perf/throughput:485.96531200650907 (TaskRunner pid=2122883) Training Progress: 10%|█ | 10/100 [10:52<1:45:57, 70.64s/it] (TaskRunner pid=2122883) step:11 - global_seqlen/min:16828 - global_seqlen/max:30842 - global_seqlen/minmax_diff:14014 - global_seqlen/balanced_min:21635 - global_seqlen/balanced_max:22284 - global_seqlen/mean:21730.625 - actor/entropy:0.16385675966739655 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4429733455181122 - training/rollout_probs_diff_mean:0.004776663146913052 - training/rollout_probs_diff_std:0.014988353475928307 - training/rollout_actor_probs_pearson_corr:0.9971790313720703 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:0.9997762441635132 - rollout_corr/rollout_is_ratio_fraction_high:2.2400179659598507e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001904015225591138 - rollout_corr/rollout_is_max:2.340348958969116 - rollout_corr/rollout_is_min:0.08248017728328705 - rollout_corr/rollout_is_std:0.04225890338420868 - rollout_corr/rollout_is_eff_sample_size:0.9982165965875319 - rollout_corr/rollout_is_seq_mean:0.9997829794883728 - rollout_corr/rollout_is_seq_std:0.003166553797200322 - rollout_corr/rollout_is_seq_max:1.0128588676452637 - rollout_corr/rollout_is_seq_min:0.9910963177680969 - rollout_corr/rollout_is_seq_max_deviation:0.012858867645263672 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2296020975336432) - rollout_corr/training_log_ppl:np.float64(0.20335380466713104) - rollout_corr/kl:np.float64(0.0012903355955415918) - rollout_corr/k3_kl:np.float64(0.0012305048087455361) - rollout_corr/rollout_ppl:np.float64(1.2280050399713218) - rollout_corr/rollout_log_ppl:np.float64(0.20206346827035304) - rollout_corr/log_ppl_diff:np.float64(0.0012903363967780024) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026630688516888767) - rollout_corr/log_ppl_diff_max:np.float64(0.017947301268577576) - rollout_corr/log_ppl_diff_min:np.float64(-0.010352224111557007) - rollout_corr/ppl_ratio:np.float64(1.0012970515526831) - rollout_corr/chi2_token:np.float64(0.0022916693706065416) - rollout_corr/chi2_seq:np.float64(0.6786705595441163) - actor/pg_loss:np.float64(0.007793928729370236) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017880768228906163) - actor/ppo_kl:np.float64(-4.9033289212019326e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.421875) - self_distillation/token_reweight_w_mean:np.float64(369570.61190717644) - self_distillation/token_reweight_w_std:np.float64(4874752.744324681) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011554462835193) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11396575406251941) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.7840257436037064) - perf/mfu/actor:np.float64(0.03512070663237796) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(142.33532333374023) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.016731534153223038 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.016731534153223038 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:348.76953125 - response_length/max:2422.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:348.76953125 - response_length_non_aborted/max:2422.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.3125 - prompt_length/max:500.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.0650909543037415e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0845849998295307) - timing_s/agent_loop/generate_sequences/max:np.float64(14.238213451579213) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3772238838864723) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.238213451579213) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:2422 - timing_s/gen:20.42856288701296 - timing_s/reward:0.0540193896740675 - timing_s/old_log_prob:2.60726017318666 - timing_s/adv:0.7217988539487123 - timing_s/update_actor:17.801132675260305 - timing_s/step:41.7183708678931 - timing_s/stop_profile:2.9888004064559937e-05 - timing_per_token_ms/gen:0.22880173474842314 - timing_per_token_ms/update_actor:0.10239657554292793 - timing_per_token_ms/adv:0.0041519678676333075 - perf/total_num_tokens:173845 - perf/time_per_step:41.7183708678931 - perf/throughput:520.8886288683942 (TaskRunner pid=2122883) Training Progress: 11%|█ | 11/100 [11:34<1:31:39, 61.79s/it] (TaskRunner pid=2122883) step:12 - global_seqlen/min:18510 - global_seqlen/max:42325 - global_seqlen/minmax_diff:23815 - global_seqlen/balanced_min:26152 - global_seqlen/balanced_max:32053 - global_seqlen/mean:26963.125 - actor/entropy:0.1360880583524704 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4592995345592499 - training/rollout_probs_diff_mean:0.0037858886644244194 - training/rollout_probs_diff_std:0.013429389335215092 - training/rollout_actor_probs_pearson_corr:0.9973995685577393 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.0000216960906982 - rollout_corr/rollout_is_ratio_fraction_high:7.299920980585739e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013788740034215152 - rollout_corr/rollout_is_max:2.5501325130462646 - rollout_corr/rollout_is_min:0.2923682630062103 - rollout_corr/rollout_is_std:0.03888509422540665 - rollout_corr/rollout_is_eff_sample_size:0.9984903510662826 - rollout_corr/rollout_is_seq_mean:0.9999573230743408 - rollout_corr/rollout_is_seq_std:0.002746041864156723 - rollout_corr/rollout_is_seq_max:1.0101428031921387 - rollout_corr/rollout_is_seq_min:0.9911365509033203 - rollout_corr/rollout_is_seq_max_deviation:0.010142803192138672 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2124483888037503) - rollout_corr/training_log_ppl:np.float64(0.18915603501955047) - rollout_corr/kl:np.float64(0.001068810922657093) - rollout_corr/k3_kl:np.float64(0.0012623913938227815) - rollout_corr/rollout_ppl:np.float64(1.211122840642929) - rollout_corr/rollout_log_ppl:np.float64(0.18808722433095681) - rollout_corr/log_ppl_diff:np.float64(0.0010688106885936577) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025167708918161225) - rollout_corr/log_ppl_diff_max:np.float64(0.013781130313873291) - rollout_corr/log_ppl_diff_min:np.float64(-0.011515498161315918) - rollout_corr/ppl_ratio:np.float64(1.0010748247150332) - rollout_corr/chi2_token:np.float64(0.002950052497908473) - rollout_corr/chi2_seq:np.float64(1.5056117225904018) - actor/pg_loss:np.float64(0.009150906931608915) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016923086934639286) - actor/ppo_kl:np.float64(6.213555671763515e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.421875) - self_distillation/token_reweight_w_mean:np.float64(284014.9290252649) - self_distillation/token_reweight_w_std:np.float64(4492645.565195698) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0025300320703536) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13327509218652267) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.8035765886306763) - perf/mfu/actor:np.float64(0.042627928291705836) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(142.23823928833008) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.04189850762486458 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.04189850762486458 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:481.59765625 - response_length/max:8192.0 - response_length/min:93.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:481.59765625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:361.0 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.380460500717163e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0855942126363516) - timing_s/agent_loop/generate_sequences/max:np.float64(50.55506406724453) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.47686035346851) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.55506406724453) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:435 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.429069597274065 - timing_s/reward:0.06438687071204185 - timing_s/old_log_prob:2.6902950555086136 - timing_s/adv:0.7278527393937111 - timing_s/update_actor:18.51123914681375 - timing_s/step:78.51152851432562 - timing_s/stop_profile:2.927146852016449e-05 - timing_per_token_ms/gen:0.45769752043794715 - timing_per_token_ms/update_actor:0.08581738553493776 - timing_per_token_ms/adv:0.00337429702322019 - perf/total_num_tokens:215705 - perf/time_per_step:78.51152851432562 - perf/throughput:343.42886338125703 (TaskRunner pid=2122883) Training Progress: 12%|█▏ | 12/100 [12:53<1:38:05, 66.88s/it] (TaskRunner pid=2122883) step:13 - global_seqlen/min:13951 - global_seqlen/max:25329 - global_seqlen/minmax_diff:11378 - global_seqlen/balanced_min:20946 - global_seqlen/balanced_max:20991 - global_seqlen/mean:20964.125 - actor/entropy:0.18220646679401398 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5347471237182617 - training/rollout_probs_diff_mean:0.004718068055808544 - training/rollout_probs_diff_std:0.014215486124157906 - training/rollout_actor_probs_pearson_corr:0.9977114200592041 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.34765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:0.9998682737350464 - rollout_corr/rollout_is_ratio_fraction_high:2.2403190087061375e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.000100814359029755 - rollout_corr/rollout_is_max:2.7408487796783447 - rollout_corr/rollout_is_min:0.15083177387714386 - rollout_corr/rollout_is_std:0.04182150959968567 - rollout_corr/rollout_is_eff_sample_size:0.9982536587188602 - rollout_corr/rollout_is_seq_mean:0.9999032616615295 - rollout_corr/rollout_is_seq_std:0.0027911446522921324 - rollout_corr/rollout_is_seq_max:1.0079991817474365 - rollout_corr/rollout_is_seq_min:0.9924850463867188 - rollout_corr/rollout_is_seq_max_deviation:0.007999181747436523 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2486738041043282) - rollout_corr/training_log_ppl:np.float64(0.21914312147418968) - rollout_corr/kl:np.float64(0.001034660266309828) - rollout_corr/k3_kl:np.float64(0.0011393489194801987) - rollout_corr/rollout_ppl:np.float64(1.2473835735581815) - rollout_corr/rollout_log_ppl:np.float64(0.21810846046719234) - rollout_corr/log_ppl_diff:np.float64(0.0010346610069973394) - rollout_corr/log_ppl_abs_diff:np.float64(0.00256905252172146) - rollout_corr/log_ppl_diff_max:np.float64(0.011253908276557922) - rollout_corr/log_ppl_diff_min:np.float64(-0.00934329628944397) - rollout_corr/ppl_ratio:np.float64(1.0010404218919575) - rollout_corr/chi2_token:np.float64(0.002539546461775899) - rollout_corr/chi2_seq:np.float64(0.8009660658426583) - actor/pg_loss:np.float64(0.007912968518212438) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001622687290137037) - actor/ppo_kl:np.float64(-7.460985321472435e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.34765625) - self_distillation/token_reweight_w_mean:np.float64(457333.55283458694) - self_distillation/token_reweight_w_std:np.float64(5738632.68207643) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0023352606222034) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12474831352301408) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.6706360727548599) - perf/mfu/actor:np.float64(0.03522404308837758) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(142.29450607299805) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.34765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006558533757925034 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006558533757925034 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:348.72265625 - response_length/max:1665.0 - response_length/min:122.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:348.72265625 - response_length_non_aborted/max:1665.0 - response_length_non_aborted/min:122.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:306.40625 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.5972723960876465e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.497961925342679) - timing_s/agent_loop/generate_sequences/max:np.float64(10.335660284385085) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4384356094669783) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.335660284385085) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1665 - timing_s/gen:16.43148054368794 - timing_s/reward:0.0586068332195282 - timing_s/old_log_prob:2.5177703872323036 - timing_s/adv:0.6034593973308802 - timing_s/update_actor:17.04535499587655 - timing_s/step:36.742922419682145 - timing_s/stop_profile:0.00011501647531986237 - timing_per_token_ms/gen:0.18405879206129444 - timing_per_token_ms/update_actor:0.1016340712757899 - timing_per_token_ms/adv:0.003598167090988058 - perf/total_num_tokens:167713 - perf/time_per_step:36.742922419682145 - perf/throughput:570.5622639523663 (TaskRunner pid=2122883) Training Progress: 13%|█▎ | 13/100 [13:29<1:23:45, 57.77s/it] (TaskRunner pid=2122883) step:14 - global_seqlen/min:12670 - global_seqlen/max:36003 - global_seqlen/minmax_diff:23333 - global_seqlen/balanced_min:21692 - global_seqlen/balanced_max:22655 - global_seqlen/mean:21828.5 - actor/entropy:0.17604441940784454 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3474702835083008 - training/rollout_probs_diff_mean:0.004344606306403875 - training/rollout_probs_diff_std:0.013226086273789406 - training/rollout_actor_probs_pearson_corr:0.9979724884033203 - self_distillation/success_group_fraction:0.53125 - self_distillation/success_sample_fraction:0.515625 - self_distillation/correct_sample_fraction:0.265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.515625 - rollout_corr/rollout_is_mean:0.9997128248214722 - rollout_corr/rollout_is_ratio_fraction_high:6.112718256190419e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.131505117285997e-05 - rollout_corr/rollout_is_max:2.5963659286499023 - rollout_corr/rollout_is_min:0.29720374941825867 - rollout_corr/rollout_is_std:0.03975746035575867 - rollout_corr/rollout_is_eff_sample_size:0.9984208288720281 - rollout_corr/rollout_is_seq_mean:0.9996223449707031 - rollout_corr/rollout_is_seq_std:0.002889478811994195 - rollout_corr/rollout_is_seq_max:1.0082389116287231 - rollout_corr/rollout_is_seq_min:0.9903880953788757 - rollout_corr/rollout_is_seq_max_deviation:0.009611904621124268 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2598396032117307) - rollout_corr/training_log_ppl:np.float64(0.22693957190494984) - rollout_corr/kl:np.float64(0.001076112781067451) - rollout_corr/k3_kl:np.float64(0.0011196938060606954) - rollout_corr/rollout_ppl:np.float64(1.2584503088146448) - rollout_corr/rollout_log_ppl:np.float64(0.2258634588069981) - rollout_corr/log_ppl_diff:np.float64(0.0010761130979517475) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023237985878949985) - rollout_corr/log_ppl_diff_max:np.float64(0.010856091976165771) - rollout_corr/log_ppl_diff_min:np.float64(-0.006189435720443726) - rollout_corr/ppl_ratio:np.float64(1.0010813102126122) - rollout_corr/chi2_token:np.float64(0.002366213593631983) - rollout_corr/chi2_seq:np.float64(0.5546517146285623) - actor/pg_loss:np.float64(0.006254076375626028) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011818245492349888) - actor/ppo_kl:np.float64(-0.0002983780827854332) - actor/pg_clipfrac_lower:np.float64(4.474513389141066e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.515625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.265625) - self_distillation/token_reweight_w_mean:np.float64(268351.6325411438) - self_distillation/token_reweight_w_std:np.float64(3542681.9470955813) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0027324005495757) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09856546502851415) - self_distillation/empty_target_batch:np.float64(0.484375) - actor/grad_norm:np.float64(0.6500967741012573) - perf/mfu/actor:np.float64(0.035992356540273746) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(142.2788848876953) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008168119937181473 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008168119937181473 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:383.421875 - response_length/max:3009.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:383.421875 - response_length_non_aborted/max:3009.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:498.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.47657972574234e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1339652109891176) - timing_s/agent_loop/generate_sequences/max:np.float64(17.04697526805103) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.657824989699293) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.04697526805103) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:3009 - timing_s/gen:23.03968021646142 - timing_s/reward:0.059686632826924324 - timing_s/old_log_prob:2.516782818362117 - timing_s/adv:0.7135154157876968 - timing_s/update_actor:17.627836145460606 - timing_s/step:44.04473150335252 - timing_s/stop_profile:0.00011361576616764069 - timing_per_token_ms/gen:0.23472513362872793 - timing_per_token_ms/update_actor:0.10094507264276407 - timing_per_token_ms/adv:0.0040859164383014 - perf/total_num_tokens:174628 - perf/time_per_step:44.04473150335252 - perf/throughput:495.5984349305999 (TaskRunner pid=2122883) Training Progress: 14%|█▍ | 14/100 [14:14<1:16:52, 53.64s/it] (TaskRunner pid=2122883) step:15 - global_seqlen/min:15607 - global_seqlen/max:29545 - global_seqlen/minmax_diff:13938 - global_seqlen/balanced_min:19738 - global_seqlen/balanced_max:20007 - global_seqlen/mean:19790.375 - actor/entropy:0.19063463807106018 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.367106169462204 - training/rollout_probs_diff_mean:0.004759525414556265 - training/rollout_probs_diff_std:0.0138920983299613 - training/rollout_actor_probs_pearson_corr:0.997823178768158 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7265625 - self_distillation/correct_sample_fraction:0.37890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7265625 - rollout_corr/rollout_is_mean:1.0001037120819092 - rollout_corr/rollout_is_ratio_fraction_high:2.396386298642028e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.585545194568112e-05 - rollout_corr/rollout_is_max:2.688039779663086 - rollout_corr/rollout_is_min:0.004564262926578522 - rollout_corr/rollout_is_std:0.040728334337472916 - rollout_corr/rollout_is_eff_sample_size:0.9983443062319399 - rollout_corr/rollout_is_seq_mean:1.0002974271774292 - rollout_corr/rollout_is_seq_std:0.003095226129516959 - rollout_corr/rollout_is_seq_max:1.0108388662338257 - rollout_corr/rollout_is_seq_min:0.9914518594741821 - rollout_corr/rollout_is_seq_max_deviation:0.010838866233825684 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2595397448167205) - rollout_corr/training_log_ppl:np.float64(0.22738848751032492) - rollout_corr/kl:np.float64(0.0008545509965216525) - rollout_corr/k3_kl:np.float64(0.0012174916512890377) - rollout_corr/rollout_ppl:np.float64(1.258439245633781) - rollout_corr/rollout_log_ppl:np.float64(0.2265339351433795) - rollout_corr/log_ppl_diff:np.float64(0.0008545523669454269) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027597737425821833) - rollout_corr/log_ppl_diff_max:np.float64(0.015086829662322998) - rollout_corr/log_ppl_diff_min:np.float64(-0.010798841714859009) - rollout_corr/ppl_ratio:np.float64(1.0008610265795141) - rollout_corr/chi2_token:np.float64(0.0030557208228856325) - rollout_corr/chi2_seq:np.float64(0.7724779043346643) - actor/pg_loss:np.float64(0.00614961888641119) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016605062769485812) - actor/ppo_kl:np.float64(0.00013565428764827914) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7265625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.37890625) - self_distillation/token_reweight_w_mean:np.float64(380883.05325318524) - self_distillation/token_reweight_w_std:np.float64(4762354.298607164) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0042932590004057) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12382737548614386) - self_distillation/empty_target_batch:np.float64(0.2734375) - actor/grad_norm:np.float64(0.6634487509727478) - perf/mfu/actor:np.float64(0.03345707490169501) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.15536880493164) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:1 - critic/score/mean:0.37890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.37890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.02440270036458969 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.02440270036458969 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:326.01171875 - response_length/max:1897.0 - response_length/min:91.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:326.01171875 - response_length_non_aborted/max:1897.0 - response_length_non_aborted/min:91.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:292.4375 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00036331452429294586 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0936180874705315) - timing_s/agent_loop/generate_sequences/max:np.float64(11.154400121420622) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.176229142532975) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.154400121420622) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:1897 - timing_s/gen:17.43267442472279 - timing_s/reward:0.054192231968045235 - timing_s/old_log_prob:2.6273234728723764 - timing_s/adv:0.6899376567453146 - timing_s/update_actor:16.920646134763956 - timing_s/step:37.82306046411395 - timing_s/stop_profile:0.0001196097582578659 - timing_per_token_ms/gen:0.20887710642019183 - timing_per_token_ms/update_actor:0.10687421369456084 - timing_per_token_ms/adv:0.00435778539280657 - perf/total_num_tokens:158323 - perf/time_per_step:37.82306046411395 - perf/throughput:523.2356863024572 (TaskRunner pid=2122883) Training Progress: 15%|█▌ | 15/100 [14:52<1:09:23, 48.98s/it] (TaskRunner pid=2122883) step:16 - global_seqlen/min:18530 - global_seqlen/max:33812 - global_seqlen/minmax_diff:15282 - global_seqlen/balanced_min:22661 - global_seqlen/balanced_max:23200 - global_seqlen/mean:22884.375 - actor/entropy:0.1758926510810852 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7469337582588196 - training/rollout_probs_diff_mean:0.0043508936651051044 - training/rollout_probs_diff_std:0.013909836299717426 - training/rollout_actor_probs_pearson_corr:0.9977051615715027 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.48046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:0.9997097253799438 - rollout_corr/rollout_is_ratio_fraction_high:2.0211207811371423e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015158405585680157 - rollout_corr/rollout_is_max:2.464538812637329 - rollout_corr/rollout_is_min:0.059084586799144745 - rollout_corr/rollout_is_std:0.03929905965924263 - rollout_corr/rollout_is_eff_sample_size:0.998457311961611 - rollout_corr/rollout_is_seq_mean:0.9996926188468933 - rollout_corr/rollout_is_seq_std:0.0028329973574727774 - rollout_corr/rollout_is_seq_max:1.0078116655349731 - rollout_corr/rollout_is_seq_min:0.9882426857948303 - rollout_corr/rollout_is_seq_max_deviation:0.011757314205169678 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2475387873128057) - rollout_corr/training_log_ppl:np.float64(0.21804306718695443) - rollout_corr/kl:np.float64(0.0014540559303597433) - rollout_corr/k3_kl:np.float64(0.00116658230115263) - rollout_corr/rollout_ppl:np.float64(1.245679139625281) - rollout_corr/rollout_log_ppl:np.float64(0.21658901014598086) - rollout_corr/log_ppl_diff:np.float64(0.0014540570409735665) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025474778813077137) - rollout_corr/log_ppl_diff_max:np.float64(0.013031005859375) - rollout_corr/log_ppl_diff_min:np.float64(-0.008066460490226746) - rollout_corr/ppl_ratio:np.float64(1.001460450468585) - rollout_corr/chi2_token:np.float64(0.0016516137402504683) - rollout_corr/chi2_seq:np.float64(0.45585528621450067) - actor/pg_loss:np.float64(0.00893199082929641) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0021208707877349298) - actor/ppo_kl:np.float64(0.00010913769898257542) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48046875) - self_distillation/token_reweight_w_mean:np.float64(445003.87521046656) - self_distillation/token_reweight_w_std:np.float64(5454650.55214807) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015151123516262) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14990338099596556) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.7883509397506714) - perf/mfu/actor:np.float64(0.03755929023339058) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.36421585083008) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:1 - critic/score/mean:0.48046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.027472082525491714 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.027472082525491714 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:386.54296875 - response_length/max:2789.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:386.54296875 - response_length_non_aborted/max:2789.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:328.59375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013099610805511475 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9211164209991693) - timing_s/agent_loop/generate_sequences/max:np.float64(16.113097727298737) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5992245447123423) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.113097727298737) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:2789 - timing_s/gen:22.836567513644695 - timing_s/reward:0.060344504192471504 - timing_s/old_log_prob:2.496923489496112 - timing_s/adv:0.7712550535798073 - timing_s/update_actor:17.49838000163436 - timing_s/step:43.75420778989792 - timing_s/stop_profile:0.00011534243822097778 - timing_per_token_ms/gen:0.2307772978994967 - timing_per_token_ms/update_actor:0.09558039055924818 - timing_per_token_ms/adv:0.004212781939531926 - perf/total_num_tokens:183075 - perf/time_per_step:43.75420778989792 - perf/throughput:523.0211254169617 (TaskRunner pid=2122883) Training Progress: 16%|█▌ | 16/100 [15:36<1:06:23, 47.43s/it] (TaskRunner pid=2122883) step:17 - global_seqlen/min:11563 - global_seqlen/max:28003 - global_seqlen/minmax_diff:16440 - global_seqlen/balanced_min:19351 - global_seqlen/balanced_max:19748 - global_seqlen/mean:19445.875 - actor/entropy:0.20303893089294434 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5752326250076294 - training/rollout_probs_diff_mean:0.005234991200268269 - training/rollout_probs_diff_std:0.014962500892579556 - training/rollout_actor_probs_pearson_corr:0.9976497292518616 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7265625 - self_distillation/correct_sample_fraction:0.32421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7265625 - rollout_corr/rollout_is_mean:0.9999104142189026 - rollout_corr/rollout_is_ratio_fraction_high:6.119126919656992e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00017133555957116187 - rollout_corr/rollout_is_max:3.0671093463897705 - rollout_corr/rollout_is_min:0.11059963703155518 - rollout_corr/rollout_is_std:0.04376046359539032 - rollout_corr/rollout_is_eff_sample_size:0.9980883258424627 - rollout_corr/rollout_is_seq_mean:0.9999815821647644 - rollout_corr/rollout_is_seq_std:0.0031001772731542587 - rollout_corr/rollout_is_seq_max:1.0088906288146973 - rollout_corr/rollout_is_seq_min:0.9919108748435974 - rollout_corr/rollout_is_seq_max_deviation:0.008890628814697266 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2740392214618623) - rollout_corr/training_log_ppl:np.float64(0.2388933266774984) - rollout_corr/kl:np.float64(0.0015981203938295607) - rollout_corr/k3_kl:np.float64(0.0013284954461596499) - rollout_corr/rollout_ppl:np.float64(1.2719845059327781) - rollout_corr/rollout_log_ppl:np.float64(0.23729520468623377) - rollout_corr/log_ppl_diff:np.float64(0.0015981219912646338) - rollout_corr/log_ppl_abs_diff:np.float64(0.002778403417323716) - rollout_corr/log_ppl_diff_max:np.float64(0.012680396437644958) - rollout_corr/log_ppl_diff_min:np.float64(-0.008751362562179565) - rollout_corr/ppl_ratio:np.float64(1.0016051123384386) - rollout_corr/chi2_token:np.float64(0.0021377962548285723) - rollout_corr/chi2_seq:np.float64(0.34157682210206985) - actor/pg_loss:np.float64(0.006546365446411073) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002759105423024266) - actor/ppo_kl:np.float64(0.0004309166163753986) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7265625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.32421875) - self_distillation/token_reweight_w_mean:np.float64(577455.9122446231) - self_distillation/token_reweight_w_std:np.float64(6992234.631524403) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.00423149834387) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14134635338268708) - self_distillation/empty_target_batch:np.float64(0.2734375) - actor/grad_norm:np.float64(0.8037751317024231) - perf/mfu/actor:np.float64(0.03241075740075369) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.29125595092773) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:1 - critic/score/mean:0.32421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.32421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.02262394316494465 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.02262394316494465 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:319.18359375 - response_length/max:2055.0 - response_length/min:83.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:319.18359375 - response_length_non_aborted/max:2055.0 - response_length_non_aborted/min:83.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:288.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001981891691684723 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9945226181298494) - timing_s/agent_loop/generate_sequences/max:np.float64(12.014069978147745) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.190817145477922) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.014069978147745) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:2055 - timing_s/gen:17.9083558563143 - timing_s/reward:0.057533686980605125 - timing_s/old_log_prob:2.403095867484808 - timing_s/adv:0.6608398947864771 - timing_s/update_actor:17.16264975257218 - timing_s/step:38.281539618968964 - timing_s/stop_profile:0.00010946765542030334 - timing_per_token_ms/gen:0.2191670136984531 - timing_per_token_ms/update_actor:0.11032320320229984 - timing_per_token_ms/adv:0.004247943939180399 - perf/total_num_tokens:155567 - perf/time_per_step:38.281539618968964 - perf/throughput:507.9700344749022 (TaskRunner pid=2122883) Training Progress: 17%|█▋ | 17/100 [16:14<1:01:49, 44.69s/it] (TaskRunner pid=2122883) step:18 - global_seqlen/min:12223 - global_seqlen/max:32263 - global_seqlen/minmax_diff:20040 - global_seqlen/balanced_min:21577 - global_seqlen/balanced_max:21639 - global_seqlen/mean:21614.375 - actor/entropy:0.1750251203775406 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3790506422519684 - training/rollout_probs_diff_mean:0.004264963325113058 - training/rollout_probs_diff_std:0.013276408426463604 - training/rollout_actor_probs_pearson_corr:0.9978918433189392 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:1.0000395774841309 - rollout_corr/rollout_is_ratio_fraction_high:3.2373984140576795e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.553929754067212e-05 - rollout_corr/rollout_is_max:2.1670196056365967 - rollout_corr/rollout_is_min:0.12257643789052963 - rollout_corr/rollout_is_std:0.03953269124031067 - rollout_corr/rollout_is_eff_sample_size:0.9984397236898125 - rollout_corr/rollout_is_seq_mean:1.0000652074813843 - rollout_corr/rollout_is_seq_std:0.0029307748191058636 - rollout_corr/rollout_is_seq_max:1.0118049383163452 - rollout_corr/rollout_is_seq_min:0.988494873046875 - rollout_corr/rollout_is_seq_max_deviation:0.011804938316345215 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2621581708081067) - rollout_corr/training_log_ppl:np.float64(0.2286899130413076) - rollout_corr/kl:np.float64(0.001075027593940625) - rollout_corr/k3_kl:np.float64(0.0012590645611680884) - rollout_corr/rollout_ppl:np.float64(1.260771079454571) - rollout_corr/rollout_log_ppl:np.float64(0.22761488259129692) - rollout_corr/log_ppl_diff:np.float64(0.0010750304500106722) - rollout_corr/log_ppl_abs_diff:np.float64(0.002587559982202947) - rollout_corr/log_ppl_diff_max:np.float64(0.012028634548187256) - rollout_corr/log_ppl_diff_min:np.float64(-0.015134572982788086) - rollout_corr/ppl_ratio:np.float64(1.0010810499079525) - rollout_corr/chi2_token:np.float64(0.0028927032835781574) - rollout_corr/chi2_seq:np.float64(1.2977143267635256) - actor/pg_loss:np.float64(0.00664244475774467) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001839155015659344) - actor/ppo_kl:np.float64(0.00014196148600653657) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.359375) - self_distillation/token_reweight_w_mean:np.float64(623915.2676190655) - self_distillation/token_reweight_w_std:np.float64(7410794.226804514) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.002825409406796) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1299431096558692) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.7296064049005508) - perf/mfu/actor:np.float64(0.036137395670919115) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.27211380004883) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:1 - critic/score/mean:0.359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002475260989740491 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002475260989740491 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:361.98046875 - response_length/max:2043.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:361.98046875 - response_length_non_aborted/max:2043.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.46875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.489618241786957e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.046178888529539) - timing_s/agent_loop/generate_sequences/max:np.float64(12.541748790070415) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4098203388639376) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.541748790070415) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:2043 - timing_s/gen:18.420527707785368 - timing_s/reward:0.060600074008107185 - timing_s/old_log_prob:2.4621987380087376 - timing_s/adv:0.54259735904634 - timing_s/update_actor:17.234087236225605 - timing_s/step:38.808553241193295 - timing_s/stop_profile:0.00011383742094039917 - timing_per_token_ms/gen:0.1987819580625829 - timing_per_token_ms/update_actor:0.09966797117789437 - timing_per_token_ms/adv:0.0031379426830890323 - perf/total_num_tokens:172915 - perf/time_per_step:38.808553241193295 - perf/throughput:556.9487444086797 (TaskRunner pid=2122883) Training Progress: 18%|█▊ | 18/100 [16:53<58:41, 42.94s/it] (TaskRunner pid=2122883) step:19 - global_seqlen/min:14234 - global_seqlen/max:27408 - global_seqlen/minmax_diff:13174 - global_seqlen/balanced_min:20990 - global_seqlen/balanced_max:21463 - global_seqlen/mean:21081.0 - actor/entropy:0.18662618100643158 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4825435280799866 - training/rollout_probs_diff_mean:0.004559643566608429 - training/rollout_probs_diff_std:0.013524374924600124 - training/rollout_actor_probs_pearson_corr:0.9978901147842407 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.6953125 - self_distillation/correct_sample_fraction:0.328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6953125 - rollout_corr/rollout_is_mean:1.0000455379486084 - rollout_corr/rollout_is_ratio_fraction_high:4.3297539377817884e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011906823783647269 - rollout_corr/rollout_is_max:4.395936012268066 - rollout_corr/rollout_is_min:0.13285663723945618 - rollout_corr/rollout_is_std:0.04109692573547363 - rollout_corr/rollout_is_eff_sample_size:0.9983140093600031 - rollout_corr/rollout_is_seq_mean:1.00007164478302 - rollout_corr/rollout_is_seq_std:0.0030954221729189157 - rollout_corr/rollout_is_seq_max:1.011547565460205 - rollout_corr/rollout_is_seq_min:0.9888474345207214 - rollout_corr/rollout_is_seq_max_deviation:0.011547565460205078 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2576180850155652) - rollout_corr/training_log_ppl:np.float64(0.22579659259645268) - rollout_corr/kl:np.float64(0.0012318556480068743) - rollout_corr/k3_kl:np.float64(0.001264088400887431) - rollout_corr/rollout_ppl:np.float64(1.256009896285832) - rollout_corr/rollout_log_ppl:np.float64(0.22456473541387822) - rollout_corr/log_ppl_diff:np.float64(0.0012318571825744584) - rollout_corr/log_ppl_abs_diff:np.float64(0.002706136365304701) - rollout_corr/log_ppl_diff_max:np.float64(0.013555288314819336) - rollout_corr/log_ppl_diff_min:np.float64(-0.009048610925674438) - rollout_corr/ppl_ratio:np.float64(1.0012385330628604) - rollout_corr/chi2_token:np.float64(0.002662915736436844) - rollout_corr/chi2_seq:np.float64(0.6268946786876768) - actor/pg_loss:np.float64(0.007152381585910916) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0019675058147186064) - actor/ppo_kl:np.float64(0.00022251309200527203) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.328125) - self_distillation/token_reweight_w_mean:np.float64(426680.76371138333) - self_distillation/token_reweight_w_std:np.float64(5434152.009025335) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0037426524795592) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13176339765777811) - self_distillation/empty_target_batch:np.float64(0.3046875) - actor/grad_norm:np.float64(0.7264038622379303) - perf/mfu/actor:np.float64(0.035313067432871936) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.18925714492798) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:1 - critic/score/mean:0.328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02298964187502861 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02298964187502861 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:360.875 - response_length/max:2247.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:360.875 - response_length_non_aborted/max:2247.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:297.90625 - prompt_length/max:498.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011053308844566345 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1291147619485855) - timing_s/agent_loop/generate_sequences/max:np.float64(13.091949408873916) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.563902350026183) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.091949408873916) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:2247 - timing_s/gen:19.058726267889142 - timing_s/reward:0.0590360201895237 - timing_s/old_log_prob:2.6232305262237787 - timing_s/adv:0.5377237945795059 - timing_s/update_actor:17.08998142555356 - timing_s/step:39.45427840948105 - timing_s/stop_profile:0.00012236833572387695 - timing_per_token_ms/gen:0.20629899406703694 - timing_per_token_ms/update_actor:0.10133521551132275 - timing_per_token_ms/adv:0.0031884386092897983 - perf/total_num_tokens:168648 - perf/time_per_step:39.45427840948105 - perf/throughput:534.3146763757346 (TaskRunner pid=2122883) Training Progress: 19%|█▉ | 19/100 [17:32<56:34, 41.91s/it] (TaskRunner pid=2122883) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2122883) validation generation end (TaskRunner pid=2122883) [prompt] system (TaskRunner pid=2122883) (TaskRunner pid=2122883) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2122883) (TaskRunner pid=2122883) A (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) user (TaskRunner pid=2122883) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2122883) (TaskRunner pid=2122883) A: -0.01 (TaskRunner pid=2122883) B: 1.69 (TaskRunner pid=2122883) C: 2.49 (TaskRunner pid=2122883) D: 0.45 (TaskRunner pid=2122883) Please reason step by step. (TaskRunner pid=2122883) assistant (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) [response] (TaskRunner pid=2122883) The folding stability score of a protein sequence is typically determined through computational methods or experimental techniques such as molecular dynamics simulations, thermodynamic analysis, or energy minimization. However, without specific computational tools or databases that provide such scores for the given sequence, it is not possible to calculate or determine the exact folding stability score. The options provided (A: -0.01, B: 1.69, C: 2.49, D: 0.45) suggest possible scores, but none can be definitively determined without further information or computation. Therefore, the correct answer cannot be determined from the given information. (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) D (TaskRunner pid=2122883) (TaskRunner pid=2122883) [ground_truth] B (TaskRunner pid=2122883) [score] 0.0 (TaskRunner pid=2122883) [acc] 0.0 (TaskRunner pid=2122883) [pred] D (TaskRunner pid=2122883) [incorrect_format] 1 (TaskRunner pid=2122883) [feedback] (TaskRunner pid=2122883) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (WorkerDict pid=2123259) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10/actor (TaskRunner pid=2122883) step:20 - global_seqlen/min:11490 - global_seqlen/max:30712 - global_seqlen/minmax_diff:19222 - global_seqlen/balanced_min:21053 - global_seqlen/balanced_max:21083 - global_seqlen/mean:21076.125 - actor/entropy:0.19185848534107208 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.605046272277832 - training/rollout_probs_diff_mean:0.004658793099224567 - training/rollout_probs_diff_std:0.013916742987930775 - training/rollout_actor_probs_pearson_corr:0.997868001461029 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.6953125 - self_distillation/correct_sample_fraction:0.36328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6953125 - rollout_corr/rollout_is_mean:1.000085473060608 - rollout_corr/rollout_is_ratio_fraction_high:5.654509368469007e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014701724285259843 - rollout_corr/rollout_is_max:3.1271145343780518 - rollout_corr/rollout_is_min:0.24705207347869873 - rollout_corr/rollout_is_std:0.041785866022109985 - rollout_corr/rollout_is_eff_sample_size:0.9982572225303701 - rollout_corr/rollout_is_seq_mean:1.0000156164169312 - rollout_corr/rollout_is_seq_std:0.002887738635763526 - rollout_corr/rollout_is_seq_max:1.0120712518692017 - rollout_corr/rollout_is_seq_min:0.992546796798706 - rollout_corr/rollout_is_seq_max_deviation:0.01207125186920166 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2571831136010587) - rollout_corr/training_log_ppl:np.float64(0.22537461307365447) - rollout_corr/kl:np.float64(0.0011899686502321316) - rollout_corr/k3_kl:np.float64(0.0011254083661924597) - rollout_corr/rollout_ppl:np.float64(1.255645914003253) - rollout_corr/rollout_log_ppl:np.float64(0.22418464263319038) - rollout_corr/log_ppl_diff:np.float64(0.0011899704404640943) - rollout_corr/log_ppl_abs_diff:np.float64(0.002389262692304328) - rollout_corr/log_ppl_diff_max:np.float64(0.010065972805023193) - rollout_corr/log_ppl_diff_min:np.float64(-0.006951630115509033) - rollout_corr/ppl_ratio:np.float64(1.001195044722408) - rollout_corr/chi2_token:np.float64(0.0022096477914601564) - rollout_corr/chi2_seq:np.float64(3.44644234678708) - actor/pg_loss:np.float64(0.006273084436543286) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014769084930321696) - actor/ppo_kl:np.float64(0.00015581477314441372) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.36328125) - self_distillation/token_reweight_w_mean:np.float64(431294.86135452124) - self_distillation/token_reweight_w_std:np.float64(5506743.177664222) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.003677363274619) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12259233223448973) - self_distillation/empty_target_batch:np.float64(0.3046875) - actor/grad_norm:np.float64(0.6836053431034088) - perf/mfu/actor:np.float64(0.03593170496025079) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.2952423095703) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.32875) - val-aux/sciknoweval/reward/std@16:np.float64(0.29435610050286326) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.45084) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.2764717653668717) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.20032000000000003) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.22610926818200705) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.32604) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.29362819649402) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.5661599999999999) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.2170131088439427) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.11420000000000002) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.13524791805681496) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.33177999999999996) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.2486997734125882) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.6531) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.14949042862248849) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.06770000000000001) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.06467335374646328) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.34218) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.20269370137809958) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.71174) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.09224539666924036) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.04794) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.025378175348871202) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.34806000000000004) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.15403633275085624) - val-aux/sciknoweval/score/mean@16:np.float64(0.32875) - val-aux/sciknoweval/score/std@16:np.float64(0.29435610050286326) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.45084) - val-aux/sciknoweval/score/best@2/std:np.float64(0.2764717653668717) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.20032000000000003) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.22610926818200705) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.32604) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.29362819649402) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.5661599999999999) - val-aux/sciknoweval/score/best@4/std:np.float64(0.2170131088439427) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.11420000000000002) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.13524791805681496) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.33177999999999996) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.2486997734125882) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.6531) - val-aux/sciknoweval/score/best@8/std:np.float64(0.14949042862248849) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.06770000000000001) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.06467335374646328) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.34218) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.20269370137809958) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.71174) - val-aux/sciknoweval/score/best@16/std:np.float64(0.09224539666924036) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.04794) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.025378175348871202) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.34806000000000004) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.15403633275085624) - val-core/sciknoweval/acc/mean@16:np.float64(0.32875) - val-aux/sciknoweval/acc/std@16:np.float64(0.29435610050286326) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.45084) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.2764717653668717) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.20032000000000003) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.22610926818200705) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.32604) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.29362819649402) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.5661599999999999) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.2170131088439427) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.11420000000000002) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.13524791805681496) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.33177999999999996) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.2486997734125882) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.6531) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.14949042862248849) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.06770000000000001) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.06467335374646328) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.34218) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.20269370137809958) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.71174) - val-core/sciknoweval/acc/best@16/std:np.float64(0.09224539666924036) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.04794) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.025378175348871202) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.34806000000000004) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.15403633275085624) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.03862677168945583) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99836) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0139506864431683) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9766399999999998) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.050116218604848536) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.98808) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.037513975347298376) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999600000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0012642784503423287) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.95852) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.061058789332634845) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99606) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.021621459553489996) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9308999999999998) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.06627509521077407) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9989999999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.010527221260855181) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8992) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.05853224575927146) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999600000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0012642784503423287) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:1 - critic/score/mean:0.36328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.36328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0057266042567789555 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0057266042567789555 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:345.41015625 - response_length/max:1419.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:345.41015625 - response_length_non_aborted/max:1419.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.21875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013404525816440582 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0222893320024014) - timing_s/agent_loop/generate_sequences/max:np.float64(9.325520938262343) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3090429624644457) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.325520938262343) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:1419 - timing_s/gen:15.228843804448843 - timing_s/reward:0.05905533581972122 - timing_s/old_log_prob:2.497826090082526 - timing_s/adv:0.5414953380823135 - timing_s/update_actor:16.95986382290721 - timing_s/step:35.37535096332431 - timing_s/testing:77.24785147979856 - timing_s/save_checkpoint:7.0980935245752335 - timing_s/stop_profile:0.00025027431547641754 - timing_per_token_ms/gen:0.1722232830585111 - timing_per_token_ms/update_actor:0.10058694270713431 - timing_per_token_ms/adv:0.0032115446867149056 - perf/total_num_tokens:168609 - perf/time_per_step:35.37535096332431 - perf/throughput:595.7856085117247 (TaskRunner pid=2122883) Training Progress: 20%|██ | 20/100 [19:32<1:27:03, 65.30s/it] (TaskRunner pid=2122883) step:21 - global_seqlen/min:13618 - global_seqlen/max:28101 - global_seqlen/minmax_diff:14483 - global_seqlen/balanced_min:19845 - global_seqlen/balanced_max:19859 - global_seqlen/mean:19853.75 - actor/entropy:0.1921236366033554 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8342254161834717 - training/rollout_probs_diff_mean:0.004657784476876259 - training/rollout_probs_diff_std:0.013867578469216824 - training/rollout_actor_probs_pearson_corr:0.9979041814804077 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:0.9997999668121338 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.753607547376305e-05 - rollout_corr/rollout_is_max:1.7968099117279053 - rollout_corr/rollout_is_min:0.10106302052736282 - rollout_corr/rollout_is_std:0.040148328989744186 - rollout_corr/rollout_is_eff_sample_size:0.9983901708780174 - rollout_corr/rollout_is_seq_mean:0.9999698400497437 - rollout_corr/rollout_is_seq_std:0.0028422540053725243 - rollout_corr/rollout_is_seq_max:1.00948166847229 - rollout_corr/rollout_is_seq_min:0.993533730506897 - rollout_corr/rollout_is_seq_max_deviation:0.009481668472290039 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2641725991852582) - rollout_corr/training_log_ppl:np.float64(0.23089192975021433) - rollout_corr/kl:np.float64(0.001236026230320597) - rollout_corr/k3_kl:np.float64(0.0011104728028499267) - rollout_corr/rollout_ppl:np.float64(1.2625776953063905) - rollout_corr/rollout_log_ppl:np.float64(0.22965590309468098) - rollout_corr/log_ppl_diff:np.float64(0.001236026655533351) - rollout_corr/log_ppl_abs_diff:np.float64(0.002413631693343632) - rollout_corr/log_ppl_diff_max:np.float64(0.010084569454193115) - rollout_corr/log_ppl_diff_min:np.float64(-0.006885886192321777) - rollout_corr/ppl_ratio:np.float64(1.0012409987393767) - rollout_corr/chi2_token:np.float64(0.0019542204681783915) - rollout_corr/chi2_seq:np.float64(0.2917343757580966) - actor/pg_loss:np.float64(0.007957952562719584) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0018407491854759428) - actor/ppo_kl:np.float64(0.00021987110629595463) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.390625) - self_distillation/token_reweight_w_mean:np.float64(397908.3636041009) - self_distillation/token_reweight_w_std:np.float64(4692473.746526102) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004959150683135) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14321871772699524) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.7271679639816284) - perf/mfu/actor:np.float64(0.033430629892783156) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.5677375793457) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:1 - critic/score/mean:0.390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013565568253397942 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013565568253397942 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:339.4609375 - response_length/max:1403.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:339.4609375 - response_length_non_aborted/max:1403.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.96875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.517954170703888e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0614673774689436) - timing_s/agent_loop/generate_sequences/max:np.float64(9.238231776282191) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.313919332322257) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.238231776282191) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1392 - timing_s/gen:14.915440939366817 - timing_s/reward:0.051850782707333565 - timing_s/old_log_prob:2.5148690696805716 - timing_s/adv:0.6088764388114214 - timing_s/update_actor:17.02783625945449 - timing_s/step:35.21148547902703 - timing_s/stop_profile:0.00012161023914813995 - timing_per_token_ms/gen:0.17163518606438077 - timing_per_token_ms/update_actor:0.10720793464367241 - timing_per_token_ms/adv:0.00383351028654172 - perf/total_num_tokens:158830 - perf/time_per_step:35.21148547902703 - perf/throughput:563.8430111625214 (TaskRunner pid=2122883) Training Progress: 21%|██ | 21/100 [20:07<1:14:06, 56.29s/it] (TaskRunner pid=2122883) step:22 - global_seqlen/min:16977 - global_seqlen/max:37961 - global_seqlen/minmax_diff:20984 - global_seqlen/balanced_min:23830 - global_seqlen/balanced_max:24580 - global_seqlen/mean:24009.875 - actor/entropy:0.17836344242095947 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3530232608318329 - training/rollout_probs_diff_mean:0.0041275168769061565 - training/rollout_probs_diff_std:0.012802300043404102 - training/rollout_actor_probs_pearson_corr:0.9980253577232361 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.46484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.0001649856567383 - rollout_corr/rollout_is_ratio_fraction_high:2.90407824650174e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.80815649300348e-05 - rollout_corr/rollout_is_max:3.3773062229156494 - rollout_corr/rollout_is_min:0.16175149381160736 - rollout_corr/rollout_is_std:0.03797151893377304 - rollout_corr/rollout_is_eff_sample_size:0.9985608338735662 - rollout_corr/rollout_is_seq_mean:1.000291109085083 - rollout_corr/rollout_is_seq_std:0.0029728421941399574 - rollout_corr/rollout_is_seq_max:1.0122034549713135 - rollout_corr/rollout_is_seq_min:0.9913510084152222 - rollout_corr/rollout_is_seq_max_deviation:0.012203454971313477 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2535668751224875) - rollout_corr/training_log_ppl:np.float64(0.22167573400656693) - rollout_corr/kl:np.float64(0.0007851657160911074) - rollout_corr/k3_kl:np.float64(0.0010411924005779838) - rollout_corr/rollout_ppl:np.float64(1.2525088130496442) - rollout_corr/rollout_log_ppl:np.float64(0.22089056581899058) - rollout_corr/log_ppl_diff:np.float64(0.0007851681875763461) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021511910745175555) - rollout_corr/log_ppl_diff_max:np.float64(0.016502737998962402) - rollout_corr/log_ppl_diff_min:np.float64(-0.007668659090995789) - rollout_corr/ppl_ratio:np.float64(1.0007898204494268) - rollout_corr/chi2_token:np.float64(0.002642852021381259) - rollout_corr/chi2_seq:np.float64(1.0275017379317433) - actor/pg_loss:np.float64(0.0077134715393185616) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001129955593341947) - actor/ppo_kl:np.float64(0.00012342048217206525) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.46484375) - self_distillation/token_reweight_w_mean:np.float64(395338.3554381167) - self_distillation/token_reweight_w_std:np.float64(5012417.347034392) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001086757518351) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1375344575062627) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.7418015450239182) - perf/mfu/actor:np.float64(0.039820292543893625) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.55425262451172) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:1 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0037849820218980312 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0037849820218980312 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:403.52734375 - response_length/max:2614.0 - response_length/min:118.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:403.52734375 - response_length_non_aborted/max:2614.0 - response_length_non_aborted/min:118.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:346.78125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.398426532745361e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0202389620244503) - timing_s/agent_loop/generate_sequences/max:np.float64(15.357408994808793) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9177660304267192) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.357408994808793) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:2614 - timing_s/gen:21.221346884965897 - timing_s/reward:0.06203651241958141 - timing_s/old_log_prob:2.4261272829025984 - timing_s/adv:0.6985401939600706 - timing_s/update_actor:17.410948403179646 - timing_s/step:41.90618650615215 - timing_s/stop_profile:3.1556934118270874e-05 - timing_per_token_ms/gen:0.20542817619009995 - timing_per_token_ms/update_actor:0.0906447264051752 - timing_per_token_ms/adv:0.003636733812442123 - perf/total_num_tokens:192079 - perf/time_per_step:41.90618650615215 - perf/throughput:572.9434482537599 (TaskRunner pid=2122883) Training Progress: 22%|██▏ | 22/100 [20:49<1:07:34, 51.98s/it] (TaskRunner pid=2122883) step:23 - global_seqlen/min:17524 - global_seqlen/max:33089 - global_seqlen/minmax_diff:15565 - global_seqlen/balanced_min:22610 - global_seqlen/balanced_max:22638 - global_seqlen/mean:22633.0 - actor/entropy:0.19653943181037903 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4863523840904236 - training/rollout_probs_diff_mean:0.004470366053283215 - training/rollout_probs_diff_std:0.013256288133561611 - training/rollout_actor_probs_pearson_corr:0.9980781674385071 - self_distillation/success_group_fraction:0.5625 - self_distillation/success_sample_fraction:0.5546875 - self_distillation/correct_sample_fraction:0.265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.5546875 - rollout_corr/rollout_is_mean:0.9998451471328735 - rollout_corr/rollout_is_ratio_fraction_high:2.0983716240152717e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010491858120076358 - rollout_corr/rollout_is_max:2.2246580123901367 - rollout_corr/rollout_is_min:0.2494165003299713 - rollout_corr/rollout_is_std:0.04001970589160919 - rollout_corr/rollout_is_eff_sample_size:0.9984005088283843 - rollout_corr/rollout_is_seq_mean:0.9997673034667969 - rollout_corr/rollout_is_seq_std:0.002586677670478821 - rollout_corr/rollout_is_seq_max:1.009292721748352 - rollout_corr/rollout_is_seq_min:0.9929970502853394 - rollout_corr/rollout_is_seq_max_deviation:0.00929272174835205 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2631977992132306) - rollout_corr/training_log_ppl:np.float64(0.22998240585729945) - rollout_corr/kl:np.float64(0.0012839085080340595) - rollout_corr/k3_kl:np.float64(0.0011792379793860164) - rollout_corr/rollout_ppl:np.float64(1.261534459888935) - rollout_corr/rollout_log_ppl:np.float64(0.22869849536800757) - rollout_corr/log_ppl_diff:np.float64(0.001283910489291884) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025792625528993085) - rollout_corr/log_ppl_diff_max:np.float64(0.014777734875679016) - rollout_corr/log_ppl_diff_min:np.float64(-0.011869877576828003) - rollout_corr/ppl_ratio:np.float64(1.001290408661589) - rollout_corr/chi2_token:np.float64(0.002155705587938428) - rollout_corr/chi2_seq:np.float64(1.0525127614382654) - actor/pg_loss:np.float64(0.006339410902000964) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016231782274189754) - actor/ppo_kl:np.float64(9.229942891020304e-05) - actor/pg_clipfrac_lower:np.float64(9.481189408688806e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.5546875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.5546875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.265625) - self_distillation/token_reweight_w_mean:np.float64(356218.2028367857) - self_distillation/token_reweight_w_std:np.float64(4629386.739539903) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010050446726382) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10248490553931333) - self_distillation/empty_target_batch:np.float64(0.4453125) - actor/grad_norm:np.float64(0.6276970654726028) - perf/mfu/actor:np.float64(0.03810892654843721) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.44520616531372) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:1 - critic/score/mean:0.265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009594804607331753 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009594804607331753 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:372.3125 - response_length/max:1512.0 - response_length/min:133.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:372.3125 - response_length_non_aborted/max:1512.0 - response_length_non_aborted/min:133.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:334.96875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.639476537704468e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5703063905239105) - timing_s/agent_loop/generate_sequences/max:np.float64(9.733793312683702) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6518209086207207) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.733793312683702) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1512 - timing_s/gen:15.654936438426375 - timing_s/reward:0.060526276007294655 - timing_s/old_log_prob:2.4749402347952127 - timing_s/adv:0.5394423864781857 - timing_s/update_actor:16.798127077519894 - timing_s/step:35.61569402553141 - timing_s/stop_profile:0.00011201202869415283 - timing_per_token_ms/gen:0.16424937508840834 - timing_per_token_ms/update_actor:0.09277452766712264 - timing_per_token_ms/adv:0.0029792912256339506 - perf/total_num_tokens:181064 - perf/time_per_step:35.61569402553141 - perf/throughput:635.4782805516957 (TaskRunner pid=2122883) Training Progress: 23%|██▎ | 23/100 [21:25<1:00:25, 47.08s/it] (TaskRunner pid=2122883) step:24 - global_seqlen/min:12679 - global_seqlen/max:29738 - global_seqlen/minmax_diff:17059 - global_seqlen/balanced_min:18589 - global_seqlen/balanced_max:18667 - global_seqlen/mean:18608.25 - actor/entropy:0.21970593929290771 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5529158115386963 - training/rollout_probs_diff_mean:0.005068894941359758 - training/rollout_probs_diff_std:0.013992728665471077 - training/rollout_actor_probs_pearson_corr:0.9980122447013855 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:1.0020556449890137 - rollout_corr/rollout_is_ratio_fraction_high:3.916346759069711e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.138142922893167e-05 - rollout_corr/rollout_is_max:179.5236053466797 - rollout_corr/rollout_is_min:0.09267795085906982 - rollout_corr/rollout_is_std:0.042068056762218475 - rollout_corr/rollout_is_eff_sample_size:0.998232513960617 - rollout_corr/rollout_is_seq_mean:1.0005260705947876 - rollout_corr/rollout_is_seq_std:0.013086115941405296 - rollout_corr/rollout_is_seq_max:1.2028028964996338 - rollout_corr/rollout_is_seq_min:0.9902984499931335 - rollout_corr/rollout_is_seq_max_deviation:0.2028028964996338 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2807346642948687) - rollout_corr/training_log_ppl:np.float64(0.24428033985896036) - rollout_corr/kl:np.float64(0.001533059075450538) - rollout_corr/k3_kl:np.float64(0.001212075309354077) - rollout_corr/rollout_ppl:np.float64(1.2787316790781915) - rollout_corr/rollout_log_ppl:np.float64(0.24274727902957238) - rollout_corr/log_ppl_diff:np.float64(0.0015330608293879777) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029383527289610356) - rollout_corr/log_ppl_diff_max:np.float64(0.012038901448249817) - rollout_corr/log_ppl_diff_min:np.float64(-0.007719486951828003) - rollout_corr/ppl_ratio:np.float64(1.0015401267446578) - rollout_corr/chi2_token:np.float64(0.005337873008102179) - rollout_corr/chi2_seq:np.float64(0.7134281191974878) - actor/pg_loss:np.float64(0.007017654599621892) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001894508582608978) - actor/ppo_kl:np.float64(0.00026200509176987907) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(555852.1664180572) - self_distillation/token_reweight_w_std:np.float64(6403345.388541037) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0016352159436792) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13323489562026225) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.6621586382389069) - perf/mfu/actor:np.float64(0.031465761930366626) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.51604080200195) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:1 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005094514694064856 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005094514694064856 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:299.2265625 - response_length/max:1621.0 - response_length/min:106.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:299.2265625 - response_length_non_aborted/max:1621.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:282.28125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015429407358169556 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2271409928798676) - timing_s/agent_loop/generate_sequences/max:np.float64(9.899323958903551) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9376148904048023) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.899323958903551) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:1621 - timing_s/gen:15.829845121130347 - timing_s/reward:0.056639643386006355 - timing_s/old_log_prob:2.396368984133005 - timing_s/adv:0.5699858013540506 - timing_s/update_actor:16.899894734844565 - timing_s/step:35.841407015919685 - timing_s/stop_profile:0.00012869946658611298 - timing_per_token_ms/gen:0.20665054595350443 - timing_per_token_ms/update_actor:0.11352420791076918 - timing_per_token_ms/adv:0.0038288514593933514 - perf/total_num_tokens:148866 - perf/time_per_step:35.841407015919685 - perf/throughput:519.1830217975196 (TaskRunner pid=2122883) Training Progress: 24%|██▍ | 24/100 [22:01<55:22, 43.72s/it] (TaskRunner pid=2122883) step:25 - global_seqlen/min:17225 - global_seqlen/max:30301 - global_seqlen/minmax_diff:13076 - global_seqlen/balanced_min:25937 - global_seqlen/balanced_max:25982 - global_seqlen/mean:25966.875 - actor/entropy:0.1851537972688675 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5962941646575928 - training/rollout_probs_diff_mean:0.004132953006774187 - training/rollout_probs_diff_std:0.013088012114167213 - training/rollout_actor_probs_pearson_corr:0.9980089664459229 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.79296875 - self_distillation/correct_sample_fraction:0.40234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.79296875 - rollout_corr/rollout_is_mean:1.0000680685043335 - rollout_corr/rollout_is_ratio_fraction_high:3.482652027742006e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00016542596858926117 - rollout_corr/rollout_is_max:2.9371936321258545 - rollout_corr/rollout_is_min:0.0107397036626935 - rollout_corr/rollout_is_std:0.038702256977558136 - rollout_corr/rollout_is_eff_sample_size:0.9985047320789843 - rollout_corr/rollout_is_seq_mean:1.0000791549682617 - rollout_corr/rollout_is_seq_std:0.002427029889076948 - rollout_corr/rollout_is_seq_max:1.0064553022384644 - rollout_corr/rollout_is_seq_min:0.9905536770820618 - rollout_corr/rollout_is_seq_max_deviation:0.009446322917938232 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.254920766223222) - rollout_corr/training_log_ppl:np.float64(0.22319715823687147) - rollout_corr/kl:np.float64(0.0009513927656978893) - rollout_corr/k3_kl:np.float64(0.001108306266019099) - rollout_corr/rollout_ppl:np.float64(1.253699362743646) - rollout_corr/rollout_log_ppl:np.float64(0.22224576224107295) - rollout_corr/log_ppl_diff:np.float64(0.0009513959957985207) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020687162032118067) - rollout_corr/log_ppl_diff_max:np.float64(0.012234926223754883) - rollout_corr/log_ppl_diff_min:np.float64(-0.015981703996658325) - rollout_corr/ppl_ratio:np.float64(1.0009557630401105) - rollout_corr/chi2_token:np.float64(0.002664993517100811) - rollout_corr/chi2_seq:np.float64(1.0995744443498552) - actor/pg_loss:np.float64(0.006765639060176909) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001574875209144011) - actor/ppo_kl:np.float64(3.649607397449017e-05) - actor/pg_clipfrac_lower:np.float64(4.1273967781307874e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.79296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.79296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.40234375) - self_distillation/token_reweight_w_mean:np.float64(353427.5153041347) - self_distillation/token_reweight_w_std:np.float64(5117519.624401459) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0020341072231531) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14214718576113228) - self_distillation/empty_target_batch:np.float64(0.20703125) - actor/grad_norm:np.float64(0.6468942165374756) - perf/mfu/actor:np.float64(0.04292664526543912) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.73140716552734) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:1 - critic/score/mean:0.40234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003380349138751626 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003380349138751626 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:448.65234375 - response_length/max:2031.0 - response_length/min:128.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:448.65234375 - response_length_non_aborted/max:2031.0 - response_length_non_aborted/min:128.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:362.8125 - prompt_length/max:499.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011907331645488739 - timing_s/agent_loop/generate_sequences/min:np.float64(1.524836651980877) - timing_s/agent_loop/generate_sequences/max:np.float64(13.144861306995153) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.346475338068558) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.144861306995153) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:2031 - timing_s/gen:18.811984948813915 - timing_s/reward:0.06409545615315437 - timing_s/old_log_prob:2.5523454006761312 - timing_s/adv:0.6613637115806341 - timing_s/update_actor:17.477145528420806 - timing_s/step:39.65423706173897 - timing_s/stop_profile:0.00011950545012950897 - timing_per_token_ms/gen:0.1637889943738968 - timing_per_token_ms/update_actor:0.08413192542624404 - timing_per_token_ms/adv:0.0031836893714618825 - perf/total_num_tokens:207735 - perf/time_per_step:39.65423706173897 - perf/throughput:654.8322934462547 (TaskRunner pid=2122883) Training Progress: 25%|██▌ | 25/100 [22:40<53:08, 42.52s/it] (TaskRunner pid=2122883) step:26 - global_seqlen/min:13828 - global_seqlen/max:27110 - global_seqlen/minmax_diff:13282 - global_seqlen/balanced_min:22022 - global_seqlen/balanced_max:22201 - global_seqlen/mean:22057.875 - actor/entropy:0.20123563706874847 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27729910612106323 - training/rollout_probs_diff_mean:0.0043639508076012135 - training/rollout_probs_diff_std:0.012655162252485752 - training/rollout_actor_probs_pearson_corr:0.998265266418457 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:1.0000321865081787 - rollout_corr/rollout_is_ratio_fraction_high:6.163265061331913e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014380952052306384 - rollout_corr/rollout_is_max:3.1523871421813965 - rollout_corr/rollout_is_min:0.22010482847690582 - rollout_corr/rollout_is_std:0.03991381824016571 - rollout_corr/rollout_is_eff_sample_size:0.9984094210263972 - rollout_corr/rollout_is_seq_mean:1.0000900030136108 - rollout_corr/rollout_is_seq_std:0.0028636239003390074 - rollout_corr/rollout_is_seq_max:1.009320616722107 - rollout_corr/rollout_is_seq_min:0.9920967221260071 - rollout_corr/rollout_is_seq_max_deviation:0.009320616722106934 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.285409896634519) - rollout_corr/training_log_ppl:np.float64(0.24616308329859748) - rollout_corr/kl:np.float64(0.0010952132663355485) - rollout_corr/k3_kl:np.float64(0.0010522639622649876) - rollout_corr/rollout_ppl:np.float64(1.2839300143532455) - rollout_corr/rollout_log_ppl:np.float64(0.24506786806159653) - rollout_corr/log_ppl_diff:np.float64(0.0010952152370009571) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022666600707452744) - rollout_corr/log_ppl_diff_max:np.float64(0.009045511484146118) - rollout_corr/log_ppl_diff_min:np.float64(-0.008367002010345459) - rollout_corr/ppl_ratio:np.float64(1.0010996800847352) - rollout_corr/chi2_token:np.float64(0.002009735209867358) - rollout_corr/chi2_seq:np.float64(0.3117946165148169) - actor/pg_loss:np.float64(0.006661892984993756) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010209764229784923) - actor/ppo_kl:np.float64(0.0002103924846554861) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(235097.68068203423) - self_distillation/token_reweight_w_std:np.float64(3208859.9599546287) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0031206426210701) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11432849438278936) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.6480773836374283) - perf/mfu/actor:np.float64(0.03729586141445862) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.68837356567383) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:1 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.012705313973128796 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.012705313973128796 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:380.27734375 - response_length/max:2030.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:380.27734375 - response_length_non_aborted/max:2030.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:309.03125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.533390402793884e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2987001277506351) - timing_s/agent_loop/generate_sequences/max:np.float64(12.182028014212847) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6352716596229584) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.182028014212847) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:424 - timing_s/agent_loop/slowest/response_length:2030 - timing_s/gen:18.078122118487954 - timing_s/reward:0.05934295244514942 - timing_s/old_log_prob:2.4004305750131607 - timing_s/adv:0.5496773198246956 - timing_s/update_actor:16.90547569282353 - timing_s/step:38.081087835133076 - timing_s/stop_profile:0.00012539885938167572 - timing_per_token_ms/gen:0.18570042545518745 - timing_per_token_ms/update_actor:0.09580181507071471 - timing_per_token_ms/adv:0.003114972089473122 - perf/total_num_tokens:176463 - perf/time_per_step:38.081087835133076 - perf/throughput:579.2343720719479 (TaskRunner pid=2122883) Training Progress: 26%|██▌ | 26/100 [23:19<50:49, 41.20s/it] (TaskRunner pid=2122883) step:27 - global_seqlen/min:13161 - global_seqlen/max:28209 - global_seqlen/minmax_diff:15048 - global_seqlen/balanced_min:21333 - global_seqlen/balanced_max:23462 - global_seqlen/mean:21600.625 - actor/entropy:0.2077469378709793 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8482835292816162 - training/rollout_probs_diff_mean:0.004673023708164692 - training/rollout_probs_diff_std:0.013583921827375889 - training/rollout_actor_probs_pearson_corr:0.9980661869049072 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.66796875 - self_distillation/correct_sample_fraction:0.3984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.66796875 - rollout_corr/rollout_is_mean:0.9999101161956787 - rollout_corr/rollout_is_ratio_fraction_high:3.2460156944580376e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001298406277783215 - rollout_corr/rollout_is_max:2.862781047821045 - rollout_corr/rollout_is_min:0.08208515495061874 - rollout_corr/rollout_is_std:0.040227677673101425 - rollout_corr/rollout_is_eff_sample_size:0.9983839919754216 - rollout_corr/rollout_is_seq_mean:1.0000474452972412 - rollout_corr/rollout_is_seq_std:0.002930537099018693 - rollout_corr/rollout_is_seq_max:1.0101268291473389 - rollout_corr/rollout_is_seq_min:0.9879516363143921 - rollout_corr/rollout_is_seq_max_deviation:0.01204836368560791 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2790318825282156) - rollout_corr/training_log_ppl:np.float64(0.24226842416101135) - rollout_corr/kl:np.float64(0.0012188144196088047) - rollout_corr/k3_kl:np.float64(0.0012009678286233338) - rollout_corr/rollout_ppl:np.float64(1.277424986474216) - rollout_corr/rollout_log_ppl:np.float64(0.2410496080556186) - rollout_corr/log_ppl_diff:np.float64(0.0012188161053927615) - rollout_corr/log_ppl_abs_diff:np.float64(0.002619924969621934) - rollout_corr/log_ppl_diff_max:np.float64(0.013451695442199707) - rollout_corr/log_ppl_diff_min:np.float64(-0.007775008678436279) - rollout_corr/ppl_ratio:np.float64(1.0012246232945472) - rollout_corr/chi2_token:np.float64(0.0023596512619405985) - rollout_corr/chi2_seq:np.float64(0.5256561439018697) - actor/pg_loss:np.float64(0.004449836676940322) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013233652471171808) - actor/ppo_kl:np.float64(0.00026928278293070207) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.66796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3984375) - self_distillation/token_reweight_w_mean:np.float64(324819.42211573245) - self_distillation/token_reweight_w_std:np.float64(4507005.021220025) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001485469052568) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10996017501747701) - self_distillation/empty_target_batch:np.float64(0.33203125) - actor/grad_norm:np.float64(0.5749102309346199) - perf/mfu/actor:np.float64(0.03605009092181908) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.96850204467773) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:1 - critic/score/mean:0.3984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.027892740443348885 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.027892740443348885 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:361.01953125 - response_length/max:3699.0 - response_length/min:127.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:361.01953125 - response_length_non_aborted/max:3699.0 - response_length_non_aborted/min:127.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.0 - prompt_length/max:499.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.86289232969284e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.594991670921445) - timing_s/agent_loop/generate_sequences/max:np.float64(19.769734585657716) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.591588780793245) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.769734585657716) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:432 - timing_s/agent_loop/slowest/response_length:3699 - timing_s/gen:25.607453675940633 - timing_s/reward:0.1120842956006527 - timing_s/old_log_prob:4.264432176947594 - timing_s/adv:0.5280121117830276 - timing_s/update_actor:17.25321297906339 - timing_s/step:47.86772947758436 - timing_s/stop_profile:0.00012224353849887848 - timing_per_token_ms/gen:0.2770739731872695 - timing_per_token_ms/update_actor:0.09984209356826129 - timing_per_token_ms/adv:0.003055537234356805 - perf/total_num_tokens:172805 - perf/time_per_step:47.86772947758436 - perf/throughput:451.2565194911783 (TaskRunner pid=2122883) Training Progress: 27%|██▋ | 27/100 [24:07<52:34, 43.22s/it] (TaskRunner pid=2122883) step:28 - global_seqlen/min:20190 - global_seqlen/max:30705 - global_seqlen/minmax_diff:10515 - global_seqlen/balanced_min:24004 - global_seqlen/balanced_max:24200 - global_seqlen/mean:24035.75 - actor/entropy:0.18564951419830322 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5823308825492859 - training/rollout_probs_diff_mean:0.004079106263816357 - training/rollout_probs_diff_std:0.012803311459720135 - training/rollout_actor_probs_pearson_corr:0.998200535774231 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.61328125 - self_distillation/correct_sample_fraction:0.35546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.61328125 - rollout_corr/rollout_is_mean:1.0000269412994385 - rollout_corr/rollout_is_ratio_fraction_high:1.9231139958719723e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.692455983487889e-05 - rollout_corr/rollout_is_max:6.242704391479492 - rollout_corr/rollout_is_min:0.23590925335884094 - rollout_corr/rollout_is_std:0.03762301430106163 - rollout_corr/rollout_is_eff_sample_size:0.9985865096788494 - rollout_corr/rollout_is_seq_mean:1.0000364780426025 - rollout_corr/rollout_is_seq_std:0.0030162427574396133 - rollout_corr/rollout_is_seq_max:1.0107171535491943 - rollout_corr/rollout_is_seq_min:0.9907180666923523 - rollout_corr/rollout_is_seq_max_deviation:0.010717153549194336 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.25667032180354) - rollout_corr/training_log_ppl:np.float64(0.22454505835776217) - rollout_corr/kl:np.float64(0.0008663110850345435) - rollout_corr/k3_kl:np.float64(0.0010345751965132877) - rollout_corr/rollout_ppl:np.float64(1.2555355904623866) - rollout_corr/rollout_log_ppl:np.float64(0.22367874927294906) - rollout_corr/log_ppl_diff:np.float64(0.0008663090848131105) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023031624587019905) - rollout_corr/log_ppl_diff_max:np.float64(0.010847002267837524) - rollout_corr/log_ppl_diff_min:np.float64(-0.008875399827957153) - rollout_corr/ppl_ratio:np.float64(1.0008710592519492) - rollout_corr/chi2_token:np.float64(0.002394293434917927) - rollout_corr/chi2_seq:np.float64(0.8952626511454582) - actor/pg_loss:np.float64(0.005589029053226113) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011645046297417139) - actor/ppo_kl:np.float64(-2.7968522828381026e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.61328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.61328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.35546875) - self_distillation/token_reweight_w_mean:np.float64(274457.87072749576) - self_distillation/token_reweight_w_std:np.float64(3580252.375037855) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010159488301724) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0962374477967387) - self_distillation/empty_target_batch:np.float64(0.38671875) - actor/grad_norm:np.float64(0.5859245583415031) - perf/mfu/actor:np.float64(0.039479981257269675) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(149.19495010375977) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:1 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002572164870798588 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002572164870798588 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:406.2421875 - response_length/max:2082.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:406.2421875 - response_length_non_aborted/max:2082.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:344.875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.947402238845825e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5013809334486723) - timing_s/agent_loop/generate_sequences/max:np.float64(13.167735269293189) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.06728039180598) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.167735269293189) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:2082 - timing_s/gen:18.91630212403834 - timing_s/reward:0.06258906796574593 - timing_s/old_log_prob:2.4403274692595005 - timing_s/adv:0.5469215027987957 - timing_s/update_actor:17.414515420794487 - timing_s/step:39.468442087993026 - timing_s/stop_profile:0.00012546591460704803 - timing_per_token_ms/gen:0.18189101832764418 - timing_per_token_ms/update_actor:0.09056569599863998 - timing_per_token_ms/adv:0.0028443126530209983 - perf/total_num_tokens:192286 - perf/time_per_step:39.468442087993026 - perf/throughput:608.9865403456623 (TaskRunner pid=2122883) Training Progress: 28%|██▊ | 28/100 [24:46<50:31, 42.11s/it] (TaskRunner pid=2122883) step:29 - global_seqlen/min:12202 - global_seqlen/max:28063 - global_seqlen/minmax_diff:15861 - global_seqlen/balanced_min:19899 - global_seqlen/balanced_max:19913 - global_seqlen/mean:19906.5 - actor/entropy:0.200120210647583 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.751859188079834 - training/rollout_probs_diff_mean:0.00466552097350359 - training/rollout_probs_diff_std:0.013881048187613487 - training/rollout_actor_probs_pearson_corr:0.9979689717292786 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.4921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:0.9998300671577454 - rollout_corr/rollout_is_ratio_fraction_high:3.581490818760358e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001790745445759967 - rollout_corr/rollout_is_max:2.2318549156188965 - rollout_corr/rollout_is_min:0.07922481745481491 - rollout_corr/rollout_is_std:0.04076636582612991 - rollout_corr/rollout_is_eff_sample_size:0.9983403853566886 - rollout_corr/rollout_is_seq_mean:0.9997302293777466 - rollout_corr/rollout_is_seq_std:0.0028203565161675215 - rollout_corr/rollout_is_seq_max:1.0098977088928223 - rollout_corr/rollout_is_seq_min:0.9884768724441528 - rollout_corr/rollout_is_seq_max_deviation:0.011523127555847168 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.268780852202326) - rollout_corr/training_log_ppl:np.float64(0.23475862003397197) - rollout_corr/kl:np.float64(0.0012591780972996958) - rollout_corr/k3_kl:np.float64(0.0014774430976558506) - rollout_corr/rollout_ppl:np.float64(1.2671397938393056) - rollout_corr/rollout_log_ppl:np.float64(0.23349944039364345) - rollout_corr/log_ppl_diff:np.float64(0.0012591796403285116) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028744479350280017) - rollout_corr/log_ppl_diff_max:np.float64(0.015908241271972656) - rollout_corr/log_ppl_diff_min:np.float64(-0.008600950241088867) - rollout_corr/ppl_ratio:np.float64(1.001266701379791) - rollout_corr/chi2_token:np.float64(0.003420991124585271) - rollout_corr/chi2_seq:np.float64(0.8588014850392938) - actor/pg_loss:np.float64(0.008906474453397095) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020709222528694227) - actor/ppo_kl:np.float64(-5.746460594657776e-05) - actor/pg_clipfrac_lower:np.float64(1.800115205696784e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4921875) - self_distillation/token_reweight_w_mean:np.float64(400565.17963223695) - self_distillation/token_reweight_w_std:np.float64(4787453.377732925) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003657368943095) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14216147220577113) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.782679945230484) - perf/mfu/actor:np.float64(0.03311267643104445) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.62977409362793) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:2 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.018026838079094887 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.018026838079094887 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:327.203125 - response_length/max:1399.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:327.203125 - response_length_non_aborted/max:1399.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:294.875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00024269148707389832 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3502626232802868) - timing_s/agent_loop/generate_sequences/max:np.float64(8.990872727707028) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.25896129708417) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.990872727707028) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:1399 - timing_s/gen:15.044752394780517 - timing_s/reward:0.05442681908607483 - timing_s/old_log_prob:2.595429075881839 - timing_s/adv:0.6318970713764429 - timing_s/update_actor:17.295033048838377 - timing_s/step:35.68772314488888 - timing_s/stop_profile:0.00011236034333705902 - timing_per_token_ms/gen:0.17960881040519217 - timing_per_token_ms/update_actor:0.1086016693594955 - timing_per_token_ms/adv:0.003967906659737039 - perf/total_num_tokens:159252 - perf/time_per_step:35.68772314488888 - perf/throughput:557.796862500346 (TaskRunner pid=2122883) Training Progress: 29%|██▉ | 29/100 [25:22<47:42, 40.32s/it] (TaskRunner pid=2122883) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2122883) validation generation end (TaskRunner pid=2122883) [prompt] system (TaskRunner pid=2122883) (TaskRunner pid=2122883) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2122883) (TaskRunner pid=2122883) A (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) user (TaskRunner pid=2122883) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2122883) (TaskRunner pid=2122883) A: -0.01 (TaskRunner pid=2122883) B: 1.69 (TaskRunner pid=2122883) C: 2.49 (TaskRunner pid=2122883) D: 0.45 (TaskRunner pid=2122883) Please reason step by step. (TaskRunner pid=2122883) assistant (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) [response] (TaskRunner pid=2122883) The folding stability score of a protein sequence is typically determined through computational methods such as energy calculations, structure prediction algorithms, or experimental techniques like circular dichroism or NMR. However, without specific computational tools or data provided, it is not possible to calculate the exact folding stability score for the given sequence. The question provides four possible scores, and without additional context or computational data, the correct answer cannot be determined through logical reasoning alone. In the absence of specific information, the answer must be selected from the given options. Based on typical folding stability scores, the most reasonable choice is 0.45. (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) D (TaskRunner pid=2122883) (TaskRunner pid=2122883) [ground_truth] B (TaskRunner pid=2122883) [score] 0.0 (TaskRunner pid=2122883) [acc] 0.0 (TaskRunner pid=2122883) [pred] D (TaskRunner pid=2122883) [incorrect_format] 1 (TaskRunner pid=2122883) [feedback] (TaskRunner pid=2122883) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (WorkerDict pid=2123259) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20/actor (TaskRunner pid=2122883) step:30 - global_seqlen/min:13709 - global_seqlen/max:34661 - global_seqlen/minmax_diff:20952 - global_seqlen/balanced_min:22528 - global_seqlen/balanced_max:22562 - global_seqlen/mean:22552.5 - actor/entropy:0.17858071625232697 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8049387335777283 - training/rollout_probs_diff_mean:0.004061528015881777 - training/rollout_probs_diff_std:0.012804483063519001 - training/rollout_actor_probs_pearson_corr:0.9980953335762024 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.703125 - self_distillation/correct_sample_fraction:0.46484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.703125 - rollout_corr/rollout_is_mean:1.0000133514404297 - rollout_corr/rollout_is_ratio_fraction_high:2.0044097254867665e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012026457989122719 - rollout_corr/rollout_is_max:2.6347193717956543 - rollout_corr/rollout_is_min:0.1211763396859169 - rollout_corr/rollout_is_std:0.03737824037671089 - rollout_corr/rollout_is_eff_sample_size:0.9986048163833603 - rollout_corr/rollout_is_seq_mean:1.0000509023666382 - rollout_corr/rollout_is_seq_std:0.002637238474562764 - rollout_corr/rollout_is_seq_max:1.0075513124465942 - rollout_corr/rollout_is_seq_min:0.9900543093681335 - rollout_corr/rollout_is_seq_max_deviation:0.009945690631866455 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2358674858696759) - rollout_corr/training_log_ppl:np.float64(0.209219249751186) - rollout_corr/kl:np.float64(0.0009768563257512852) - rollout_corr/k3_kl:np.float64(0.000992308898958072) - rollout_corr/rollout_ppl:np.float64(1.2346490789204836) - rollout_corr/rollout_log_ppl:np.float64(0.2082423923711758) - rollout_corr/log_ppl_diff:np.float64(0.0009768573800101876) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023018353385850787) - rollout_corr/log_ppl_diff_max:np.float64(0.01429024338722229) - rollout_corr/log_ppl_diff_min:np.float64(-0.008996948599815369) - rollout_corr/ppl_ratio:np.float64(1.0009819658007473) - rollout_corr/chi2_token:np.float64(0.0020118139218539) - rollout_corr/chi2_seq:np.float64(0.9899320646654814) - actor/pg_loss:np.float64(0.005333056440576911) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007797006080636493) - actor/ppo_kl:np.float64(0.00013445281576274226) - actor/pg_clipfrac_lower:np.float64(5.455656264530262e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.46484375) - self_distillation/token_reweight_w_mean:np.float64(224932.845420863) - self_distillation/token_reweight_w_std:np.float64(3210198.2877844325) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003551209811121) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08737150816887151) - self_distillation/empty_target_batch:np.float64(0.296875) - actor/grad_norm:np.float64(0.4824788570404053) - perf/mfu/actor:np.float64(0.03816828358530715) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.78467178344727) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.3625) - val-aux/sciknoweval/reward/std@16:np.float64(0.2837411495944002) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.48234) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.23591887606702464) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.24532) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.24386037303171185) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.3643) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.28320348160575504) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.57412) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1685773153105991) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.1459) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.18180547299046135) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.37833999999999995) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.23334153448406283) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.63576) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.11234238956935315) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.07646) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.11232669827952936) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.39524000000000004) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1772001294855729) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.68072) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.07299155127558939) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.0341) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06470539559920649) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.40950000000000003) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.13626189541452347) - val-aux/sciknoweval/score/mean@16:np.float64(0.3625) - val-aux/sciknoweval/score/std@16:np.float64(0.2837411495944002) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.48234) - val-aux/sciknoweval/score/best@2/std:np.float64(0.23591887606702464) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.24532) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.24386037303171185) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.3643) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.28320348160575504) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.57412) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1685773153105991) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.1459) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.18180547299046135) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.37833999999999995) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.23334153448406283) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.63576) - val-aux/sciknoweval/score/best@8/std:np.float64(0.11234238956935315) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.07646) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.11232669827952936) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.39524000000000004) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.1772001294855729) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.68072) - val-aux/sciknoweval/score/best@16/std:np.float64(0.07299155127558939) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.0341) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06470539559920649) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.40950000000000003) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.13626189541452347) - val-core/sciknoweval/acc/mean@16:np.float64(0.3625) - val-aux/sciknoweval/acc/std@16:np.float64(0.2837411495944002) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.48234) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.23591887606702464) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.24532) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.24386037303171185) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.3643) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.28320348160575504) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.57412) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1685773153105991) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.1459) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.18180547299046135) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.37833999999999995) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.23334153448406283) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.63576) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.11234238956935315) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.07646) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.11232669827952936) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.39524000000000004) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.1772001294855729) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.68072) - val-core/sciknoweval/acc/best@16/std:np.float64(0.07299155127558939) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.0341) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06470539559920649) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.40950000000000003) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.13626189541452347) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.014523687548277814) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99976) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.003714879512080724) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9930199999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.01923124573696262) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.99664) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.013794079474263716) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.98634) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.02515921626743821) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99882) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.008310850581084193) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9766800000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0292466087573783) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99988) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0021559110231923074) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.96274) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.029101825861985193) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:2 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003663058625534177 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003663058625534177 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:389.765625 - response_length/max:1653.0 - response_length/min:124.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:389.765625 - response_length_non_aborted/max:1653.0 - response_length_non_aborted/min:124.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.0 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010188855230808258 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4586653970181942) - timing_s/agent_loop/generate_sequences/max:np.float64(10.59416701644659) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8259682986681582) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.59416701644659) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:1653 - timing_s/gen:16.57048099115491 - timing_s/reward:0.06174835376441479 - timing_s/old_log_prob:2.359806180000305 - timing_s/adv:0.5572093706578016 - timing_s/update_actor:17.026692068204284 - timing_s/step:36.666580606251955 - timing_s/testing:73.57902169972658 - timing_s/save_checkpoint:6.81705866008997 - timing_s/stop_profile:0.000125223770737648 - timing_per_token_ms/gen:0.16607016427294957 - timing_per_token_ms/update_actor:0.09437253113958699 - timing_per_token_ms/adv:0.0030884013449606566 - perf/total_num_tokens:180420 - perf/time_per_step:36.666580606251955 - perf/throughput:615.0696254494648 (TaskRunner pid=2122883) Training Progress: 30%|███ | 30/100 [27:19<1:13:55, 63.36s/it] (TaskRunner pid=2122883) step:31 - global_seqlen/min:14691 - global_seqlen/max:32016 - global_seqlen/minmax_diff:17325 - global_seqlen/balanced_min:21111 - global_seqlen/balanced_max:23092 - global_seqlen/mean:21372.875 - actor/entropy:0.18767856061458588 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4564788341522217 - training/rollout_probs_diff_mean:0.0042718215845525265 - training/rollout_probs_diff_std:0.012846951372921467 - training/rollout_actor_probs_pearson_corr:0.9981147050857544 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.640625 - self_distillation/correct_sample_fraction:0.3828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.640625 - rollout_corr/rollout_is_mean:0.9999961256980896 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.654637425323017e-05 - rollout_corr/rollout_is_max:1.8308576345443726 - rollout_corr/rollout_is_min:0.2759072780609131 - rollout_corr/rollout_is_std:0.038326188921928406 - rollout_corr/rollout_is_eff_sample_size:0.9985331387675961 - rollout_corr/rollout_is_seq_mean:1.0000230073928833 - rollout_corr/rollout_is_seq_std:0.002981269033625722 - rollout_corr/rollout_is_seq_max:1.0086121559143066 - rollout_corr/rollout_is_seq_min:0.9902340173721313 - rollout_corr/rollout_is_seq_max_deviation:0.009765982627868652 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2463521412573755) - rollout_corr/training_log_ppl:np.float64(0.21727092210494447) - rollout_corr/kl:np.float64(0.0009897176428506071) - rollout_corr/k3_kl:np.float64(0.0010703946789476504) - rollout_corr/rollout_ppl:np.float64(1.2450866210274398) - rollout_corr/rollout_log_ppl:np.float64(0.21628120230161585) - rollout_corr/log_ppl_diff:np.float64(0.0009897198033286259) - rollout_corr/log_ppl_abs_diff:np.float64(0.002593131721368991) - rollout_corr/log_ppl_diff_max:np.float64(0.013398468494415283) - rollout_corr/log_ppl_diff_min:np.float64(-0.008827343583106995) - rollout_corr/ppl_ratio:np.float64(1.0009958858136088) - rollout_corr/chi2_token:np.float64(0.0023120150435715914) - rollout_corr/chi2_seq:np.float64(0.798170599155128) - actor/pg_loss:np.float64(0.004743849392980337) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009500816249783384) - actor/ppo_kl:np.float64(0.00011712995112134195) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.640625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3828125) - self_distillation/token_reweight_w_mean:np.float64(405654.28687982494) - self_distillation/token_reweight_w_std:np.float64(5009496.861719544) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019375609699637) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10904305649455637) - self_distillation/empty_target_batch:np.float64(0.359375) - actor/grad_norm:np.float64(0.5787822306156158) - perf/mfu/actor:np.float64(0.03477636111962868) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.78802490234375) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:2 - critic/score/mean:0.3828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0031128779519349337 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0031128779519349337 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:345.40234375 - response_length/max:3325.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:345.40234375 - response_length_non_aborted/max:3325.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:322.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001608133316040039 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1231673527508974) - timing_s/agent_loop/generate_sequences/max:np.float64(18.172121800482273) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3470864338232786) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.172121800482273) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:3325 - timing_s/gen:23.99966412782669 - timing_s/reward:0.05674261972308159 - timing_s/old_log_prob:2.4767898563295603 - timing_s/adv:0.7946092803031206 - timing_s/update_actor:17.73706099949777 - timing_s/step:45.163662960752845 - timing_s/stop_profile:0.00011231191456317902 - timing_per_token_ms/gen:0.2714187951983838 - timing_per_token_ms/update_actor:0.10373581583840365 - timing_per_token_ms/adv:0.004647299908781111 - perf/total_num_tokens:170983 - perf/time_per_step:45.163662960752845 - perf/throughput:473.23165569127985 (TaskRunner pid=2122883) Training Progress: 31%|███ | 31/100 [28:05<1:06:36, 57.92s/it] (TaskRunner pid=2122883) step:32 - global_seqlen/min:16951 - global_seqlen/max:28535 - global_seqlen/minmax_diff:11584 - global_seqlen/balanced_min:22298 - global_seqlen/balanced_max:22302 - global_seqlen/mean:22300.0 - actor/entropy:0.18449454009532928 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.508357584476471 - training/rollout_probs_diff_mean:0.004164127632975578 - training/rollout_probs_diff_std:0.012944702059030533 - training/rollout_actor_probs_pearson_corr:0.9981325268745422 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.765625 - self_distillation/correct_sample_fraction:0.4765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.765625 - rollout_corr/rollout_is_mean:1.0003676414489746 - rollout_corr/rollout_is_ratio_fraction_high:4.371011527837254e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.834775846684352e-05 - rollout_corr/rollout_is_max:2.890216827392578 - rollout_corr/rollout_is_min:0.006242398638278246 - rollout_corr/rollout_is_std:0.038238998502492905 - rollout_corr/rollout_is_eff_sample_size:0.9985409835772864 - rollout_corr/rollout_is_seq_mean:1.0004096031188965 - rollout_corr/rollout_is_seq_std:0.0029842921067029238 - rollout_corr/rollout_is_seq_max:1.0088164806365967 - rollout_corr/rollout_is_seq_min:0.9924617409706116 - rollout_corr/rollout_is_seq_max_deviation:0.00881648063659668 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2568057388998568) - rollout_corr/training_log_ppl:np.float64(0.22407739836489782) - rollout_corr/kl:np.float64(0.0006947221124469216) - rollout_corr/k3_kl:np.float64(0.0011480630730034136) - rollout_corr/rollout_ppl:np.float64(1.2559078889898956) - rollout_corr/rollout_log_ppl:np.float64(0.22338267411396373) - rollout_corr/log_ppl_diff:np.float64(0.0006947242509340867) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024179289903258905) - rollout_corr/log_ppl_diff_max:np.float64(0.014996767044067383) - rollout_corr/log_ppl_diff_min:np.float64(-0.00923621654510498) - rollout_corr/ppl_ratio:np.float64(1.000700187170878) - rollout_corr/chi2_token:np.float64(0.0031753990333527327) - rollout_corr/chi2_seq:np.float64(0.8116656658239663) - actor/pg_loss:np.float64(0.006690458743833005) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001721466425806284) - actor/ppo_kl:np.float64(0.0001673200823937293) - actor/pg_clipfrac_lower:np.float64(7.89141449786257e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.765625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4765625) - self_distillation/token_reweight_w_mean:np.float64(378276.8863981217) - self_distillation/token_reweight_w_std:np.float64(4767918.035825024) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.99896085774526) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13528311986010522) - self_distillation/empty_target_batch:np.float64(0.234375) - actor/grad_norm:np.float64(0.7006156742572784) - perf/mfu/actor:np.float64(0.03776695122166555) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.70980548858643) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:2 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.023166360333561897 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.023166360333561897 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:357.46875 - response_length/max:1160.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:357.46875 - response_length_non_aborted/max:1160.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:339.40625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.496323764324188e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1231606528162956) - timing_s/agent_loop/generate_sequences/max:np.float64(8.21338595636189) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.436626313610759) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.21338595636189) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:1160 - timing_s/gen:14.110106036067009 - timing_s/reward:0.05653159320354462 - timing_s/old_log_prob:2.364559957757592 - timing_s/adv:0.5321131441742182 - timing_s/update_actor:17.017004685476422 - timing_s/step:34.17690355889499 - timing_s/stop_profile:0.00011146441102027893 - timing_per_token_ms/gen:0.15418858768322197 - timing_per_token_ms/update_actor:0.09538679756432972 - timing_per_token_ms/adv:0.0029826969964922546 - perf/total_num_tokens:178400 - perf/time_per_step:34.17690355889499 - perf/throughput:652.4874309216386 (TaskRunner pid=2122883) Training Progress: 32%|███▏ | 32/100 [28:39<57:35, 50.81s/it] (TaskRunner pid=2122883) step:33 - global_seqlen/min:13524 - global_seqlen/max:31971 - global_seqlen/minmax_diff:18447 - global_seqlen/balanced_min:23235 - global_seqlen/balanced_max:23263 - global_seqlen/mean:23257.125 - actor/entropy:0.18245993554592133 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8571684956550598 - training/rollout_probs_diff_mean:0.004132695496082306 - training/rollout_probs_diff_std:0.013101710006594658 - training/rollout_actor_probs_pearson_corr:0.9980323314666748 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.000072956085205 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00011418250505812466 - rollout_corr/rollout_is_max:1.9873102903366089 - rollout_corr/rollout_is_min:0.05242491513490677 - rollout_corr/rollout_is_std:0.038057755678892136 - rollout_corr/rollout_is_eff_sample_size:0.9985539396692393 - rollout_corr/rollout_is_seq_mean:1.0000982284545898 - rollout_corr/rollout_is_seq_std:0.002705891616642475 - rollout_corr/rollout_is_seq_max:1.0117093324661255 - rollout_corr/rollout_is_seq_min:0.9912111759185791 - rollout_corr/rollout_is_seq_max_deviation:0.011709332466125488 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2483157757669687) - rollout_corr/training_log_ppl:np.float64(0.21742995794920716) - rollout_corr/kl:np.float64(0.001100853100311383) - rollout_corr/k3_kl:np.float64(0.0011483149023092665) - rollout_corr/rollout_ppl:np.float64(1.2468630676157773) - rollout_corr/rollout_log_ppl:np.float64(0.21632910224434454) - rollout_corr/log_ppl_diff:np.float64(0.0011008557048626244) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024914654495660216) - rollout_corr/log_ppl_diff_max:np.float64(0.01017981767654419) - rollout_corr/log_ppl_diff_min:np.float64(-0.013981759548187256) - rollout_corr/ppl_ratio:np.float64(1.001106784446165) - rollout_corr/chi2_token:np.float64(0.0023680569138377905) - rollout_corr/chi2_seq:np.float64(3.1154466739390045) - actor/pg_loss:np.float64(0.006939625251106918) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017952929888451763) - actor/ppo_kl:np.float64(0.00026459825367375345) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.484375) - self_distillation/token_reweight_w_mean:np.float64(452102.9714282772) - self_distillation/token_reweight_w_std:np.float64(5585046.191044878) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0017326495144516) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12278274190612137) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.6988479644060135) - perf/mfu/actor:np.float64(0.038683908285535445) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.7328338623047) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:2 - critic/score/mean:0.484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007901948411017656 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0007901948411017656 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:376.31640625 - response_length/max:1468.0 - response_length/min:106.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:376.31640625 - response_length_non_aborted/max:1468.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:350.46875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.475834667682648e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3122014049440622) - timing_s/agent_loop/generate_sequences/max:np.float64(9.940794771537185) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.687579854595242) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.940794771537185) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:1468 - timing_s/gen:15.825185041874647 - timing_s/reward:0.06117861159145832 - timing_s/old_log_prob:2.3811042718589306 - timing_s/adv:0.5547373704612255 - timing_s/update_actor:17.3419884853065 - timing_s/step:36.251486498862505 - timing_s/stop_profile:0.00011686794459819794 - timing_per_token_ms/gen:0.16426902479706287 - timing_per_token_ms/update_actor:0.09320793351127074 - timing_per_token_ms/adv:0.002981545281613836 - perf/total_num_tokens:186057 - perf/time_per_step:36.251486498862505 - perf/throughput:641.5495541328977 (TaskRunner pid=2122883) Training Progress: 33%|███▎ | 33/100 [29:15<51:52, 46.46s/it] (TaskRunner pid=2122883) step:34 - global_seqlen/min:12850 - global_seqlen/max:27960 - global_seqlen/minmax_diff:15110 - global_seqlen/balanced_min:19894 - global_seqlen/balanced_max:19913 - global_seqlen/mean:19909.5 - actor/entropy:0.1824377477169037 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8196942806243896 - training/rollout_probs_diff_mean:0.004456874914467335 - training/rollout_probs_diff_std:0.013786375522613525 - training/rollout_actor_probs_pearson_corr:0.9977785348892212 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.37890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:0.9999569654464722 - rollout_corr/rollout_is_ratio_fraction_high:6.199474591994658e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001363884366583079 - rollout_corr/rollout_is_max:5.546136856079102 - rollout_corr/rollout_is_min:0.31902217864990234 - rollout_corr/rollout_is_std:0.04029429703950882 - rollout_corr/rollout_is_eff_sample_size:0.9983787637329614 - rollout_corr/rollout_is_seq_mean:0.9998646378517151 - rollout_corr/rollout_is_seq_std:0.0028597437776625156 - rollout_corr/rollout_is_seq_max:1.0103107690811157 - rollout_corr/rollout_is_seq_min:0.992056131362915 - rollout_corr/rollout_is_seq_max_deviation:0.010310769081115723 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.236810412723571) - rollout_corr/training_log_ppl:np.float64(0.2092786459106719) - rollout_corr/kl:np.float64(0.0015006025356960606) - rollout_corr/k3_kl:np.float64(0.0015799338922874995) - rollout_corr/rollout_ppl:np.float64(1.2348826574161649) - rollout_corr/rollout_log_ppl:np.float64(0.20777804144017864) - rollout_corr/log_ppl_diff:np.float64(0.001500604470493272) - rollout_corr/log_ppl_abs_diff:np.float64(0.002933290699729696) - rollout_corr/log_ppl_diff_max:np.float64(0.01829591393470764) - rollout_corr/log_ppl_diff_min:np.float64(-0.015577703714370728) - rollout_corr/ppl_ratio:np.float64(1.0015093663241714) - rollout_corr/chi2_token:np.float64(0.003512197406962514) - rollout_corr/chi2_seq:np.float64(1.3710486507043242) - actor/pg_loss:np.float64(0.005380709771998227) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017725324896673555) - actor/ppo_kl:np.float64(0.00034614508305708114) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.37890625) - self_distillation/token_reweight_w_mean:np.float64(492608.6426240674) - self_distillation/token_reweight_w_std:np.float64(5396903.603151628) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0017004983965307) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09993290061538573) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.6676810383796692) - perf/mfu/actor:np.float64(0.033570212113188724) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.6746826171875) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:2 - critic/score/mean:0.37890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.37890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00712474575266242 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00712474575266242 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:315.046875 - response_length/max:1277.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:315.046875 - response_length_non_aborted/max:1277.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:307.125 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.605847299098969e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2697386275976896) - timing_s/agent_loop/generate_sequences/max:np.float64(8.63854743912816) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0711136244499357) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.63854743912816) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:1277 - timing_s/gen:14.853934342041612 - timing_s/reward:0.05592981539666653 - timing_s/old_log_prob:2.4257997516542673 - timing_s/adv:0.5230183508247137 - timing_s/update_actor:16.553620226681232 - timing_s/step:34.49927463196218 - timing_s/stop_profile:0.00011586584150791168 - timing_per_token_ms/gen:0.18417316795667327 - timing_per_token_ms/update_actor:0.1039304115289261 - timing_per_token_ms/adv:0.003283723541680565 - perf/total_num_tokens:159276 - perf/time_per_step:34.49927463196218 - perf/throughput:577.099090122743 (TaskRunner pid=2122883) Training Progress: 34%|███▍ | 34/100 [29:50<47:10, 42.88s/it] (TaskRunner pid=2122883) step:35 - global_seqlen/min:11375 - global_seqlen/max:30625 - global_seqlen/minmax_diff:19250 - global_seqlen/balanced_min:20219 - global_seqlen/balanced_max:25072 - global_seqlen/mean:20838.875 - actor/entropy:0.16443729400634766 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5101940035820007 - training/rollout_probs_diff_mean:0.003969142679125071 - training/rollout_probs_diff_std:0.012587097473442554 - training/rollout_actor_probs_pearson_corr:0.9980363845825195 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.4921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:0.9996427893638611 - rollout_corr/rollout_is_ratio_fraction_high:2.3485479687224142e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014091287448536605 - rollout_corr/rollout_is_max:2.2103183269500732 - rollout_corr/rollout_is_min:0.18281003832817078 - rollout_corr/rollout_is_std:0.03684016689658165 - rollout_corr/rollout_is_eff_sample_size:0.9986435717515305 - rollout_corr/rollout_is_seq_mean:0.9995678663253784 - rollout_corr/rollout_is_seq_std:0.0028214508201926947 - rollout_corr/rollout_is_seq_max:1.007384181022644 - rollout_corr/rollout_is_seq_min:0.9897209405899048 - rollout_corr/rollout_is_seq_max_deviation:0.010279059410095215 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2260203030891716) - rollout_corr/training_log_ppl:np.float64(0.2011226593167521) - rollout_corr/kl:np.float64(0.0013201789048196133) - rollout_corr/k3_kl:np.float64(0.0011584838699150168) - rollout_corr/rollout_ppl:np.float64(1.2243667864240706) - rollout_corr/rollout_log_ppl:np.float64(0.1998024799686391) - rollout_corr/log_ppl_diff:np.float64(0.0013201793481130153) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027976697601843625) - rollout_corr/log_ppl_diff_max:np.float64(0.01783977448940277) - rollout_corr/log_ppl_diff_min:np.float64(-0.006872743368148804) - rollout_corr/ppl_ratio:np.float64(1.001327101374045) - rollout_corr/chi2_token:np.float64(0.0018826127052307129) - rollout_corr/chi2_seq:np.float64(0.425460196333006) - actor/pg_loss:np.float64(0.005690077436156571) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011552226678759325) - actor/ppo_kl:np.float64(-3.631153518313113e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4921875) - self_distillation/token_reweight_w_mean:np.float64(351749.36200882215) - self_distillation/token_reweight_w_std:np.float64(3531892.006555579) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999812446301803) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09159481288224924) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.683478482067585) - perf/mfu/actor:np.float64(0.032190014645935336) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.899169921875) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:2 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005905130412429571 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005905130412429571 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:332.65234375 - response_length/max:6393.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:332.65234375 - response_length_non_aborted/max:6393.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:318.5625 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.329454481601715e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1087023857980967) - timing_s/agent_loop/generate_sequences/max:np.float64(32.07997610978782) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0471223126878613) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(32.07997610978782) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:6393 - timing_s/gen:38.27626817114651 - timing_s/reward:0.04927484318614006 - timing_s/old_log_prob:2.462094532325864 - timing_s/adv:1.0785468369722366 - timing_s/update_actor:18.53664114885032 - timing_s/step:60.43870029784739 - timing_s/stop_profile:0.00011680647730827332 - timing_per_token_ms/gen:0.44946826725474126 - timing_per_token_ms/update_actor:0.11119027028120713 - timing_per_token_ms/adv:0.006469560118841808 - perf/total_num_tokens:166711 - perf/time_per_step:60.43870029784739 - perf/throughput:344.7935659983444 (TaskRunner pid=2122883) Training Progress: 35%|███▌ | 35/100 [30:50<52:10, 48.16s/it] (TaskRunner pid=2122883) step:36 - global_seqlen/min:13174 - global_seqlen/max:28443 - global_seqlen/minmax_diff:15269 - global_seqlen/balanced_min:19485 - global_seqlen/balanced_max:19725 - global_seqlen/mean:19531.25 - actor/entropy:0.1771612912416458 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.46555420756340027 - training/rollout_probs_diff_mean:0.004134125541895628 - training/rollout_probs_diff_std:0.012505745515227318 - training/rollout_actor_probs_pearson_corr:0.9981387853622437 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.58203125 - self_distillation/correct_sample_fraction:0.33203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.58203125 - rollout_corr/rollout_is_mean:0.9998790621757507 - rollout_corr/rollout_is_ratio_fraction_high:1.2940963642904535e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.058674186235294e-05 - rollout_corr/rollout_is_max:2.0104777812957764 - rollout_corr/rollout_is_min:0.375485897064209 - rollout_corr/rollout_is_std:0.037845734506845474 - rollout_corr/rollout_is_eff_sample_size:0.9985693923286193 - rollout_corr/rollout_is_seq_mean:0.9999881386756897 - rollout_corr/rollout_is_seq_std:0.0026660633739084005 - rollout_corr/rollout_is_seq_max:1.010130763053894 - rollout_corr/rollout_is_seq_min:0.9915474057197571 - rollout_corr/rollout_is_seq_max_deviation:0.010130763053894043 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2288217013701797) - rollout_corr/training_log_ppl:np.float64(0.20217849771142937) - rollout_corr/kl:np.float64(0.0010742654510309535) - rollout_corr/k3_kl:np.float64(0.0011284284318264781) - rollout_corr/rollout_ppl:np.float64(1.2274580257944763) - rollout_corr/rollout_log_ppl:np.float64(0.2011042319209082) - rollout_corr/log_ppl_diff:np.float64(0.0010742657905211672) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025326748873339966) - rollout_corr/log_ppl_diff_max:np.float64(0.01431196928024292) - rollout_corr/log_ppl_diff_min:np.float64(-0.012727819383144379) - rollout_corr/ppl_ratio:np.float64(1.0010803742334247) - rollout_corr/chi2_token:np.float64(0.0023828367702662945) - rollout_corr/chi2_seq:np.float64(0.3923698428552598) - actor/pg_loss:np.float64(0.0049333591014146805) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015366055267804768) - actor/ppo_kl:np.float64(0.00018557149614117918) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.58203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.58203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.33203125) - self_distillation/token_reweight_w_mean:np.float64(456639.86721927836) - self_distillation/token_reweight_w_std:np.float64(5056710.059664073) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003420263528824) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09454223301145248) - self_distillation/empty_target_batch:np.float64(0.41796875) - actor/grad_norm:np.float64(0.5455604642629623) - perf/mfu/actor:np.float64(0.03263065275624072) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.63262939453125) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:2 - critic/score/mean:0.33203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.33203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006163133657537401 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006163133657537401 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:301.8515625 - response_length/max:1835.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:301.8515625 - response_length_non_aborted/max:1835.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:308.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001458302140235901 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2272989992052317) - timing_s/agent_loop/generate_sequences/max:np.float64(10.854171784594655) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8903689130966086) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.854171784594655) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:432 - timing_s/agent_loop/slowest/response_length:1835 - timing_s/gen:16.89564960449934 - timing_s/reward:0.05678628943860531 - timing_s/old_log_prob:2.4063503593206406 - timing_s/adv:0.5976713821291924 - timing_s/update_actor:17.047957718372345 - timing_s/step:37.09008174203336 - timing_s/stop_profile:2.9616057872772217e-05 - timing_per_token_ms/gen:0.21864598188911327 - timing_per_token_ms/update_actor:0.10910692939758301 - timing_per_token_ms/adv:0.003825096845626831 - perf/total_num_tokens:156250 - perf/time_per_step:37.09008174203336 - perf/throughput:526.5895647209014 (TaskRunner pid=2122883) Training Progress: 36%|███▌ | 36/100 [31:27<47:50, 44.85s/it] (TaskRunner pid=2122883) step:37 - global_seqlen/min:9999 - global_seqlen/max:26921 - global_seqlen/minmax_diff:16922 - global_seqlen/balanced_min:17705 - global_seqlen/balanced_max:17719 - global_seqlen/mean:17715.375 - actor/entropy:0.18285039067268372 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2932346761226654 - training/rollout_probs_diff_mean:0.0044187577441334724 - training/rollout_probs_diff_std:0.012828843668103218 - training/rollout_actor_probs_pearson_corr:0.9980629086494446 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73046875 - self_distillation/correct_sample_fraction:0.3828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73046875 - rollout_corr/rollout_is_mean:0.9999576210975647 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.058861981728114e-05 - rollout_corr/rollout_is_max:1.6349238157272339 - rollout_corr/rollout_is_min:0.33974406123161316 - rollout_corr/rollout_is_std:0.0383153036236763 - rollout_corr/rollout_is_eff_sample_size:0.998533970787023 - rollout_corr/rollout_is_seq_mean:1.000052809715271 - rollout_corr/rollout_is_seq_std:0.0032219039276242256 - rollout_corr/rollout_is_seq_max:1.0131325721740723 - rollout_corr/rollout_is_seq_min:0.9905257225036621 - rollout_corr/rollout_is_seq_max_deviation:0.013132572174072266 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.227560852188617) - rollout_corr/training_log_ppl:np.float64(0.20202259431243874) - rollout_corr/kl:np.float64(0.001208621201424176) - rollout_corr/k3_kl:np.float64(0.0011305918565653883) - rollout_corr/rollout_ppl:np.float64(1.226063481066376) - rollout_corr/rollout_log_ppl:np.float64(0.20081397160538472) - rollout_corr/log_ppl_diff:np.float64(0.001208622707054019) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029012424347456545) - rollout_corr/log_ppl_diff_max:np.float64(0.02870221436023712) - rollout_corr/log_ppl_diff_min:np.float64(-0.010198116302490234) - rollout_corr/ppl_ratio:np.float64(1.001217273529619) - rollout_corr/chi2_token:np.float64(0.0019719612319022417) - rollout_corr/chi2_seq:np.float64(0.3310001604259014) - actor/pg_loss:np.float64(0.0050835959846153855) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015021474537206814) - actor/ppo_kl:np.float64(0.0003870755170929385) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3828125) - self_distillation/token_reweight_w_mean:np.float64(1010404.8824171) - self_distillation/token_reweight_w_std:np.float64(9621199.624808073) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0038477664347738) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11127460616989993) - self_distillation/empty_target_batch:np.float64(0.26953125) - actor/grad_norm:np.float64(0.6599202007055283) - perf/mfu/actor:np.float64(0.030020447907232815) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.63792419433594) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:2 - critic/score/mean:0.3828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.024739090353250504 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.024739090353250504 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:257.88671875 - response_length/max:1172.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:257.88671875 - response_length_non_aborted/max:1172.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.71875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.444721758365631e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1212733164429665) - timing_s/agent_loop/generate_sequences/max:np.float64(7.338839437812567) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.4418798948754556) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.338839437812567) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1172 - timing_s/gen:12.895120618864894 - timing_s/reward:0.05457581765949726 - timing_s/old_log_prob:2.357027480378747 - timing_s/adv:0.533143537119031 - timing_s/update_actor:16.804465716704726 - timing_s/step:32.729485500603914 - timing_s/stop_profile:0.0001118052750825882 - timing_per_token_ms/gen:0.19532438568995128 - timing_per_token_ms/update_actor:0.11857260795145973 - timing_per_token_ms/adv:0.0037618702477299447 - perf/total_num_tokens:141723 - perf/time_per_step:32.729485500603914 - perf/throughput:541.2665286068466 (TaskRunner pid=2122883) Training Progress: 37%|███▋ | 37/100 [32:00<43:17, 41.22s/it] (TaskRunner pid=2122883) step:38 - global_seqlen/min:20533 - global_seqlen/max:27910 - global_seqlen/minmax_diff:7377 - global_seqlen/balanced_min:22402 - global_seqlen/balanced_max:22407 - global_seqlen/mean:22404.75 - actor/entropy:0.16610024869441986 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27665841579437256 - training/rollout_probs_diff_mean:0.0037956703454256058 - training/rollout_probs_diff_std:0.01207498274743557 - training/rollout_actor_probs_pearson_corr:0.9981511831283569 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.50390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:0.9999752044677734 - rollout_corr/rollout_is_ratio_fraction_high:2.2189184164744802e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.656755431322381e-05 - rollout_corr/rollout_is_max:2.013848304748535 - rollout_corr/rollout_is_min:0.1109480932354927 - rollout_corr/rollout_is_std:0.03653473034501076 - rollout_corr/rollout_is_eff_sample_size:0.9986667550813965 - rollout_corr/rollout_is_seq_mean:0.9999271631240845 - rollout_corr/rollout_is_seq_std:0.0029462629463523626 - rollout_corr/rollout_is_seq_max:1.010036587715149 - rollout_corr/rollout_is_seq_min:0.9871081113815308 - rollout_corr/rollout_is_seq_max_deviation:0.012891888618469238 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2210421110503376) - rollout_corr/training_log_ppl:np.float64(0.19586755773343612) - rollout_corr/kl:np.float64(0.0014365888042209285) - rollout_corr/k3_kl:np.float64(0.0013220797646340543) - rollout_corr/rollout_ppl:np.float64(1.219245539046824) - rollout_corr/rollout_log_ppl:np.float64(0.19443096709437668) - rollout_corr/log_ppl_diff:np.float64(0.0014365906390594319) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028802211018046364) - rollout_corr/log_ppl_diff_max:np.float64(0.02172189950942993) - rollout_corr/log_ppl_diff_min:np.float64(-0.012236267328262329) - rollout_corr/ppl_ratio:np.float64(1.0014454184565693) - rollout_corr/chi2_token:np.float64(0.002300215419381857) - rollout_corr/chi2_seq:np.float64(1.5557226871605963) - actor/pg_loss:np.float64(0.0065355622209608555) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0022695219383876974) - actor/ppo_kl:np.float64(0.0004982942221190001) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.50390625) - self_distillation/token_reweight_w_mean:np.float64(707782.7439720565) - self_distillation/token_reweight_w_std:np.float64(6931265.693862626) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0016436446458101) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11134860079619102) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.7043651267886162) - perf/mfu/actor:np.float64(0.037222466709103126) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.62496185302734) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:2 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008276565931737423 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008276565931737423 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:352.0859375 - response_length/max:1133.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:352.0859375 - response_length_non_aborted/max:1133.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:348.0625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016898848116397858 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1377350594848394) - timing_s/agent_loop/generate_sequences/max:np.float64(8.493686899542809) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.369795570768474) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.493686899542809) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:1133 - timing_s/gen:14.338888732716441 - timing_s/reward:0.059045134112238884 - timing_s/old_log_prob:2.425033800303936 - timing_s/adv:0.5446427445858717 - timing_s/update_actor:16.951616071164608 - timing_s/step:34.40807002224028 - timing_s/stop_profile:0.00013046711683273315 - timing_per_token_ms/gen:0.1590841273294921 - timing_per_token_ms/update_actor:0.0945760166435946 - timing_per_token_ms/adv:0.003038656672055433 - perf/total_num_tokens:179238 - perf/time_per_step:34.40807002224028 - perf/throughput:651.14811686672 (TaskRunner pid=2122883) Training Progress: 38%|███▊ | 38/100 [32:34<40:30, 39.19s/it] (TaskRunner pid=2122883) step:39 - global_seqlen/min:16482 - global_seqlen/max:26691 - global_seqlen/minmax_diff:10209 - global_seqlen/balanced_min:20524 - global_seqlen/balanced_max:22575 - global_seqlen/mean:20860.375 - actor/entropy:0.15684472024440765 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3085607886314392 - training/rollout_probs_diff_mean:0.003755766898393631 - training/rollout_probs_diff_std:0.012273234315216541 - training/rollout_actor_probs_pearson_corr:0.9981006979942322 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8203125 - self_distillation/correct_sample_fraction:0.4140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8203125 - rollout_corr/rollout_is_mean:0.9999597072601318 - rollout_corr/rollout_is_ratio_fraction_high:3.471458694548346e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.6286117139970884e-05 - rollout_corr/rollout_is_max:2.4837749004364014 - rollout_corr/rollout_is_min:0.06344368308782578 - rollout_corr/rollout_is_std:0.036595042794942856 - rollout_corr/rollout_is_eff_sample_size:0.998662356110423 - rollout_corr/rollout_is_seq_mean:0.9999072551727295 - rollout_corr/rollout_is_seq_std:0.0031039086170494556 - rollout_corr/rollout_is_seq_max:1.0133365392684937 - rollout_corr/rollout_is_seq_min:0.9888757467269897 - rollout_corr/rollout_is_seq_max_deviation:0.013336539268493652 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.226109724957496) - rollout_corr/training_log_ppl:np.float64(0.20032589661423117) - rollout_corr/kl:np.float64(0.001165302209578556) - rollout_corr/k3_kl:np.float64(0.0011575412143542962) - rollout_corr/rollout_ppl:np.float64(1.2246328527107835) - rollout_corr/rollout_log_ppl:np.float64(0.19916059231036343) - rollout_corr/log_ppl_diff:np.float64(0.0011653043038677424) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028654705092776567) - rollout_corr/log_ppl_diff_max:np.float64(0.016280263662338257) - rollout_corr/log_ppl_diff_min:np.float64(-0.011811941862106323) - rollout_corr/ppl_ratio:np.float64(1.0011730468831956) - rollout_corr/chi2_token:np.float64(0.0023117719683796167) - rollout_corr/chi2_seq:np.float64(0.42900348524563015) - actor/pg_loss:np.float64(0.008025142014957964) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002576867142693118) - actor/ppo_kl:np.float64(0.0001556854837474475) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4140625) - self_distillation/token_reweight_w_mean:np.float64(1097881.6426437146) - self_distillation/token_reweight_w_std:np.float64(11470257.020113224) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0032665592152625) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1485372949828161) - self_distillation/empty_target_batch:np.float64(0.1796875) - actor/grad_norm:np.float64(0.9034479111433029) - perf/mfu/actor:np.float64(0.03487259499439749) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.6538429260254) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:2 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.021414561197161674 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.021414561197161674 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:337.57421875 - response_length/max:3975.0 - response_length/min:84.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:337.57421875 - response_length_non_aborted/max:3975.0 - response_length_non_aborted/min:84.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.3125 - prompt_length/max:498.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015822425484657288 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0175010822713375) - timing_s/agent_loop/generate_sequences/max:np.float64(21.103810861706734) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.054622892734187) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.103810861706734) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:3975 - timing_s/gen:27.206532564014196 - timing_s/reward:0.05892925336956978 - timing_s/old_log_prob:2.558308009058237 - timing_s/adv:0.6190387345850468 - timing_s/update_actor:17.218717677518725 - timing_s/step:47.74760019779205 - timing_s/stop_profile:0.00010721571743488312 - timing_per_token_ms/gen:0.31482119168254896 - timing_per_token_ms/update_actor:0.10317838052718806 - timing_per_token_ms/adv:0.0037094175834869146 - perf/total_num_tokens:166883 - perf/time_per_step:47.74760019779205 - perf/throughput:436.88844912805956 (TaskRunner pid=2122883) Training Progress: 39%|███▉ | 39/100 [33:22<42:28, 41.78s/it] (TaskRunner pid=2122883) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2122883) validation generation end (TaskRunner pid=2122883) [prompt] system (TaskRunner pid=2122883) (TaskRunner pid=2122883) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2122883) (TaskRunner pid=2122883) A (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) user (TaskRunner pid=2122883) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2122883) (TaskRunner pid=2122883) A: -0.01 (TaskRunner pid=2122883) B: 1.69 (TaskRunner pid=2122883) C: 2.49 (TaskRunner pid=2122883) D: 0.45 (TaskRunner pid=2122883) Please reason step by step. (TaskRunner pid=2122883) assistant (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) [response] (TaskRunner pid=2122883) The folding stability score of a protein sequence is typically determined through computational methods or experimental techniques that assess the stability of the protein's folded structure. Without specific computational tools or experimental data provided, it is not possible to calculate the exact folding stability score for the given sequence. However, based on the options provided, the most reasonable value for a folding stability score would be a positive number, as stability scores are often reported as positive values. Among the options, 1.69 is the most plausible value for a folding stability score. Therefore, the correct answer is 1.69. (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) B (TaskRunner pid=2122883) (TaskRunner pid=2122883) [ground_truth] B (TaskRunner pid=2122883) [score] 1.0 (TaskRunner pid=2122883) [acc] 1.0 (TaskRunner pid=2122883) [pred] B (TaskRunner pid=2122883) [incorrect_format] 1 (TaskRunner pid=2122883) [feedback] (TaskRunner pid=2122883) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (WorkerDict pid=2123259) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30/actor (TaskRunner pid=2122883) step:40 - global_seqlen/min:13634 - global_seqlen/max:24094 - global_seqlen/minmax_diff:10460 - global_seqlen/balanced_min:18012 - global_seqlen/balanced_max:18015 - global_seqlen/mean:18013.375 - actor/entropy:0.17955990135669708 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3510846793651581 - training/rollout_probs_diff_mean:0.004470017272979021 - training/rollout_probs_diff_std:0.013084249570965767 - training/rollout_actor_probs_pearson_corr:0.9979696869850159 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.47265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:0.999790608882904 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00012665642134379596 - rollout_corr/rollout_is_max:1.8637934923171997 - rollout_corr/rollout_is_min:0.17486874759197235 - rollout_corr/rollout_is_std:0.03934907540678978 - rollout_corr/rollout_is_eff_sample_size:0.9984533901986787 - rollout_corr/rollout_is_seq_mean:0.9998258948326111 - rollout_corr/rollout_is_seq_std:0.0032514510676264763 - rollout_corr/rollout_is_seq_max:1.010748267173767 - rollout_corr/rollout_is_seq_min:0.9913337826728821 - rollout_corr/rollout_is_seq_max_deviation:0.01074826717376709 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2218255074694753) - rollout_corr/training_log_ppl:np.float64(0.197417336050421) - rollout_corr/kl:np.float64(0.0015267702667483007) - rollout_corr/k3_kl:np.float64(0.001369252413724098) - rollout_corr/rollout_ppl:np.float64(1.219967634882778) - rollout_corr/rollout_log_ppl:np.float64(0.195890564340516) - rollout_corr/log_ppl_diff:np.float64(0.001526771709905006) - rollout_corr/log_ppl_abs_diff:np.float64(0.003275623734225519) - rollout_corr/log_ppl_diff_max:np.float64(0.022372066974639893) - rollout_corr/log_ppl_diff_min:np.float64(-0.009259283542633057) - rollout_corr/ppl_ratio:np.float64(1.0015366142615676) - rollout_corr/chi2_token:np.float64(0.0024506186600774527) - rollout_corr/chi2_seq:np.float64(0.4685141814406961) - actor/pg_loss:np.float64(0.0045470668701455) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0022142869106573926) - actor/ppo_kl:np.float64(0.00046563480048433803) - actor/pg_clipfrac_lower:np.float64(1.6075102394097485e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.47265625) - self_distillation/token_reweight_w_mean:np.float64(405614.0228516003) - self_distillation/token_reweight_w_std:np.float64(4572729.550735184) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009943647310138) - self_distillation/token_reweight_w_clip_frac:np.float64(0.087815267717815) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.6629709005355835) - perf/mfu/actor:np.float64(0.030678227294980077) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.72424697875977) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.37125) - val-aux/sciknoweval/reward/std@16:np.float64(0.2511125750137152) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.48288000000000003) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.21298088413286884) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.26238) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.21440655701062628) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.37195999999999996) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.25064049383261017) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.57182) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1477984422603609) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.17532) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.14787677257029436) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.37837999999999994) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.22110761151134656) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.62862) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07917449340268014) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.12079999999999998) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0794394529731964) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.38301999999999997) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.17816152607171362) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.65348) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.029104144112121873) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.09342) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03397205873230382) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.38182000000000005) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.13291962364918583) - val-aux/sciknoweval/score/mean@16:np.float64(0.37125) - val-aux/sciknoweval/score/std@16:np.float64(0.2511125750137152) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.48288000000000003) - val-aux/sciknoweval/score/best@2/std:np.float64(0.21298088413286884) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.26238) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.21440655701062628) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.37195999999999996) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.25064049383261017) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.57182) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1477984422603609) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.17532) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.14787677257029436) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.37837999999999994) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.22110761151134656) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.62862) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07917449340268014) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.12079999999999998) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0794394529731964) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.38301999999999997) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.17816152607171362) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.65348) - val-aux/sciknoweval/score/best@16/std:np.float64(0.029104144112121873) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.09342) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03397205873230382) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.38182000000000005) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.13291962364918583) - val-core/sciknoweval/acc/mean@16:np.float64(0.37125) - val-aux/sciknoweval/acc/std@16:np.float64(0.2511125750137152) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.48288000000000003) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.21298088413286884) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.26238) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.21440655701062628) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.37195999999999996) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.25064049383261017) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.57182) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1477984422603609) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.17532) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.14787677257029436) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.37837999999999994) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.22110761151134656) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.62862) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07917449340268014) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.12079999999999998) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0794394529731964) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.38301999999999997) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.17816152607171362) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.65348) - val-core/sciknoweval/acc/best@16/std:np.float64(0.029104144112121873) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.09342) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03397205873230382) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.38182000000000005) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.13291962364918583) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.014523687548277814) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99982) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.003070210307423338) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9933799999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.018787298197088173) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.99684) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.013382865284667094) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.98726) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.02452256547346986) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99914) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.007094879776413394) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9765) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.029286838934616793) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9997800000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.003531871795362199) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9619200000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.028882305431774648) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999800000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0006321392251711644) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:2 - critic/score/mean:0.47265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.47265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008571078069508076 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008571078069508076 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:246.73046875 - response_length/max:798.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:246.73046875 - response_length_non_aborted/max:798.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:316.1875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.31955873966217e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1871335823088884) - timing_s/agent_loop/generate_sequences/max:np.float64(5.571412174031138) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.4915980326622957) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.571412174031138) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:764 - timing_s/gen:11.46059394814074 - timing_s/reward:0.052289627492427826 - timing_s/old_log_prob:2.3004838787019253 - timing_s/adv:0.5110577456653118 - timing_s/update_actor:16.656289303675294 - timing_s/step:31.060887914150953 - timing_s/testing:68.4701924789697 - timing_s/save_checkpoint:6.864336807280779 - timing_s/stop_profile:0.00012974068522453308 - timing_per_token_ms/gen:0.18144473739595554 - timing_per_token_ms/update_actor:0.11558279128477655 - timing_per_token_ms/adv:0.0035463769675679307 - perf/total_num_tokens:144107 - perf/time_per_step:31.060887914150953 - perf/throughput:579.9375423454437 (TaskRunner pid=2122883) Training Progress: 40%|████ | 40/100 [35:09<1:01:10, 61.18s/it] (TaskRunner pid=2122883) step:41 - global_seqlen/min:11576 - global_seqlen/max:23533 - global_seqlen/minmax_diff:11957 - global_seqlen/balanced_min:20248 - global_seqlen/balanced_max:20537 - global_seqlen/mean:20291.25 - actor/entropy:0.18663230538368225 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6220853924751282 - training/rollout_probs_diff_mean:0.004448506515473127 - training/rollout_probs_diff_std:0.01353462878614664 - training/rollout_actor_probs_pearson_corr:0.99795001745224 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.54296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0001215934753418 - rollout_corr/rollout_is_ratio_fraction_high:3.953923442168161e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011861771054100245 - rollout_corr/rollout_is_max:4.517718315124512 - rollout_corr/rollout_is_min:0.06017683818936348 - rollout_corr/rollout_is_std:0.03926188126206398 - rollout_corr/rollout_is_eff_sample_size:0.9984612337553515 - rollout_corr/rollout_is_seq_mean:1.0000065565109253 - rollout_corr/rollout_is_seq_std:0.003107651136815548 - rollout_corr/rollout_is_seq_max:1.0133030414581299 - rollout_corr/rollout_is_seq_min:0.9923636317253113 - rollout_corr/rollout_is_seq_max_deviation:0.013303041458129883 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2377123944461346) - rollout_corr/training_log_ppl:np.float64(0.20936558903486002) - rollout_corr/kl:np.float64(0.001277374319045066) - rollout_corr/k3_kl:np.float64(0.0011814869795614413) - rollout_corr/rollout_ppl:np.float64(1.2360735745169222) - rollout_corr/rollout_log_ppl:np.float64(0.20808821474201977) - rollout_corr/log_ppl_diff:np.float64(0.0012773742928402498) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028327976033324376) - rollout_corr/log_ppl_diff_max:np.float64(0.01583242416381836) - rollout_corr/log_ppl_diff_min:np.float64(-0.007822096347808838) - rollout_corr/ppl_ratio:np.float64(1.0012840463314205) - rollout_corr/chi2_token:np.float64(0.0022241747938096523) - rollout_corr/chi2_seq:np.float64(0.5387362614274025) - actor/pg_loss:np.float64(0.005529339890927076) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001543819114431244) - actor/ppo_kl:np.float64(0.0003531992533591932) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.54296875) - self_distillation/token_reweight_w_mean:np.float64(397424.65681143315) - self_distillation/token_reweight_w_std:np.float64(4439488.2469981825) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0017797024920583) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11050213062844705) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.6619970500469208) - perf/mfu/actor:np.float64(0.03406975692051313) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.76568222045898) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:2 - critic/score/mean:0.54296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.54296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0033542450983077288 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0033542450983077288 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:296.3828125 - response_length/max:1648.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:296.3828125 - response_length_non_aborted/max:1648.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:337.71875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.941519677639008e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1566278170794249) - timing_s/agent_loop/generate_sequences/max:np.float64(9.40533797070384) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9047601231723092) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.40533797070384) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:1648 - timing_s/gen:15.249274028465152 - timing_s/reward:0.04902481287717819 - timing_s/old_log_prob:2.395838249474764 - timing_s/adv:0.6379478648304939 - timing_s/update_actor:16.939913291484118 - timing_s/step:35.36231600493193 - timing_s/stop_profile:3.432855010032654e-05 - timing_per_token_ms/gen:0.2009815487316492 - timing_per_token_ms/update_actor:0.10435479142169726 - timing_per_token_ms/adv:0.003929944340728725 - perf/total_num_tokens:162330 - perf/time_per_step:35.36231600493193 - perf/throughput:573.809984537495 (TaskRunner pid=2122883) Training Progress: 41%|████ | 41/100 [35:44<52:33, 53.44s/it] (TaskRunner pid=2122883) step:42 - global_seqlen/min:10264 - global_seqlen/max:20936 - global_seqlen/minmax_diff:10672 - global_seqlen/balanced_min:15308 - global_seqlen/balanced_max:15324 - global_seqlen/mean:15318.5 - actor/entropy:0.17958307266235352 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3282223343849182 - training/rollout_probs_diff_mean:0.004483028315007687 - training/rollout_probs_diff_std:0.013422226533293724 - training/rollout_actor_probs_pearson_corr:0.997865617275238 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.6171875 - self_distillation/correct_sample_fraction:0.3125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6171875 - rollout_corr/rollout_is_mean:0.9998947978019714 - rollout_corr/rollout_is_ratio_fraction_high:1.716207952995319e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010297247354174033 - rollout_corr/rollout_is_max:2.239044189453125 - rollout_corr/rollout_is_min:0.3549353778362274 - rollout_corr/rollout_is_std:0.039695192128419876 - rollout_corr/rollout_is_eff_sample_size:0.9984264140581771 - rollout_corr/rollout_is_seq_mean:0.9999154806137085 - rollout_corr/rollout_is_seq_std:0.0030749160796403885 - rollout_corr/rollout_is_seq_max:1.0121992826461792 - rollout_corr/rollout_is_seq_min:0.9852494597434998 - rollout_corr/rollout_is_seq_max_deviation:0.014750540256500244 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2074014260433614) - rollout_corr/training_log_ppl:np.float64(0.18636732350569218) - rollout_corr/kl:np.float64(0.0009534069710754878) - rollout_corr/k3_kl:np.float64(0.001279137213884951) - rollout_corr/rollout_ppl:np.float64(1.2062194678001106) - rollout_corr/rollout_log_ppl:np.float64(0.1854139148199465) - rollout_corr/log_ppl_diff:np.float64(0.0009534086857456714) - rollout_corr/log_ppl_abs_diff:np.float64(0.002904396125813946) - rollout_corr/log_ppl_diff_max:np.float64(0.01598384976387024) - rollout_corr/log_ppl_diff_min:np.float64(-0.010572642087936401) - rollout_corr/ppl_ratio:np.float64(1.0009608941618353) - rollout_corr/chi2_token:np.float64(0.0036230282858014107) - rollout_corr/chi2_seq:np.float64(0.45362843945622444) - actor/pg_loss:np.float64(0.0053381690522655845) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014900016758474521) - actor/ppo_kl:np.float64(-2.4106089924202934e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6171875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3125) - self_distillation/token_reweight_w_mean:np.float64(811433.6940884823) - self_distillation/token_reweight_w_std:np.float64(8261526.878942402) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001308246748522) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10496872605290264) - self_distillation/empty_target_batch:np.float64(0.3828125) - actor/grad_norm:np.float64(0.6776371896266937) - perf/mfu/actor:np.float64(0.026046903222768404) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.86044692993164) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:2 - critic/score/mean:0.3125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002994782757014036 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002994782757014036 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:227.609375 - response_length/max:957.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:227.609375 - response_length_non_aborted/max:957.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.09375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.9033049941062927e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.001108106225729) - timing_s/agent_loop/generate_sequences/max:np.float64(6.198202773928642) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.2058102839364437) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.198202773928642) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:957 - timing_s/gen:12.159217644482851 - timing_s/reward:0.05021318793296814 - timing_s/old_log_prob:2.2670325599610806 - timing_s/adv:0.49279571883380413 - timing_s/update_actor:16.662710631266236 - timing_s/step:31.7205575350672 - timing_s/stop_profile:0.00012237019836902618 - timing_per_token_ms/gen:0.20867744979204453 - timing_per_token_ms/update_actor:0.1359688500119646 - timing_per_token_ms/adv:0.004021246522454908 - perf/total_num_tokens:122548 - perf/time_per_step:31.7205575350672 - perf/throughput:482.92026339906977 (TaskRunner pid=2122883) Training Progress: 42%|████▏ | 42/100 [36:16<45:22, 46.94s/it] (TaskRunner pid=2122883) step:43 - global_seqlen/min:12638 - global_seqlen/max:28379 - global_seqlen/minmax_diff:15741 - global_seqlen/balanced_min:19084 - global_seqlen/balanced_max:19115 - global_seqlen/mean:19106.75 - actor/entropy:0.17867739498615265 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4741092324256897 - training/rollout_probs_diff_mean:0.004286498762667179 - training/rollout_probs_diff_std:0.01305386796593666 - training/rollout_actor_probs_pearson_corr:0.9979863166809082 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.796875 - self_distillation/correct_sample_fraction:0.4921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.796875 - rollout_corr/rollout_is_mean:1.0000731945037842 - rollout_corr/rollout_is_ratio_fraction_high:2.8147605917183682e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012666422117035836 - rollout_corr/rollout_is_max:4.272411823272705 - rollout_corr/rollout_is_min:0.2297939509153366 - rollout_corr/rollout_is_std:0.03817035257816315 - rollout_corr/rollout_is_eff_sample_size:0.99854502489099 - rollout_corr/rollout_is_seq_mean:1.0000548362731934 - rollout_corr/rollout_is_seq_std:0.0034973809961229563 - rollout_corr/rollout_is_seq_max:1.0184221267700195 - rollout_corr/rollout_is_seq_min:0.98951256275177 - rollout_corr/rollout_is_seq_max_deviation:0.01842212677001953 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2184172831475735) - rollout_corr/training_log_ppl:np.float64(0.1945186269149417) - rollout_corr/kl:np.float64(0.0017048975041937808) - rollout_corr/k3_kl:np.float64(0.002482297865981309) - rollout_corr/rollout_ppl:np.float64(1.2163156094029546) - rollout_corr/rollout_log_ppl:np.float64(0.19281372765544802) - rollout_corr/log_ppl_diff:np.float64(0.0017048992594936863) - rollout_corr/log_ppl_abs_diff:np.float64(0.0035877060872735456) - rollout_corr/log_ppl_diff_max:np.float64(0.02438490092754364) - rollout_corr/log_ppl_diff_min:np.float64(-0.022404611110687256) - rollout_corr/ppl_ratio:np.float64(1.0017193660605699) - rollout_corr/chi2_token:np.float64(0.0598582336679101) - rollout_corr/chi2_seq:np.float64(42.330826848512515) - actor/pg_loss:np.float64(0.0056234917137771845) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002230035815955489) - actor/ppo_kl:np.float64(0.0008539994699017228) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4921875) - self_distillation/token_reweight_w_mean:np.float64(725459.6291632976) - self_distillation/token_reweight_w_std:np.float64(7201750.69503868) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014563573058695) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11069456930272281) - self_distillation/empty_target_batch:np.float64(0.203125) - actor/grad_norm:np.float64(0.7029270678758621) - perf/mfu/actor:np.float64(0.03253319803382335) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.79828262329102) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:3 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0038878880441188812 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0038878880441188812 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:277.5546875 - response_length/max:1388.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:277.5546875 - response_length_non_aborted/max:1388.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:319.53125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00031706131994724274 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8820750936865807) - timing_s/agent_loop/generate_sequences/max:np.float64(8.526029285043478) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.7013816293983837) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.526029285043478) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:1388 - timing_s/gen:14.14283044822514 - timing_s/reward:0.05622094310820103 - timing_s/old_log_prob:2.6203188207000494 - timing_s/adv:0.5772702526301146 - timing_s/update_actor:16.599831921979785 - timing_s/step:34.095423594117165 - timing_s/stop_profile:0.00011685304343700409 - timing_per_token_ms/gen:0.19904340991675543 - timing_per_token_ms/update_actor:0.1085992641473549 - timing_per_token_ms/adv:0.003776612012967371 - perf/total_num_tokens:152854 - perf/time_per_step:34.095423594117165 - perf/throughput:560.3904567209039 (TaskRunner pid=2122883) Training Progress: 43%|████▎ | 43/100 [36:50<41:04, 43.23s/it] (TaskRunner pid=2122883) step:44 - global_seqlen/min:13160 - global_seqlen/max:23356 - global_seqlen/minmax_diff:10196 - global_seqlen/balanced_min:19260 - global_seqlen/balanced_max:19276 - global_seqlen/mean:19267.375 - actor/entropy:0.1776508092880249 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44105371832847595 - training/rollout_probs_diff_mean:0.004229406826198101 - training/rollout_probs_diff_std:0.013126127421855927 - training/rollout_actor_probs_pearson_corr:0.9979387521743774 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.52734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:0.9998519420623779 - rollout_corr/rollout_is_ratio_fraction_high:4.204801734886132e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015417607210110873 - rollout_corr/rollout_is_max:2.684969186782837 - rollout_corr/rollout_is_min:0.17924728989601135 - rollout_corr/rollout_is_std:0.038983073085546494 - rollout_corr/rollout_is_eff_sample_size:0.9984822693566601 - rollout_corr/rollout_is_seq_mean:0.9997416734695435 - rollout_corr/rollout_is_seq_std:0.003170690732076764 - rollout_corr/rollout_is_seq_max:1.01134192943573 - rollout_corr/rollout_is_seq_min:0.9876351952552795 - rollout_corr/rollout_is_seq_max_deviation:0.012364804744720459 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2239833851344883) - rollout_corr/training_log_ppl:np.float64(0.19885029891156591) - rollout_corr/kl:np.float64(0.0017507878198053106) - rollout_corr/k3_kl:np.float64(0.0012392329912813693) - rollout_corr/rollout_ppl:np.float64(1.2217664504423738) - rollout_corr/rollout_log_ppl:np.float64(0.19709950988180935) - rollout_corr/log_ppl_diff:np.float64(0.001750789029756561) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031125989626161754) - rollout_corr/log_ppl_diff_max:np.float64(0.02598017454147339) - rollout_corr/log_ppl_diff_min:np.float64(-0.010924175381660461) - rollout_corr/ppl_ratio:np.float64(1.0017614769749343) - rollout_corr/chi2_token:np.float64(0.0012613397557288408) - rollout_corr/chi2_seq:np.float64(0.3330452733207494) - actor/pg_loss:np.float64(0.00551921222358942) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012254381422280858) - actor/ppo_kl:np.float64(0.0004983369169835328) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.52734375) - self_distillation/token_reweight_w_mean:np.float64(371231.27284245985) - self_distillation/token_reweight_w_std:np.float64(4606640.552745781) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003671881277114) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09728244636789896) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.6762457638978958) - perf/mfu/actor:np.float64(0.032491280735104155) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.8349723815918) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:3 - critic/score/mean:0.52734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.52734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001862376811914146 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001862376811914146 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:278.69921875 - response_length/max:1369.0 - response_length/min:100.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:278.69921875 - response_length_non_aborted/max:1369.0 - response_length_non_aborted/min:100.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:323.40625 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016621500253677368 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2336528431624174) - timing_s/agent_loop/generate_sequences/max:np.float64(8.673735836520791) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.68211255150527) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.673735836520791) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:1369 - timing_s/gen:14.567327432334423 - timing_s/reward:0.05603113956749439 - timing_s/old_log_prob:2.3046666514128447 - timing_s/adv:0.5825918577611446 - timing_s/update_actor:16.785698210820556 - timing_s/step:34.38467370159924 - timing_s/stop_profile:5.710311233997345e-05 - timing_per_token_ms/gen:0.20417575276233652 - timing_per_token_ms/update_actor:0.10889974770058555 - timing_per_token_ms/adv:0.003779652506900555 - perf/total_num_tokens:154139 - perf/time_per_step:34.38467370159924 - perf/throughput:560.3477632857068 (TaskRunner pid=2122883) Training Progress: 44%|████▍ | 44/100 [37:25<37:52, 40.59s/it] (TaskRunner pid=2122883) step:45 - global_seqlen/min:10370 - global_seqlen/max:27688 - global_seqlen/minmax_diff:17318 - global_seqlen/balanced_min:19743 - global_seqlen/balanced_max:20142 - global_seqlen/mean:19803.0 - actor/entropy:0.18040679395198822 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5990896821022034 - training/rollout_probs_diff_mean:0.004071939270943403 - training/rollout_probs_diff_std:0.013194762170314789 - training/rollout_actor_probs_pearson_corr:0.998019278049469 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.4140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:0.9999013543128967 - rollout_corr/rollout_is_ratio_fraction_high:3.890041625709273e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.780083251418546e-05 - rollout_corr/rollout_is_max:2.47705340385437 - rollout_corr/rollout_is_min:0.16637524962425232 - rollout_corr/rollout_is_std:0.037288833409547806 - rollout_corr/rollout_is_eff_sample_size:0.9986111168979587 - rollout_corr/rollout_is_seq_mean:0.9998700618743896 - rollout_corr/rollout_is_seq_std:0.003353060455992818 - rollout_corr/rollout_is_seq_max:1.012258768081665 - rollout_corr/rollout_is_seq_min:0.991085410118103 - rollout_corr/rollout_is_seq_max_deviation:0.012258768081665039 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2377176005393267) - rollout_corr/training_log_ppl:np.float64(0.20978421579638962) - rollout_corr/kl:np.float64(0.0012163608913304813) - rollout_corr/k3_kl:np.float64(0.0014924378000671368) - rollout_corr/rollout_ppl:np.float64(1.2362172487191856) - rollout_corr/rollout_log_ppl:np.float64(0.2085678537550848) - rollout_corr/log_ppl_diff:np.float64(0.0012163620413048193) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032108214363688603) - rollout_corr/log_ppl_diff_max:np.float64(0.03155073523521423) - rollout_corr/log_ppl_diff_min:np.float64(-0.008211210370063782) - rollout_corr/ppl_ratio:np.float64(1.001229275483638) - rollout_corr/chi2_token:np.float64(0.003470740979537368) - rollout_corr/chi2_seq:np.float64(1.005021903431043) - actor/pg_loss:np.float64(0.005921048694290221) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013443831210224744) - actor/ppo_kl:np.float64(0.00014919952121861257) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4140625) - self_distillation/token_reweight_w_mean:np.float64(383251.54019788885) - self_distillation/token_reweight_w_std:np.float64(4420001.027808327) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0017411878798157) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10430864432419185) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.6809583902359009) - perf/mfu/actor:np.float64(0.03341682138075278) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.90021133422852) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:3 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01745494082570076 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01745494082570076 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:301.25 - response_length/max:1940.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:301.25 - response_length_non_aborted/max:1940.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:317.59375 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.563612699508667e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.061364471912384) - timing_s/agent_loop/generate_sequences/max:np.float64(11.541284376755357) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.907274785611662) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.541284376755357) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:1940 - timing_s/gen:18.06968740746379 - timing_s/reward:0.05686299502849579 - timing_s/old_log_prob:2.346899025142193 - timing_s/adv:0.6176035478711128 - timing_s/update_actor:16.911936789751053 - timing_s/step:38.08804814517498 - timing_s/stop_profile:0.00011461786925792694 - timing_per_token_ms/gen:0.23430611264864873 - timing_per_token_ms/update_actor:0.10675110330348339 - timing_per_token_ms/adv:0.0038984216272225976 - perf/total_num_tokens:158424 - perf/time_per_step:38.08804814517498 - perf/throughput:519.926879017786 (TaskRunner pid=2122883) Training Progress: 45%|████▌ | 45/100 [38:03<36:31, 39.85s/it] (TaskRunner pid=2122883) step:46 - global_seqlen/min:11122 - global_seqlen/max:29892 - global_seqlen/minmax_diff:18770 - global_seqlen/balanced_min:16452 - global_seqlen/balanced_max:16787 - global_seqlen/mean:16539.25 - actor/entropy:0.18795309960842133 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6190823316574097 - training/rollout_probs_diff_mean:0.004448164254426956 - training/rollout_probs_diff_std:0.013683785684406757 - training/rollout_actor_probs_pearson_corr:0.9979138970375061 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.5390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.999754011631012 - rollout_corr/rollout_is_ratio_fraction_high:7.895152521086857e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001263224403373897 - rollout_corr/rollout_is_max:2.5501413345336914 - rollout_corr/rollout_is_min:0.03999684005975723 - rollout_corr/rollout_is_std:0.039844319224357605 - rollout_corr/rollout_is_eff_sample_size:0.9984140554322439 - rollout_corr/rollout_is_seq_mean:0.9995894432067871 - rollout_corr/rollout_is_seq_std:0.0033229426480829716 - rollout_corr/rollout_is_seq_max:1.0124613046646118 - rollout_corr/rollout_is_seq_min:0.990471601486206 - rollout_corr/rollout_is_seq_max_deviation:0.012461304664611816 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2413132754154503) - rollout_corr/training_log_ppl:np.float64(0.21247280396346468) - rollout_corr/kl:np.float64(0.002923060385167986) - rollout_corr/k3_kl:np.float64(0.0021046519883611836) - rollout_corr/rollout_ppl:np.float64(1.2376095172949135) - rollout_corr/rollout_log_ppl:np.float64(0.20954974222695455) - rollout_corr/log_ppl_diff:np.float64(0.0029230617365101352) - rollout_corr/log_ppl_abs_diff:np.float64(0.004334593817475252) - rollout_corr/log_ppl_diff_max:np.float64(0.025646045804023743) - rollout_corr/log_ppl_diff_min:np.float64(-0.013347774744033813) - rollout_corr/ppl_ratio:np.float64(1.0029449705034494) - rollout_corr/chi2_token:np.float64(0.0022582970559597015) - rollout_corr/chi2_seq:np.float64(0.4936607242561877) - actor/pg_loss:np.float64(0.0058343312703073025) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0026905024278676137) - actor/ppo_kl:np.float64(0.001411904266461761) - actor/pg_clipfrac_lower:np.float64(1.5625000742147677e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5390625) - self_distillation/token_reweight_w_mean:np.float64(1044202.6667105879) - self_distillation/token_reweight_w_std:np.float64(8739516.905448329) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014959073159844) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10200972099846695) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.7633071690797806) - perf/mfu/actor:np.float64(0.027947938564247113) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.91358947753906) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:3 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.026450734585523605 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.026450734585523605 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:247.3828125 - response_length/max:1912.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:247.3828125 - response_length_non_aborted/max:1912.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.46875 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012016668915748596 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0365642439574003) - timing_s/agent_loop/generate_sequences/max:np.float64(10.925699835643172) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3518986795606907) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.925699835643172) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:1912 - timing_s/gen:16.643203927204013 - timing_s/reward:0.05389646999537945 - timing_s/old_log_prob:2.2957371938973665 - timing_s/adv:0.5990908946841955 - timing_s/update_actor:16.899991875514388 - timing_s/step:36.581113524734974 - timing_s/stop_profile:0.0001400299370288849 - timing_per_token_ms/gen:0.2628012620749094 - timing_per_token_ms/update_actor:0.12772640745132327 - timing_per_token_ms/adv:0.004527796716025481 - perf/total_num_tokens:132314 - perf/time_per_step:36.581113524734974 - perf/throughput:452.12538401316544 (TaskRunner pid=2122883) Training Progress: 46%|████▌ | 46/100 [38:40<34:59, 38.88s/it] (TaskRunner pid=2122883) step:47 - global_seqlen/min:19683 - global_seqlen/max:34953 - global_seqlen/minmax_diff:15270 - global_seqlen/balanced_min:24795 - global_seqlen/balanced_max:24818 - global_seqlen/mean:24812.5 - actor/entropy:0.14859639108181 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5783280730247498 - training/rollout_probs_diff_mean:0.003343893215060234 - training/rollout_probs_diff_std:0.012050571851432323 - training/rollout_actor_probs_pearson_corr:0.9980299472808838 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.000154733657837 - rollout_corr/rollout_is_ratio_fraction_high:4.733234891318716e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013253057841211557 - rollout_corr/rollout_is_max:6.984804630279541 - rollout_corr/rollout_is_min:0.15933217108249664 - rollout_corr/rollout_is_std:0.0349300317466259 - rollout_corr/rollout_is_eff_sample_size:0.9987817365255202 - rollout_corr/rollout_is_seq_mean:1.0001826286315918 - rollout_corr/rollout_is_seq_std:0.003950812388211489 - rollout_corr/rollout_is_seq_max:1.04475998878479 - rollout_corr/rollout_is_seq_min:0.9881705641746521 - rollout_corr/rollout_is_seq_max_deviation:0.04475998878479004 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2160145845264196) - rollout_corr/training_log_ppl:np.float64(0.19086833937035408) - rollout_corr/kl:np.float64(0.015507104080729306) - rollout_corr/k3_kl:np.float64(0.014919463526666732) - rollout_corr/rollout_ppl:np.float64(1.196146132890135) - rollout_corr/rollout_log_ppl:np.float64(0.1753612344764406) - rollout_corr/log_ppl_diff:np.float64(0.015507104864809662) - rollout_corr/log_ppl_abs_diff:np.float64(0.016464031563373283) - rollout_corr/log_ppl_diff_max:np.float64(0.2085978388786316) - rollout_corr/log_ppl_diff_min:np.float64(-0.013799071311950684) - rollout_corr/ppl_ratio:np.float64(1.0166863345075399) - rollout_corr/chi2_token:np.float64(0.002912049414590001) - rollout_corr/chi2_seq:np.float64(0.4900429293047637) - actor/pg_loss:np.float64(0.006711778463795781) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001963024042424877) - actor/ppo_kl:np.float64(0.013789263467803181) - actor/pg_clipfrac_lower:np.float64(1.4103383364272304e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.484375) - self_distillation/token_reweight_w_mean:np.float64(396123.6195182537) - self_distillation/token_reweight_w_std:np.float64(4634816.637059211) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006289314478636) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1108224172348855) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.6453308165073395) - perf/mfu/actor:np.float64(0.04141419655296415) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.81275177001953) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:3 - critic/score/mean:0.484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006173321511596441 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006173321511596441 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:412.640625 - response_length/max:1648.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:412.640625 - response_length_non_aborted/max:1648.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:362.75 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.221956133842468e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0964402798563242) - timing_s/agent_loop/generate_sequences/max:np.float64(11.21230830438435) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.87971582760656) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.21230830438435) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:416 - timing_s/agent_loop/slowest/response_length:1648 - timing_s/gen:16.986663652583957 - timing_s/reward:0.06316123157739639 - timing_s/old_log_prob:2.3301994521170855 - timing_s/adv:0.6273217760026455 - timing_s/update_actor:17.14669720456004 - timing_s/step:37.24170481786132 - timing_s/stop_profile:0.00011117756366729736 - timing_per_token_ms/gen:0.1608037378600473 - timing_per_token_ms/update_actor:0.08638134611869039 - timing_per_token_ms/adv:0.0031603112141191207 - perf/total_num_tokens:198500 - perf/time_per_step:37.24170481786132 - perf/throughput:666.2557506792706 (TaskRunner pid=2122883) Training Progress: 47%|████▋ | 47/100 [39:17<33:55, 38.41s/it] (TaskRunner pid=2122883) step:48 - global_seqlen/min:15753 - global_seqlen/max:27003 - global_seqlen/minmax_diff:11250 - global_seqlen/balanced_min:20074 - global_seqlen/balanced_max:20085 - global_seqlen/mean:20081.125 - actor/entropy:0.18119651079177856 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5287033319473267 - training/rollout_probs_diff_mean:0.0041629173792898655 - training/rollout_probs_diff_std:0.013067073188722134 - training/rollout_actor_probs_pearson_corr:0.9980469942092896 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.46484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9998913407325745 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0001484836102463305 - rollout_corr/rollout_is_max:1.8580189943313599 - rollout_corr/rollout_is_min:0.12150196731090546 - rollout_corr/rollout_is_std:0.03766418248414993 - rollout_corr/rollout_is_eff_sample_size:0.9985830623875143 - rollout_corr/rollout_is_seq_mean:0.999824047088623 - rollout_corr/rollout_is_seq_std:0.0027110264636576176 - rollout_corr/rollout_is_seq_max:1.0075005292892456 - rollout_corr/rollout_is_seq_min:0.9895845055580139 - rollout_corr/rollout_is_seq_max_deviation:0.010415494441986084 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2318859449587762) - rollout_corr/training_log_ppl:np.float64(0.20496501913294196) - rollout_corr/kl:np.float64(0.0021551217895456887) - rollout_corr/k3_kl:np.float64(0.0023452171697613267) - rollout_corr/rollout_ppl:np.float64(1.2291390299797058) - rollout_corr/rollout_log_ppl:np.float64(0.20280989579623565) - rollout_corr/log_ppl_diff:np.float64(0.0021551233367063105) - rollout_corr/log_ppl_abs_diff:np.float64(0.0036582308530341834) - rollout_corr/log_ppl_diff_max:np.float64(0.03887230157852173) - rollout_corr/log_ppl_diff_min:np.float64(-0.013254612684249878) - rollout_corr/ppl_ratio:np.float64(1.0021720596123487) - rollout_corr/chi2_token:np.float64(0.0768868934828788) - rollout_corr/chi2_seq:np.float64(9.817465823842213) - actor/pg_loss:np.float64(0.007189736585132778) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001849453532258849) - actor/ppo_kl:np.float64(0.001085506058089436) - actor/pg_clipfrac_lower:np.float64(3.0127496302156942e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.46484375) - self_distillation/token_reweight_w_mean:np.float64(593021.7767671708) - self_distillation/token_reweight_w_std:np.float64(6946430.695254925) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.002022563945502) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10840969366836362) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.7073291689157486) - perf/mfu/actor:np.float64(0.03384095922161414) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.00584030151367) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:3 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0143998172134161 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0143998172134161 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:315.69140625 - response_length/max:1345.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:315.69140625 - response_length_non_aborted/max:1345.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:311.84375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.757490336894989e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0721273869276047) - timing_s/agent_loop/generate_sequences/max:np.float64(8.888256212696433) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9832990684881224) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.888256212696433) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:436 - timing_s/agent_loop/slowest/response_length:1345 - timing_s/gen:14.835985887795687 - timing_s/reward:0.056042348966002464 - timing_s/old_log_prob:2.3734769448637962 - timing_s/adv:0.5705869812518358 - timing_s/update_actor:16.890875345095992 - timing_s/step:34.81418928876519 - timing_s/stop_profile:0.00010911934077739716 - timing_per_token_ms/gen:0.18357506326386389 - timing_per_token_ms/update_actor:0.10514149073505588 - timing_per_token_ms/adv:0.003551761799026672 - perf/total_num_tokens:160649 - perf/time_per_step:34.81418928876519 - perf/throughput:576.8086349343868 (TaskRunner pid=2122883) Training Progress: 48%|████▊ | 48/100 [39:52<32:21, 37.34s/it] (TaskRunner pid=2122883) step:49 - global_seqlen/min:12849 - global_seqlen/max:30904 - global_seqlen/minmax_diff:18055 - global_seqlen/balanced_min:19385 - global_seqlen/balanced_max:25931 - global_seqlen/mean:20211.375 - actor/entropy:0.15795230865478516 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.399758517742157 - training/rollout_probs_diff_mean:0.0037395432591438293 - training/rollout_probs_diff_std:0.012383424676954746 - training/rollout_actor_probs_pearson_corr:0.9980095028877258 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.43359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:0.9998835921287537 - rollout_corr/rollout_is_ratio_fraction_high:2.3283158952835947e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.149105269694701e-05 - rollout_corr/rollout_is_max:3.340711832046509 - rollout_corr/rollout_is_min:0.10319963097572327 - rollout_corr/rollout_is_std:0.03628097102046013 - rollout_corr/rollout_is_eff_sample_size:0.9986851836242739 - rollout_corr/rollout_is_seq_mean:0.9996575713157654 - rollout_corr/rollout_is_seq_std:0.002907124115154147 - rollout_corr/rollout_is_seq_max:1.0098154544830322 - rollout_corr/rollout_is_seq_min:0.9917104840278625 - rollout_corr/rollout_is_seq_max_deviation:0.009815454483032227 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2227676422335207) - rollout_corr/training_log_ppl:np.float64(0.19771692524227547) - rollout_corr/kl:np.float64(0.0017005499959275028) - rollout_corr/k3_kl:np.float64(0.001488717432039266) - rollout_corr/rollout_ppl:np.float64(1.2206510826945305) - rollout_corr/rollout_log_ppl:np.float64(0.19601637365849456) - rollout_corr/log_ppl_diff:np.float64(0.001700551583780907) - rollout_corr/log_ppl_abs_diff:np.float64(0.003168961076880805) - rollout_corr/log_ppl_diff_max:np.float64(0.017631560564041138) - rollout_corr/log_ppl_diff_min:np.float64(-0.011207103729248047) - rollout_corr/ppl_ratio:np.float64(1.0017099685501307) - rollout_corr/chi2_token:np.float64(0.002383438404649496) - rollout_corr/chi2_seq:np.float64(0.4581478724721819) - actor/pg_loss:np.float64(0.006989955902099609) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020739615124512056) - actor/ppo_kl:np.float64(0.0004834777322839656) - actor/pg_clipfrac_lower:np.float64(4.919710136164213e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.43359375) - self_distillation/token_reweight_w_mean:np.float64(483083.6963332107) - self_distillation/token_reweight_w_std:np.float64(5497377.112956069) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001461925683543) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10720286803552881) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.6698927134275436) - perf/mfu/actor:np.float64(0.03237159299727145) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.85754013061523) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:3 - critic/score/mean:0.43359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.43359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017634082585573196 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017634082585573196 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:335.54296875 - response_length/max:8192.0 - response_length/min:101.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:335.54296875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:296.0625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.936040103435516e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2021586298942566) - timing_s/agent_loop/generate_sequences/max:np.float64(48.67229326814413) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.981824685790343) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(48.67229326814413) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:54.559863248839974 - timing_s/reward:0.05820300057530403 - timing_s/old_log_prob:2.477139912545681 - timing_s/adv:0.5340056866407394 - timing_s/update_actor:18.079693160951138 - timing_s/step:75.79790463671088 - timing_s/stop_profile:0.00011322461068630219 - timing_per_token_ms/gen:0.6351629617206251 - timing_per_token_ms/update_actor:0.11181632348709042 - timing_per_token_ms/adv:0.003302630861586232 - perf/total_num_tokens:161691 - perf/time_per_step:75.79790463671088 - perf/throughput:266.6482021748542 (TaskRunner pid=2122883) Training Progress: 49%|████▉ | 49/100 [41:08<41:33, 48.89s/it] (TaskRunner pid=2122883) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2122883) validation generation end (TaskRunner pid=2122883) [prompt] system (TaskRunner pid=2122883) (TaskRunner pid=2122883) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2122883) (TaskRunner pid=2122883) A (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) user (TaskRunner pid=2122883) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2122883) (TaskRunner pid=2122883) A: -0.01 (TaskRunner pid=2122883) B: 1.69 (TaskRunner pid=2122883) C: 2.49 (TaskRunner pid=2122883) D: 0.45 (TaskRunner pid=2122883) Please reason step by step. (TaskRunner pid=2122883) assistant (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) [response] (TaskRunner pid=2122883) The folding stability score of a protein sequence is typically determined through computational methods or experimental techniques such as molecular dynamics simulations or free energy calculations. However, without access to specific computational tools or databases that can calculate this score for the given sequence, it is not possible to determine the exact value. Among the provided options, the most reasonable value for a folding stability score is 0.45, as it is a common range for such scores in many computational contexts. Therefore, the most plausible answer is 0.45. (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) D (TaskRunner pid=2122883) (TaskRunner pid=2122883) [ground_truth] B (TaskRunner pid=2122883) [score] 0.0 (TaskRunner pid=2122883) [acc] 0.0 (TaskRunner pid=2122883) [pred] D (TaskRunner pid=2122883) [incorrect_format] 1 (TaskRunner pid=2122883) [feedback] (TaskRunner pid=2122883) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (WorkerDict pid=2123259) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40/actor (TaskRunner pid=2122883) step:50 - global_seqlen/min:10411 - global_seqlen/max:26898 - global_seqlen/minmax_diff:16487 - global_seqlen/balanced_min:20939 - global_seqlen/balanced_max:20959 - global_seqlen/mean:20951.625 - actor/entropy:0.16414521634578705 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6031620502471924 - training/rollout_probs_diff_mean:0.003770680632442236 - training/rollout_probs_diff_std:0.012851644307374954 - training/rollout_actor_probs_pearson_corr:0.9979996085166931 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.56640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:1.0001113414764404 - rollout_corr/rollout_is_ratio_fraction_high:3.634557288023643e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014538229152094573 - rollout_corr/rollout_is_max:3.7972795963287354 - rollout_corr/rollout_is_min:0.11608143150806427 - rollout_corr/rollout_is_std:0.03673160448670387 - rollout_corr/rollout_is_eff_sample_size:0.9986529638426448 - rollout_corr/rollout_is_seq_mean:1.0002330541610718 - rollout_corr/rollout_is_seq_std:0.003209285903722048 - rollout_corr/rollout_is_seq_max:1.0156590938568115 - rollout_corr/rollout_is_seq_min:0.991116464138031 - rollout_corr/rollout_is_seq_max_deviation:0.015659093856811523 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.21245695091784) - rollout_corr/training_log_ppl:np.float64(0.18853073121863417) - rollout_corr/kl:np.float64(0.000940453585307921) - rollout_corr/k3_kl:np.float64(0.0011123225885398824) - rollout_corr/rollout_ppl:np.float64(1.211254812311381) - rollout_corr/rollout_log_ppl:np.float64(0.18759027698251884) - rollout_corr/log_ppl_diff:np.float64(0.000940454236115329) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025964345986722037) - rollout_corr/log_ppl_diff_max:np.float64(0.012625694274902344) - rollout_corr/log_ppl_diff_min:np.float64(-0.010028764605522156) - rollout_corr/ppl_ratio:np.float64(1.000946962274611) - rollout_corr/chi2_token:np.float64(0.003119073808193207) - rollout_corr/chi2_seq:np.float64(0.5444498066790402) - actor/pg_loss:np.float64(0.00562474865000695) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012370409526738513) - actor/ppo_kl:np.float64(0.00027735871690294164) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.56640625) - self_distillation/token_reweight_w_mean:np.float64(552116.4400627858) - self_distillation/token_reweight_w_std:np.float64(5173609.766112542) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9990502987056971) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09921138118079398) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.6405464708805084) - perf/mfu/actor:np.float64(0.03547942389442762) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.8638153076172) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.41125) - val-aux/sciknoweval/reward/std@16:np.float64(0.282579055086121) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.53366) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.22278060446414918) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.28978000000000004) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2565024603314222) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.41237999999999997) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.28174299705440914) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.61846) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1429779412119699) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.17951999999999999) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.19721838418827287) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.435) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.23734796957612123) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.66646) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07860442389870713) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.10271999999999998) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.11817685371390609) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.45018) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.18446648963991358) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.69634) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.0437133055991171) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.061799999999999994) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05167005663723248) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.46228) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.13956489639286812) - val-aux/sciknoweval/score/mean@16:np.float64(0.41125) - val-aux/sciknoweval/score/std@16:np.float64(0.282579055086121) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.53366) - val-aux/sciknoweval/score/best@2/std:np.float64(0.22278060446414918) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.28978000000000004) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.2565024603314222) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.41237999999999997) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.28174299705440914) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.61846) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1429779412119699) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.17951999999999999) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.19721838418827287) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.435) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.23734796957612123) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.66646) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07860442389870713) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.10271999999999998) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.11817685371390609) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.45018) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.18446648963991358) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.69634) - val-aux/sciknoweval/score/best@16/std:np.float64(0.0437133055991171) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.061799999999999994) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05167005663723248) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.46228) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.13956489639286812) - val-core/sciknoweval/acc/mean@16:np.float64(0.41125) - val-aux/sciknoweval/acc/std@16:np.float64(0.282579055086121) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.53366) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.22278060446414918) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.28978000000000004) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.2565024603314222) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.41237999999999997) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.28174299705440914) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.61846) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1429779412119699) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.17951999999999999) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.19721838418827287) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.435) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.23734796957612123) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.66646) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07860442389870713) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.10271999999999998) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.11817685371390609) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.45018) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.18446648963991358) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.69634) - val-core/sciknoweval/acc/best@16/std:np.float64(0.0437133055991171) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.061799999999999994) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05167005663723248) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.46228) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.13956489639286812) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.04534421173973564) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9991) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.011920881683771797) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9766400000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.05986222536212083) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.98834) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.04394565641819693) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.95654) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0761555953412347) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99666) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.023358458330384288) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.92376) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.08787697982491473) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99946) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.008275134421193774) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.88028) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.08368042461510791) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999800000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0006321392251711644) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:3 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.015225766226649284 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.015225766226649284 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:322.42578125 - response_length/max:1273.0 - response_length/min:91.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:322.42578125 - response_length_non_aborted/max:1273.0 - response_length_non_aborted/min:91.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:332.3125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011311471462249756 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7200889475643635) - timing_s/agent_loop/generate_sequences/max:np.float64(7.788614375516772) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9231499576562783) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.788614375516772) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:1273 - timing_s/gen:13.558132458478212 - timing_s/reward:0.05939585529267788 - timing_s/old_log_prob:2.340417541563511 - timing_s/adv:0.5785594340413809 - timing_s/update_actor:16.77560812421143 - timing_s/step:33.39719614572823 - timing_s/testing:73.396837702021 - timing_s/save_checkpoint:7.072169976308942 - timing_s/stop_profile:0.00019020028412342072 - timing_per_token_ms/gen:0.16425936756858062 - timing_per_token_ms/update_actor:0.10008536404820288 - timing_per_token_ms/adv:0.003451757525021215 - perf/total_num_tokens:167613 - perf/time_per_step:33.39719614572823 - perf/throughput:627.3468260202999 (TaskRunner pid=2122883) Training Progress: 50%|█████ | 50/100 [43:02<57:00, 68.41s/it] (TaskRunner pid=2122883) step:51 - global_seqlen/min:14724 - global_seqlen/max:24248 - global_seqlen/minmax_diff:9524 - global_seqlen/balanced_min:20154 - global_seqlen/balanced_max:20164 - global_seqlen/mean:20159.0 - actor/entropy:0.170307457447052 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45877787470817566 - training/rollout_probs_diff_mean:0.004090906120836735 - training/rollout_probs_diff_std:0.012903047725558281 - training/rollout_actor_probs_pearson_corr:0.9979671835899353 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.62109375 - self_distillation/correct_sample_fraction:0.37109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.62109375 - rollout_corr/rollout_is_mean:1.0002143383026123 - rollout_corr/rollout_is_ratio_fraction_high:5.2099614549661055e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.907471182174049e-05 - rollout_corr/rollout_is_max:2.633291482925415 - rollout_corr/rollout_is_min:0.09349794685840607 - rollout_corr/rollout_is_std:0.038960132747888565 - rollout_corr/rollout_is_eff_sample_size:0.9984850028614497 - rollout_corr/rollout_is_seq_mean:1.0002855062484741 - rollout_corr/rollout_is_seq_std:0.003179777879267931 - rollout_corr/rollout_is_seq_max:1.0115247964859009 - rollout_corr/rollout_is_seq_min:0.9862546324729919 - rollout_corr/rollout_is_seq_max_deviation:0.013745367527008057 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2051998022943735) - rollout_corr/training_log_ppl:np.float64(0.1835695655317977) - rollout_corr/kl:np.float64(0.0019460881946997688) - rollout_corr/k3_kl:np.float64(0.0018434013255159698) - rollout_corr/rollout_ppl:np.float64(1.2028441824950278) - rollout_corr/rollout_log_ppl:np.float64(0.18162347658653744) - rollout_corr/log_ppl_diff:np.float64(0.0019460889452602714) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031690965115558356) - rollout_corr/log_ppl_diff_max:np.float64(0.05628243088722229) - rollout_corr/log_ppl_diff_min:np.float64(-0.006894886493682861) - rollout_corr/ppl_ratio:np.float64(1.0019650857429951) - rollout_corr/chi2_token:np.float64(0.0026275888085365295) - rollout_corr/chi2_seq:np.float64(0.2529987192247063) - actor/pg_loss:np.float64(0.005627889535389841) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001572497030338127) - actor/ppo_kl:np.float64(0.0013443404310486962) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.62109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.37109375) - self_distillation/token_reweight_w_mean:np.float64(655015.5512642735) - self_distillation/token_reweight_w_std:np.float64(7001571.50137981) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004862013738602) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09912433978752233) - self_distillation/empty_target_batch:np.float64(0.37890625) - actor/grad_norm:np.float64(0.678714781999588) - perf/mfu/actor:np.float64(0.034301143289589886) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.97393035888672) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:3 - critic/score/mean:0.37109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.37109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005240895785391331 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005240895785391331 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:299.90625 - response_length/max:1032.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:299.90625 - response_length_non_aborted/max:1032.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.0625 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011244602501392365 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1639153379946947) - timing_s/agent_loop/generate_sequences/max:np.float64(7.034878183156252) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9088878281690995) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.034878183156252) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:432 - timing_s/agent_loop/slowest/response_length:1032 - timing_s/gen:12.909786026924849 - timing_s/reward:0.05316934175789356 - timing_s/old_log_prob:2.4393669683486223 - timing_s/adv:0.5470726881176233 - timing_s/update_actor:16.608018811792135 - timing_s/step:32.66919903643429 - timing_s/stop_profile:0.00011144764721393585 - timing_per_token_ms/gen:0.16814871870017778 - timing_per_token_ms/update_actor:0.10298141532189181 - timing_per_token_ms/adv:0.0033922360243416297 - perf/total_num_tokens:161272 - perf/time_per_step:32.66919903643429 - perf/throughput:617.064409124867 (TaskRunner pid=2122883) Training Progress: 51%|█████ | 51/100 [43:34<47:07, 57.71s/it] (TaskRunner pid=2122883) step:52 - global_seqlen/min:11438 - global_seqlen/max:23198 - global_seqlen/minmax_diff:11760 - global_seqlen/balanced_min:16158 - global_seqlen/balanced_max:16373 - global_seqlen/mean:16195.625 - actor/entropy:0.1720925271511078 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7303562164306641 - training/rollout_probs_diff_mean:0.0043999128974974155 - training/rollout_probs_diff_std:0.0137338163331151 - training/rollout_actor_probs_pearson_corr:0.9976982474327087 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.6171875 - self_distillation/correct_sample_fraction:0.4609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6171875 - rollout_corr/rollout_is_mean:0.999703049659729 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00013763441529590636 - rollout_corr/rollout_is_max:1.7618439197540283 - rollout_corr/rollout_is_min:0.03414538502693176 - rollout_corr/rollout_is_std:0.03829740732908249 - rollout_corr/rollout_is_eff_sample_size:0.9985346839476352 - rollout_corr/rollout_is_seq_mean:0.9996437430381775 - rollout_corr/rollout_is_seq_std:0.003205541055649519 - rollout_corr/rollout_is_seq_max:1.008802890777588 - rollout_corr/rollout_is_seq_min:0.9890128374099731 - rollout_corr/rollout_is_seq_max_deviation:0.010987162590026855 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2097071488387883) - rollout_corr/training_log_ppl:np.float64(0.18765684607205912) - rollout_corr/kl:np.float64(0.0014238237613923843) - rollout_corr/k3_kl:np.float64(0.0011359687883611969) - rollout_corr/rollout_ppl:np.float64(1.2079426697455347) - rollout_corr/rollout_log_ppl:np.float64(0.18623302060586866) - rollout_corr/log_ppl_diff:np.float64(0.0014238254661904648) - rollout_corr/log_ppl_abs_diff:np.float64(0.0030733039166079834) - rollout_corr/log_ppl_diff_max:np.float64(0.016380250453948975) - rollout_corr/log_ppl_diff_min:np.float64(-0.01520422101020813) - rollout_corr/ppl_ratio:np.float64(1.0014326500240713) - rollout_corr/chi2_token:np.float64(0.001753966324031353) - rollout_corr/chi2_seq:np.float64(0.811463154386729) - actor/pg_loss:np.float64(0.0041076281340792775) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010653491672201199) - actor/ppo_kl:np.float64(0.00019217796503312456) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6171875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4609375) - self_distillation/token_reweight_w_mean:np.float64(546943.084549105) - self_distillation/token_reweight_w_std:np.float64(5536941.048060441) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003938139416277) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07578522652329411) - self_distillation/empty_target_batch:np.float64(0.3828125) - actor/grad_norm:np.float64(0.6149652451276779) - perf/mfu/actor:np.float64(0.02765064554259175) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.88561248779297) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:3 - critic/score/mean:0.4609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01032258104532957 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01032258104532957 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:227.05078125 - response_length/max:1591.0 - response_length/min:91.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:227.05078125 - response_length_non_aborted/max:1591.0 - response_length_non_aborted/min:91.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.0625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001474972814321518 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1343602985143661) - timing_s/agent_loop/generate_sequences/max:np.float64(8.92716247588396) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.213136707934609) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.92716247588396) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1591 - timing_s/gen:14.800771605223417 - timing_s/reward:0.05206748843193054 - timing_s/old_log_prob:2.2524862699210644 - timing_s/adv:0.5898469276726246 - timing_s/update_actor:16.551839008927345 - timing_s/step:34.33575841598213 - timing_s/stop_profile:2.8537586331367493e-05 - timing_per_token_ms/gen:0.2546369308425534 - timing_per_token_ms/update_actor:0.12774930736639792 - timing_per_token_ms/adv:0.0045525174829052955 - perf/total_num_tokens:129565 - perf/time_per_step:34.33575841598213 - perf/throughput:471.68391633549845 (TaskRunner pid=2122883) Training Progress: 52%|█████▏ | 52/100 [44:09<40:33, 50.70s/it] (TaskRunner pid=2122883) step:53 - global_seqlen/min:13746 - global_seqlen/max:27399 - global_seqlen/minmax_diff:13653 - global_seqlen/balanced_min:20157 - global_seqlen/balanced_max:20178 - global_seqlen/mean:20171.5 - actor/entropy:0.17745019495487213 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.717710018157959 - training/rollout_probs_diff_mean:0.003805093001574278 - training/rollout_probs_diff_std:0.012700757943093777 - training/rollout_actor_probs_pearson_corr:0.9981307983398438 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6796875 - self_distillation/correct_sample_fraction:0.4453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6796875 - rollout_corr/rollout_is_mean:0.999994158744812 - rollout_corr/rollout_is_ratio_fraction_high:6.55514159006998e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001573234039824456 - rollout_corr/rollout_is_max:5.53007698059082 - rollout_corr/rollout_is_min:0.014541520737111568 - rollout_corr/rollout_is_std:0.036859575659036636 - rollout_corr/rollout_is_eff_sample_size:0.9986429773213285 - rollout_corr/rollout_is_seq_mean:1.0000602006912231 - rollout_corr/rollout_is_seq_std:0.003147674258798361 - rollout_corr/rollout_is_seq_max:1.0142172574996948 - rollout_corr/rollout_is_seq_min:0.9902289509773254 - rollout_corr/rollout_is_seq_max_deviation:0.014217257499694824 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2197709884494543) - rollout_corr/training_log_ppl:np.float64(0.19364592210331466) - rollout_corr/kl:np.float64(0.0007903323407845164) - rollout_corr/k3_kl:np.float64(0.0009314046881740978) - rollout_corr/rollout_ppl:np.float64(1.218785626348108) - rollout_corr/rollout_log_ppl:np.float64(0.1928555882332148) - rollout_corr/log_ppl_diff:np.float64(0.0007903338700998574) - rollout_corr/log_ppl_abs_diff:np.float64(0.002397695294348523) - rollout_corr/log_ppl_diff_max:np.float64(0.010247737169265747) - rollout_corr/log_ppl_diff_min:np.float64(-0.007017850875854492) - rollout_corr/ppl_ratio:np.float64(1.0007952337618917) - rollout_corr/chi2_token:np.float64(0.0022767260670661926) - rollout_corr/chi2_seq:np.float64(0.6853257885668427) - actor/pg_loss:np.float64(0.003560527227818966) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000947475578641388) - actor/ppo_kl:np.float64(-5.2774539890165784e-05) - actor/pg_clipfrac_lower:np.float64(1.4051259313418996e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4453125) - self_distillation/token_reweight_w_mean:np.float64(511538.4202222314) - self_distillation/token_reweight_w_std:np.float64(5941356.751985744) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015118459705263) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08175502343510743) - self_distillation/empty_target_batch:np.float64(0.3203125) - actor/grad_norm:np.float64(0.4588034823536873) - perf/mfu/actor:np.float64(0.034160046659816826) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.91546630859375) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:3 - critic/score/mean:0.4453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.016197755932807922 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.016197755932807922 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:297.953125 - response_length/max:1277.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:297.953125 - response_length_non_aborted/max:1277.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:332.40625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.4145469069480896e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7408090196549892) - timing_s/agent_loop/generate_sequences/max:np.float64(8.517019476741552) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.6340090800222242) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.517019476741552) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1277 - timing_s/gen:14.336498888209462 - timing_s/reward:0.05450882762670517 - timing_s/old_log_prob:2.330040417611599 - timing_s/adv:0.5633099637925625 - timing_s/update_actor:16.740238409489393 - timing_s/step:34.11834845691919 - timing_s/stop_profile:0.0001228339970111847 - timing_per_token_ms/gen:0.18795556778291286 - timing_per_token_ms/update_actor:0.10373694574950669 - timing_per_token_ms/adv:0.003490754057659089 - perf/total_num_tokens:161372 - perf/time_per_step:34.11834845691919 - perf/throughput:591.2214662286569 (TaskRunner pid=2122883) Training Progress: 53%|█████▎ | 53/100 [44:43<35:49, 45.74s/it] (TaskRunner pid=2122883) step:54 - global_seqlen/min:11593 - global_seqlen/max:21800 - global_seqlen/minmax_diff:10207 - global_seqlen/balanced_min:18264 - global_seqlen/balanced_max:18625 - global_seqlen/mean:18316.875 - actor/entropy:0.19040422141551971 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5750171542167664 - training/rollout_probs_diff_mean:0.004019932821393013 - training/rollout_probs_diff_std:0.012364245019853115 - training/rollout_actor_probs_pearson_corr:0.9983108639717102 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.82421875 - self_distillation/correct_sample_fraction:0.51171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.82421875 - rollout_corr/rollout_is_mean:0.9999626874923706 - rollout_corr/rollout_is_ratio_fraction_high:4.251760765328072e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014172536612022668 - rollout_corr/rollout_is_max:3.3674063682556152 - rollout_corr/rollout_is_min:0.07618959248065948 - rollout_corr/rollout_is_std:0.03785360977053642 - rollout_corr/rollout_is_eff_sample_size:0.9985689168550453 - rollout_corr/rollout_is_seq_mean:0.9999775290489197 - rollout_corr/rollout_is_seq_std:0.002974021015688777 - rollout_corr/rollout_is_seq_max:1.0119009017944336 - rollout_corr/rollout_is_seq_min:0.9925116896629333 - rollout_corr/rollout_is_seq_max_deviation:0.011900901794433594 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2383890594355762) - rollout_corr/training_log_ppl:np.float64(0.2090079904155573) - rollout_corr/kl:np.float64(0.001236176363808461) - rollout_corr/k3_kl:np.float64(0.0011164603104845128) - rollout_corr/rollout_ppl:np.float64(1.2368075558915734) - rollout_corr/rollout_log_ppl:np.float64(0.2077718133223243) - rollout_corr/log_ppl_diff:np.float64(0.0012361770932329819) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026738402812043205) - rollout_corr/log_ppl_diff_max:np.float64(0.011941254138946533) - rollout_corr/log_ppl_diff_min:np.float64(-0.008814394474029541) - rollout_corr/ppl_ratio:np.float64(1.0012423689477146) - rollout_corr/chi2_token:np.float64(0.00228139734826982) - rollout_corr/chi2_seq:np.float64(0.4417079407721758) - actor/pg_loss:np.float64(0.004966978449374437) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016709646415620227) - actor/ppo_kl:np.float64(0.000317133653719992) - actor/pg_clipfrac_lower:np.float64(1.306438116444042e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.82421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.82421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.51171875) - self_distillation/token_reweight_w_mean:np.float64(693708.3943432588) - self_distillation/token_reweight_w_std:np.float64(7959025.133003578) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0026721898466349) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1158001013827743) - self_distillation/empty_target_batch:np.float64(0.17578125) - actor/grad_norm:np.float64(0.660299763083458) - perf/mfu/actor:np.float64(0.030901043838188515) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.95029067993164) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:3 - critic/score/mean:0.51171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.02446356974542141 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.02446356974542141 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:275.62109375 - response_length/max:1859.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:275.62109375 - response_length_non_aborted/max:1859.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.78125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.864196181297302e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6340610478073359) - timing_s/agent_loop/generate_sequences/max:np.float64(10.501076608896255) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3894221871960326) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.501076608896255) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:1859 - timing_s/gen:16.715160904452205 - timing_s/reward:0.07296787202358246 - timing_s/old_log_prob:2.283285263925791 - timing_s/adv:0.6101323831826448 - timing_s/update_actor:16.79112665913999 - timing_s/step:36.570522367954254 - timing_s/stop_profile:0.0001197177916765213 - timing_per_token_ms/gen:0.23689622733389368 - timing_per_token_ms/update_actor:0.11458782310806286 - timing_per_token_ms/adv:0.004163731416949158 - perf/total_num_tokens:146535 - perf/time_per_step:36.570522367954254 - perf/throughput:500.86446170237303 (TaskRunner pid=2122883) Training Progress: 54%|█████▍ | 54/100 [45:19<32:58, 43.00s/it] (TaskRunner pid=2122883) step:55 - global_seqlen/min:14316 - global_seqlen/max:31135 - global_seqlen/minmax_diff:16819 - global_seqlen/balanced_min:21521 - global_seqlen/balanced_max:21741 - global_seqlen/mean:21562.625 - actor/entropy:0.1820475459098816 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6819348335266113 - training/rollout_probs_diff_mean:0.0036203861236572266 - training/rollout_probs_diff_std:0.01213095709681511 - training/rollout_actor_probs_pearson_corr:0.9983159303665161 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.5546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:1.0000014305114746 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.1028050519526e-05 - rollout_corr/rollout_is_max:1.9132329225540161 - rollout_corr/rollout_is_min:0.05214012414216995 - rollout_corr/rollout_is_std:0.03391006961464882 - rollout_corr/rollout_is_eff_sample_size:0.9988514279277574 - rollout_corr/rollout_is_seq_mean:0.9999558925628662 - rollout_corr/rollout_is_seq_std:0.002855920698493719 - rollout_corr/rollout_is_seq_max:1.0095938444137573 - rollout_corr/rollout_is_seq_min:0.9906392693519592 - rollout_corr/rollout_is_seq_max_deviation:0.009593844413757324 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2300612814724445) - rollout_corr/training_log_ppl:np.float64(0.2026208189490717) - rollout_corr/kl:np.float64(0.0009841419239613458) - rollout_corr/k3_kl:np.float64(0.0009232644666212764) - rollout_corr/rollout_ppl:np.float64(1.2287878943607211) - rollout_corr/rollout_log_ppl:np.float64(0.20163667347515002) - rollout_corr/log_ppl_diff:np.float64(0.0009841454739216715) - rollout_corr/log_ppl_abs_diff:np.float64(0.002437286515487358) - rollout_corr/log_ppl_diff_max:np.float64(0.011938422918319702) - rollout_corr/log_ppl_diff_min:np.float64(-0.008513659238815308) - rollout_corr/ppl_ratio:np.float64(1.0009896873962134) - rollout_corr/chi2_token:np.float64(0.0016990478616207838) - rollout_corr/chi2_seq:np.float64(0.6990436653140932) - actor/pg_loss:np.float64(0.0053936580661684275) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010540454873080307) - actor/ppo_kl:np.float64(0.0001687855753793599) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5546875) - self_distillation/token_reweight_w_mean:np.float64(464849.280399007) - self_distillation/token_reweight_w_std:np.float64(4769288.998040117) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998792761471123) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11009695939719677) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.6255688071250916) - perf/mfu/actor:np.float64(0.035841929135174845) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.8391571044922) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:3 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0347784049808979 - critic/advantages/max:0.75 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0347784049808979 - critic/returns/max:0.75 - critic/returns/min:-0.75 - response_length/mean:343.30078125 - response_length/max:1852.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:343.30078125 - response_length_non_aborted/max:1852.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.53125 - prompt_length/max:498.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010434910655021667 - timing_s/agent_loop/generate_sequences/min:np.float64(1.195070143789053) - timing_s/agent_loop/generate_sequences/max:np.float64(11.193672521039844) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2173899157933192) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.193672521039844) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:432 - timing_s/agent_loop/slowest/response_length:1852 - timing_s/gen:17.2607198394835 - timing_s/reward:0.060050373896956444 - timing_s/old_log_prob:2.3352784402668476 - timing_s/adv:0.6170326750725508 - timing_s/update_actor:17.13137748092413 - timing_s/step:37.49287431128323 - timing_s/stop_profile:0.00011833943426609039 - timing_per_token_ms/gen:0.19640120429519828 - timing_per_token_ms/update_actor:0.09931175750241523 - timing_per_token_ms/adv:0.0035769802787957794 - perf/total_num_tokens:172501 - perf/time_per_step:37.49287431128323 - perf/throughput:575.112615292631 (TaskRunner pid=2122883) Training Progress: 55%|█████▌ | 55/100 [45:57<31:01, 41.36s/it] (TaskRunner pid=2122883) step:56 - global_seqlen/min:11656 - global_seqlen/max:32815 - global_seqlen/minmax_diff:21159 - global_seqlen/balanced_min:19457 - global_seqlen/balanced_max:20231 - global_seqlen/mean:19623.75 - actor/entropy:0.2085179090499878 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5829838514328003 - training/rollout_probs_diff_mean:0.00421536061912775 - training/rollout_probs_diff_std:0.012636987492442131 - training/rollout_actor_probs_pearson_corr:0.9983205199241638 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.765625 - self_distillation/correct_sample_fraction:0.51953125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.765625 - rollout_corr/rollout_is_mean:1.0000214576721191 - rollout_corr/rollout_is_ratio_fraction_high:4.0095157601172104e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.355536894872785e-05 - rollout_corr/rollout_is_max:2.7182812690734863 - rollout_corr/rollout_is_min:0.10317684710025787 - rollout_corr/rollout_is_std:0.037414900958538055 - rollout_corr/rollout_is_eff_sample_size:0.9986023199750391 - rollout_corr/rollout_is_seq_mean:1.0000838041305542 - rollout_corr/rollout_is_seq_std:0.0029523512348532677 - rollout_corr/rollout_is_seq_max:1.0097237825393677 - rollout_corr/rollout_is_seq_min:0.9897148013114929 - rollout_corr/rollout_is_seq_max_deviation:0.01028519868850708 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2682426059618592) - rollout_corr/training_log_ppl:np.float64(0.23284589135437272) - rollout_corr/kl:np.float64(0.001132030002182205) - rollout_corr/k3_kl:np.float64(0.0013115676476758154) - rollout_corr/rollout_ppl:np.float64(1.2667685416527092) - rollout_corr/rollout_log_ppl:np.float64(0.23171385923342314) - rollout_corr/log_ppl_diff:np.float64(0.0011320321209495887) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028422162431525066) - rollout_corr/log_ppl_diff_max:np.float64(0.017789900302886963) - rollout_corr/log_ppl_diff_min:np.float64(-0.010757237672805786) - rollout_corr/ppl_ratio:np.float64(1.0011398843489587) - rollout_corr/chi2_token:np.float64(0.0028780815191566944) - rollout_corr/chi2_seq:np.float64(0.5465702745132148) - actor/pg_loss:np.float64(0.004153699497692287) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017756936499608855) - actor/ppo_kl:np.float64(0.00032732845443295133) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.765625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.51953125) - self_distillation/token_reweight_w_mean:np.float64(616782.949018436) - self_distillation/token_reweight_w_std:np.float64(6379718.014931552) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0017636623233557) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10283011330466252) - self_distillation/empty_target_batch:np.float64(0.234375) - actor/grad_norm:np.float64(0.5960900634527206) - perf/mfu/actor:np.float64(0.032728240701524886) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.97694778442383) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:3 - critic/score/mean:0.51953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006097805453464389 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006097805453464389 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:292.2734375 - response_length/max:2520.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:292.2734375 - response_length_non_aborted/max:2520.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:320.96875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016748160123825073 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7558852564543486) - timing_s/agent_loop/generate_sequences/max:np.float64(13.91053842753172) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.4859224638566957) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.91053842753172) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:435 - timing_s/agent_loop/slowest/response_length:2520 - timing_s/gen:19.682910569012165 - timing_s/reward:0.058423759415745735 - timing_s/old_log_prob:2.3729702048003674 - timing_s/adv:0.6314276736229658 - timing_s/update_actor:17.3532410915941 - timing_s/step:40.18750685453415 - timing_s/stop_profile:0.00011737458407878876 - timing_per_token_ms/gen:0.2630631441155297 - timing_per_token_ms/update_actor:0.1105372386240786 - timing_per_token_ms/adv:0.004022088500050741 - perf/total_num_tokens:156990 - perf/time_per_step:40.18750685453415 - perf/throughput:488.30473786372625 (TaskRunner pid=2122883) Training Progress: 56%|█████▌ | 56/100 [46:37<30:05, 41.03s/it] (TaskRunner pid=2122883) step:57 - global_seqlen/min:16389 - global_seqlen/max:26616 - global_seqlen/minmax_diff:10227 - global_seqlen/balanced_min:20467 - global_seqlen/balanced_max:20671 - global_seqlen/mean:20501.625 - actor/entropy:0.18013454973697662 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5182572603225708 - training/rollout_probs_diff_mean:0.0037767167668789625 - training/rollout_probs_diff_std:0.011676754802465439 - training/rollout_actor_probs_pearson_corr:0.9983941316604614 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.60546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.0001684427261353 - rollout_corr/rollout_is_ratio_fraction_high:2.403008511464577e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.2015042557322886e-05 - rollout_corr/rollout_is_max:4.898484230041504 - rollout_corr/rollout_is_min:0.0921703428030014 - rollout_corr/rollout_is_std:0.03423544391989708 - rollout_corr/rollout_is_eff_sample_size:0.9988296631861381 - rollout_corr/rollout_is_seq_mean:1.0001211166381836 - rollout_corr/rollout_is_seq_std:0.0027834821958094835 - rollout_corr/rollout_is_seq_max:1.0092366933822632 - rollout_corr/rollout_is_seq_min:0.9900839328765869 - rollout_corr/rollout_is_seq_max_deviation:0.009916067123413086 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.225984800606966) - rollout_corr/training_log_ppl:np.float64(0.199987623404013) - rollout_corr/kl:np.float64(0.0013101380760858206) - rollout_corr/k3_kl:np.float64(0.0010357795563322725) - rollout_corr/rollout_ppl:np.float64(1.2243200689554214) - rollout_corr/rollout_log_ppl:np.float64(0.19867748432443477) - rollout_corr/log_ppl_diff:np.float64(0.0013101390795782208) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026235540281049907) - rollout_corr/log_ppl_diff_max:np.float64(0.028497636318206787) - rollout_corr/log_ppl_diff_min:np.float64(-0.01019953191280365) - rollout_corr/ppl_ratio:np.float64(1.0013179222587496) - rollout_corr/chi2_token:np.float64(0.001341148978099227) - rollout_corr/chi2_seq:np.float64(0.48321546288207173) - actor/pg_loss:np.float64(0.00407763512339443) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010287672778304113) - actor/ppo_kl:np.float64(0.0006644770689980817) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.60546875) - self_distillation/token_reweight_w_mean:np.float64(196380.35628112592) - self_distillation/token_reweight_w_std:np.float64(3012296.647777552) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001179282553494) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08650214066437911) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.47980305552482605) - perf/mfu/actor:np.float64(0.03425298709048464) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.7446174621582) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:4 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015083384700119495 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015083384700119495 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:325.11328125 - response_length/max:1789.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:325.11328125 - response_length_non_aborted/max:1789.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.5625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0005521513521671295 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1972173005342484) - timing_s/agent_loop/generate_sequences/max:np.float64(10.842901173979044) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.050342304566584) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.842901173979044) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:1789 - timing_s/gen:17.024502815678716 - timing_s/reward:0.07034870982170105 - timing_s/old_log_prob:2.4137939233332872 - timing_s/adv:0.6624380126595497 - timing_s/update_actor:16.968600936233997 - timing_s/step:37.206819297745824 - timing_s/stop_profile:0.00011276639997959137 - timing_per_token_ms/gen:0.20455013055159518 - timing_per_token_ms/update_actor:0.10345887787086389 - timing_per_token_ms/adv:0.004038936015191172 - perf/total_num_tokens:164013 - perf/time_per_step:37.206819297745824 - perf/throughput:551.0179420588659 (TaskRunner pid=2122883) Training Progress: 57%|█████▋ | 57/100 [47:15<28:41, 40.03s/it] (TaskRunner pid=2122883) step:58 - global_seqlen/min:11162 - global_seqlen/max:25766 - global_seqlen/minmax_diff:14604 - global_seqlen/balanced_min:18802 - global_seqlen/balanced_max:18833 - global_seqlen/mean:18818.25 - actor/entropy:0.19926981627941132 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5310959815979004 - training/rollout_probs_diff_mean:0.004314659163355827 - training/rollout_probs_diff_std:0.012799865566194057 - training/rollout_actor_probs_pearson_corr:0.9982891082763672 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.796875 - self_distillation/correct_sample_fraction:0.5390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.796875 - rollout_corr/rollout_is_mean:1.0001503229141235 - rollout_corr/rollout_is_ratio_fraction_high:5.6700591812841594e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.8350295906420797e-05 - rollout_corr/rollout_is_max:6.627471923828125 - rollout_corr/rollout_is_min:0.4177379608154297 - rollout_corr/rollout_is_std:0.0387515090405941 - rollout_corr/rollout_is_eff_sample_size:0.9985008099435283 - rollout_corr/rollout_is_seq_mean:1.000192403793335 - rollout_corr/rollout_is_seq_std:0.003255679737776518 - rollout_corr/rollout_is_seq_max:1.0241525173187256 - rollout_corr/rollout_is_seq_min:0.9918437004089355 - rollout_corr/rollout_is_seq_max_deviation:0.024152517318725586 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2417523465119302) - rollout_corr/training_log_ppl:np.float64(0.21233980829128996) - rollout_corr/kl:np.float64(0.001212370029840315) - rollout_corr/k3_kl:np.float64(0.0011637865334535036) - rollout_corr/rollout_ppl:np.float64(1.2402377198450267) - rollout_corr/rollout_log_ppl:np.float64(0.21112743644334842) - rollout_corr/log_ppl_diff:np.float64(0.0012123718479415402) - rollout_corr/log_ppl_abs_diff:np.float64(0.002773932719719596) - rollout_corr/log_ppl_diff_max:np.float64(0.011079519987106323) - rollout_corr/log_ppl_diff_min:np.float64(-0.012328743934631348) - rollout_corr/ppl_ratio:np.float64(1.0012194933369756) - rollout_corr/chi2_token:np.float64(0.0026497936341911554) - rollout_corr/chi2_seq:np.float64(0.6413168597500771) - actor/pg_loss:np.float64(0.004801217350177467) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016218156224567792) - actor/ppo_kl:np.float64(0.0004635169452154031) - actor/pg_clipfrac_lower:np.float64(1.594387686054688e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5390625) - self_distillation/token_reweight_w_mean:np.float64(672229.3530433087) - self_distillation/token_reweight_w_std:np.float64(7644332.062132216) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019736965186894) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09522422318696044) - self_distillation/empty_target_batch:np.float64(0.203125) - actor/grad_norm:np.float64(0.655727356672287) - perf/mfu/actor:np.float64(0.031754083710448994) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.72417449951172) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:4 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005737391300499439 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005737391300499439 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:275.5703125 - response_length/max:1404.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:275.5703125 - response_length_non_aborted/max:1404.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.204843521118164e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0642540324479342) - timing_s/agent_loop/generate_sequences/max:np.float64(8.915480151772499) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.6220424274288234) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.915480151772499) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1404 - timing_s/gen:14.68929124251008 - timing_s/reward:0.08670529164373875 - timing_s/old_log_prob:2.2912397608160973 - timing_s/adv:0.5384073723107576 - timing_s/update_actor:16.798460753634572 - timing_s/step:34.49156256392598 - timing_s/stop_profile:0.0001234114170074463 - timing_per_token_ms/gen:0.208222879291669 - timing_per_token_ms/update_actor:0.11158357414766631 - timing_per_token_ms/adv:0.0035763645152362576 - perf/total_num_tokens:150546 - perf/time_per_step:34.49156256392598 - perf/throughput:545.5899530536672 (TaskRunner pid=2122883) Training Progress: 58%|█████▊ | 58/100 [47:49<26:52, 38.38s/it] (TaskRunner pid=2122883) step:59 - global_seqlen/min:14214 - global_seqlen/max:27268 - global_seqlen/minmax_diff:13054 - global_seqlen/balanced_min:19736 - global_seqlen/balanced_max:19748 - global_seqlen/mean:19742.625 - actor/entropy:0.20446516573429108 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7574851512908936 - training/rollout_probs_diff_mean:0.004479140043258667 - training/rollout_probs_diff_std:0.01373197603970766 - training/rollout_actor_probs_pearson_corr:0.9980234503746033 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.53515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:1.0000429153442383 - rollout_corr/rollout_is_ratio_fraction_high:5.430281817098148e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001357570436084643 - rollout_corr/rollout_is_max:2.6759519577026367 - rollout_corr/rollout_is_min:0.12958838045597076 - rollout_corr/rollout_is_std:0.03966965898871422 - rollout_corr/rollout_is_eff_sample_size:0.9984289095870793 - rollout_corr/rollout_is_seq_mean:1.0000368356704712 - rollout_corr/rollout_is_seq_std:0.003124730195850134 - rollout_corr/rollout_is_seq_max:1.0123945474624634 - rollout_corr/rollout_is_seq_min:0.9908593893051147 - rollout_corr/rollout_is_seq_max_deviation:0.012394547462463379 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2470811461098492) - rollout_corr/training_log_ppl:np.float64(0.2167818500311114) - rollout_corr/kl:np.float64(0.00108407483580919) - rollout_corr/k3_kl:np.float64(0.0011458040500684774) - rollout_corr/rollout_ppl:np.float64(1.2456995188258588) - rollout_corr/rollout_log_ppl:np.float64(0.2156977732811356) - rollout_corr/log_ppl_diff:np.float64(0.001084076749975793) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028995445900363848) - rollout_corr/log_ppl_diff_max:np.float64(0.022052377462387085) - rollout_corr/log_ppl_diff_min:np.float64(-0.008070424199104309) - rollout_corr/ppl_ratio:np.float64(1.001091847429052) - rollout_corr/chi2_token:np.float64(0.0023028659634292126) - rollout_corr/chi2_seq:np.float64(0.7561908534262329) - actor/pg_loss:np.float64(0.00433993327897042) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012840861065797071) - actor/ppo_kl:np.float64(0.00017092815455699295) - actor/pg_clipfrac_lower:np.float64(1.198236168420408e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53515625) - self_distillation/token_reweight_w_mean:np.float64(333998.30719705485) - self_distillation/token_reweight_w_std:np.float64(3725829.127407816) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999660367378965) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10178501263726503) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.561421312391758) - perf/mfu/actor:np.float64(0.03354169990330243) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.7161407470703) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:4 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005727250594645739 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005727250594645739 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:287.73828125 - response_length/max:1149.0 - response_length/min:91.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:287.73828125 - response_length_non_aborted/max:1149.0 - response_length_non_aborted/min:91.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:329.21875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011886097490787506 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9756097439676523) - timing_s/agent_loop/generate_sequences/max:np.float64(7.369482506066561) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.742447040465777) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.369482506066561) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:1127 - timing_s/gen:13.280925517901778 - timing_s/reward:0.05712872929871082 - timing_s/old_log_prob:2.2685175091028214 - timing_s/adv:0.5434238649904728 - timing_s/update_actor:16.652599392458797 - timing_s/step:32.89270040392876 - timing_s/stop_profile:0.00012143328785896301 - timing_per_token_ms/gen:0.1802979258753177 - timing_per_token_ms/update_actor:0.10543557019683804 - timing_per_token_ms/adv:0.003440676360099485 - perf/total_num_tokens:157941 - perf/time_per_step:32.89270040392876 - perf/throughput:600.2129578160725 (TaskRunner pid=2122883) Training Progress: 59%|█████▉ | 59/100 [48:22<25:06, 36.75s/it] (TaskRunner pid=2122883) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2122883) validation generation end (TaskRunner pid=2122883) [prompt] system (TaskRunner pid=2122883) (TaskRunner pid=2122883) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2122883) (TaskRunner pid=2122883) A (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) user (TaskRunner pid=2122883) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2122883) (TaskRunner pid=2122883) A: -0.01 (TaskRunner pid=2122883) B: 1.69 (TaskRunner pid=2122883) C: 2.49 (TaskRunner pid=2122883) D: 0.45 (TaskRunner pid=2122883) Please reason step by step. (TaskRunner pid=2122883) assistant (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) [response] (TaskRunner pid=2122883) The folding stability score of a protein sequence is typically determined through computational methods or experimental techniques, which are not provided here. Since the question does not provide the necessary context or data to calculate the score, the answer must be based on the given options. Among the options, 1.69 is the most reasonable value for a folding stability score, as it is a positive value that indicates a stable structure. Therefore, the most plausible answer is 1.69. (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) B (TaskRunner pid=2122883) (TaskRunner pid=2122883) [ground_truth] B (TaskRunner pid=2122883) [score] 1.0 (TaskRunner pid=2122883) [acc] 1.0 (TaskRunner pid=2122883) [pred] B (TaskRunner pid=2122883) [incorrect_format] 1 (TaskRunner pid=2122883) [feedback] (TaskRunner pid=2122883) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (WorkerDict pid=2123259) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50/actor (TaskRunner pid=2122883) step:60 - global_seqlen/min:10032 - global_seqlen/max:24065 - global_seqlen/minmax_diff:14033 - global_seqlen/balanced_min:15921 - global_seqlen/balanced_max:15934 - global_seqlen/mean:15929.5 - actor/entropy:0.20668578147888184 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7802466750144958 - training/rollout_probs_diff_mean:0.004730229265987873 - training/rollout_probs_diff_std:0.014051761478185654 - training/rollout_actor_probs_pearson_corr:0.997917652130127 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.5703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:0.9999483227729797 - rollout_corr/rollout_is_ratio_fraction_high:8.867134602041915e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010640561959007755 - rollout_corr/rollout_is_max:2.6651501655578613 - rollout_corr/rollout_is_min:0.12561684846878052 - rollout_corr/rollout_is_std:0.040434580296278 - rollout_corr/rollout_is_eff_sample_size:0.9983674756689369 - rollout_corr/rollout_is_seq_mean:0.9998451471328735 - rollout_corr/rollout_is_seq_std:0.0032432135194540024 - rollout_corr/rollout_is_seq_max:1.0128782987594604 - rollout_corr/rollout_is_seq_min:0.9875000715255737 - rollout_corr/rollout_is_seq_max_deviation:0.01287829875946045 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2383267022669315) - rollout_corr/training_log_ppl:np.float64(0.2105867949721869) - rollout_corr/kl:np.float64(0.0015159379858360467) - rollout_corr/k3_kl:np.float64(0.001174004459585376) - rollout_corr/rollout_ppl:np.float64(1.2364251278340816) - rollout_corr/rollout_log_ppl:np.float64(0.20907085489307065) - rollout_corr/log_ppl_diff:np.float64(0.0015159400791162625) - rollout_corr/log_ppl_abs_diff:np.float64(0.003122222449746914) - rollout_corr/log_ppl_diff_max:np.float64(0.01887175440788269) - rollout_corr/log_ppl_diff_min:np.float64(-0.01356002688407898) - rollout_corr/ppl_ratio:np.float64(1.0015250875148922) - rollout_corr/chi2_token:np.float64(0.001692107180133462) - rollout_corr/chi2_seq:np.float64(1.5169144957326353) - actor/pg_loss:np.float64(0.005143438349477947) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001860541247879155) - actor/ppo_kl:np.float64(0.000468954139691391) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5703125) - self_distillation/token_reweight_w_mean:np.float64(806074.5182150835) - self_distillation/token_reweight_w_std:np.float64(8268406.419227212) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000429536215961) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12600182130699977) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.6441592276096344) - perf/mfu/actor:np.float64(0.027118054410325103) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.64035034179688) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.40625) - val-aux/sciknoweval/reward/std@16:np.float64(0.23654045275840702) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5031399999999999) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1897584829312906) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.30985999999999997) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2073300701260786) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.4065) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2355960011007953) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.5783400000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.13053708350897217) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.22804) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.16287994961540497) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.41476) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.18970814666766017) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.6298999999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08378253995073645) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.15946) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.11946173439217271) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.41358000000000006) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13246666664586865) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.6596000000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04367187754785185) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.10722) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07124780446176789) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.40738) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.08970724337030629) - val-aux/sciknoweval/score/mean@16:np.float64(0.40625) - val-aux/sciknoweval/score/std@16:np.float64(0.23654045275840702) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.5031399999999999) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1897584829312906) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.30985999999999997) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.2073300701260786) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.4065) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2355960011007953) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.5783400000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.13053708350897217) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.22804) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.16287994961540497) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.41476) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.18970814666766017) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.6298999999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08378253995073645) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.15946) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.11946173439217271) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.41358000000000006) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.13246666664586865) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.6596000000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04367187754785185) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.10722) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07124780446176789) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.40738) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.08970724337030629) - val-core/sciknoweval/acc/mean@16:np.float64(0.40625) - val-aux/sciknoweval/acc/std@16:np.float64(0.23654045275840702) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.5031399999999999) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1897584829312906) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.30985999999999997) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.2073300701260786) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.4065) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2355960011007953) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.5783400000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.13053708350897217) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.22804) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.16287994961540497) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.41476) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.18970814666766017) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.6298999999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08378253995073645) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.15946) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.11946173439217271) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.41358000000000006) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.13246666664586865) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.6596000000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04367187754785185) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.10722) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07124780446176789) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.40738) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.08970724337030629) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.004841229182759271) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99992) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0012623787070447602) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9973400000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.006791494680848981) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.99866) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.005000439980641703) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9951399999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.008577901841359575) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9996200000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0027304944607158617) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9918399999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.009829262434180908) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9999000000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0014106735979665884) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.98726) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.009617296917533533) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:4 - critic/score/mean:0.5703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.019443409517407417 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.019443409517407417 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:220.265625 - response_length/max:1047.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:220.265625 - response_length_non_aborted/max:1047.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013137049973011017 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9903028048574924) - timing_s/agent_loop/generate_sequences/max:np.float64(6.458359025418758) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.1430912712676218) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.458359025418758) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:1047 - timing_s/gen:12.397510398179293 - timing_s/reward:0.04977481998503208 - timing_s/old_log_prob:2.2000094763934612 - timing_s/adv:0.503861652687192 - timing_s/update_actor:16.62571001984179 - timing_s/step:31.86980871669948 - timing_s/testing:66.30790391378105 - timing_s/save_checkpoint:6.745530238375068 - timing_s/stop_profile:3.4166499972343445e-05 - timing_per_token_ms/gen:0.21986079304425218 - timing_per_token_ms/update_actor:0.13046321306257094 - timing_per_token_ms/adv:0.003953840772522615 - perf/total_num_tokens:127436 - perf/time_per_step:31.86980871669948 - perf/throughput:499.8304238849445 (TaskRunner pid=2122883) Training Progress: 60%|██████ | 60/100 [50:07<38:08, 57.22s/it] (TaskRunner pid=2122883) step:61 - global_seqlen/min:11855 - global_seqlen/max:23118 - global_seqlen/minmax_diff:11263 - global_seqlen/balanced_min:16706 - global_seqlen/balanced_max:16857 - global_seqlen/mean:16734.875 - actor/entropy:0.23181812465190887 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7169718742370605 - training/rollout_probs_diff_mean:0.004951599985361099 - training/rollout_probs_diff_std:0.01399071142077446 - training/rollout_actor_probs_pearson_corr:0.99810391664505 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.4296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:1.0000967979431152 - rollout_corr/rollout_is_ratio_fraction_high:5.225843415246345e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012193634756840765 - rollout_corr/rollout_is_max:3.395094156265259 - rollout_corr/rollout_is_min:0.23580829799175262 - rollout_corr/rollout_is_std:0.04068733751773834 - rollout_corr/rollout_is_eff_sample_size:0.9983476330593353 - rollout_corr/rollout_is_seq_mean:1.0001016855239868 - rollout_corr/rollout_is_seq_std:0.0031705843284726143 - rollout_corr/rollout_is_seq_max:1.012374997138977 - rollout_corr/rollout_is_seq_min:0.9923515915870667 - rollout_corr/rollout_is_seq_max_deviation:0.01237499713897705 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2725287731736898) - rollout_corr/training_log_ppl:np.float64(0.2377089764631819) - rollout_corr/kl:np.float64(0.0013348886182598108) - rollout_corr/k3_kl:np.float64(0.001518071228900908) - rollout_corr/rollout_ppl:np.float64(1.2708931332454085) - rollout_corr/rollout_log_ppl:np.float64(0.23637408643844537) - rollout_corr/log_ppl_diff:np.float64(0.0013348900247365236) - rollout_corr/log_ppl_abs_diff:np.float64(0.003386163560207933) - rollout_corr/log_ppl_diff_max:np.float64(0.022667646408081055) - rollout_corr/log_ppl_diff_min:np.float64(-0.016180813312530518) - rollout_corr/ppl_ratio:np.float64(1.0013469324912876) - rollout_corr/chi2_token:np.float64(0.003425400471314788) - rollout_corr/chi2_seq:np.float64(1.5097244810312986) - actor/pg_loss:np.float64(0.006231680745258927) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015820763669580629) - actor/ppo_kl:np.float64(0.00055076309819313) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4296875) - self_distillation/token_reweight_w_mean:np.float64(578582.0100038159) - self_distillation/token_reweight_w_std:np.float64(6712461.685952961) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001360923051834) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11931510880822316) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.691587895154953) - perf/mfu/actor:np.float64(0.02863750714432481) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.93616485595703) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:4 - critic/score/mean:0.4296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005421812646090984 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005421812646090984 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:224.24609375 - response_length/max:1217.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:224.24609375 - response_length_non_aborted/max:1217.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.6639550924301147e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.926751097664237) - timing_s/agent_loop/generate_sequences/max:np.float64(7.615216936916113) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.241809466169798) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.615216936916113) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1217 - timing_s/gen:13.372081184759736 - timing_s/reward:0.05007507652044296 - timing_s/old_log_prob:2.441918570548296 - timing_s/adv:0.5068533644080162 - timing_s/update_actor:16.665765838697553 - timing_s/step:33.135390520095825 - timing_s/stop_profile:0.0001241099089384079 - timing_per_token_ms/gen:0.2329346801741902 - timing_per_token_ms/update_actor:0.12448379386384387 - timing_per_token_ms/adv:0.003785906411072806 - perf/total_num_tokens:133879 - perf/time_per_step:33.135390520095825 - perf/throughput:505.04535293919946 (TaskRunner pid=2122883) Training Progress: 61%|██████ | 61/100 [50:41<32:30, 50.01s/it] (TaskRunner pid=2122883) step:62 - global_seqlen/min:11541 - global_seqlen/max:23017 - global_seqlen/minmax_diff:11476 - global_seqlen/balanced_min:17683 - global_seqlen/balanced_max:17705 - global_seqlen/mean:17697.125 - actor/entropy:0.19811317324638367 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5048303008079529 - training/rollout_probs_diff_mean:0.004269558470696211 - training/rollout_probs_diff_std:0.012749578803777695 - training/rollout_actor_probs_pearson_corr:0.9981906414031982 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.68359375 - self_distillation/correct_sample_fraction:0.51171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.68359375 - rollout_corr/rollout_is_mean:1.0000804662704468 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.53727144910954e-05 - rollout_corr/rollout_is_max:1.944337010383606 - rollout_corr/rollout_is_min:0.35625290870666504 - rollout_corr/rollout_is_std:0.03687412664294243 - rollout_corr/rollout_is_eff_sample_size:0.9986423828918339 - rollout_corr/rollout_is_seq_mean:1.0000014305114746 - rollout_corr/rollout_is_seq_std:0.0029181051068007946 - rollout_corr/rollout_is_seq_max:1.008512258529663 - rollout_corr/rollout_is_seq_min:0.9900585412979126 - rollout_corr/rollout_is_seq_max_deviation:0.009941458702087402 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2438335889019072) - rollout_corr/training_log_ppl:np.float64(0.2145167674170807) - rollout_corr/kl:np.float64(0.0009783608772053753) - rollout_corr/k3_kl:np.float64(0.0011070482911037516) - rollout_corr/rollout_ppl:np.float64(1.242590891662985) - rollout_corr/rollout_log_ppl:np.float64(0.21353840544179548) - rollout_corr/log_ppl_diff:np.float64(0.0009783619752852246) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027773703768616542) - rollout_corr/log_ppl_diff_max:np.float64(0.011119604110717773) - rollout_corr/log_ppl_diff_min:np.float64(-0.014483973383903503) - rollout_corr/ppl_ratio:np.float64(1.0009853248484433) - rollout_corr/chi2_token:np.float64(0.0024234983138740063) - rollout_corr/chi2_seq:np.float64(0.7688207423780113) - actor/pg_loss:np.float64(0.0037221149541437626) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010598614235277637) - actor/ppo_kl:np.float64(0.00016833341987876338) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.68359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.51171875) - self_distillation/token_reweight_w_mean:np.float64(444199.96810927615) - self_distillation/token_reweight_w_std:np.float64(4653540.987991009) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000051069073379) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07645721273729578) - self_distillation/empty_target_batch:np.float64(0.31640625) - actor/grad_norm:np.float64(0.5245166197419167) - perf/mfu/actor:np.float64(0.030100797751914456) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.8307876586914) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:4 - critic/score/mean:0.51171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006728899199515581 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006728899199515581 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:259.12890625 - response_length/max:1326.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:259.12890625 - response_length_non_aborted/max:1326.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.90625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013496167957782745 - timing_s/agent_loop/generate_sequences/min:np.float64(1.061777088791132) - timing_s/agent_loop/generate_sequences/max:np.float64(8.351498238742352) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.499284286684997) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.351498238742352) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:436 - timing_s/agent_loop/slowest/response_length:1326 - timing_s/gen:14.470805488526821 - timing_s/reward:0.05649738386273384 - timing_s/old_log_prob:2.302702836692333 - timing_s/adv:0.5540448240935802 - timing_s/update_actor:16.689813977107406 - timing_s/step:34.16248192638159 - timing_s/stop_profile:0.00013281218707561493 - timing_per_token_ms/gen:0.21814078852716917 - timing_per_token_ms/update_actor:0.11788506591541992 - timing_per_token_ms/adv:0.003913381581002425 - perf/total_num_tokens:141577 - perf/time_per_step:34.16248192638159 - perf/throughput:518.0280823312663 (TaskRunner pid=2122883) Training Progress: 62%|██████▏ | 62/100 [51:15<28:40, 45.27s/it] (TaskRunner pid=2122883) step:63 - global_seqlen/min:14930 - global_seqlen/max:23507 - global_seqlen/minmax_diff:8577 - global_seqlen/balanced_min:19130 - global_seqlen/balanced_max:19152 - global_seqlen/mean:19145.25 - actor/entropy:0.21696506440639496 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.49148619174957275 - training/rollout_probs_diff_mean:0.004545337986201048 - training/rollout_probs_diff_std:0.012920488603413105 - training/rollout_actor_probs_pearson_corr:0.9983041882514954 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.60546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.0000380277633667 - rollout_corr/rollout_is_ratio_fraction_high:1.4841639313090127e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.936655725236051e-05 - rollout_corr/rollout_is_max:2.3148374557495117 - rollout_corr/rollout_is_min:0.14844170212745667 - rollout_corr/rollout_is_std:0.03836194425821304 - rollout_corr/rollout_is_eff_sample_size:0.9985305238584241 - rollout_corr/rollout_is_seq_mean:1.000018835067749 - rollout_corr/rollout_is_seq_std:0.00292221549898386 - rollout_corr/rollout_is_seq_max:1.0082601308822632 - rollout_corr/rollout_is_seq_min:0.9906377792358398 - rollout_corr/rollout_is_seq_max_deviation:0.009362220764160156 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2752052140422165) - rollout_corr/training_log_ppl:np.float64(0.23854760386166163) - rollout_corr/kl:np.float64(0.0010151628624388565) - rollout_corr/k3_kl:np.float64(0.0012473572794249321) - rollout_corr/rollout_ppl:np.float64(1.2738490118645132) - rollout_corr/rollout_log_ppl:np.float64(0.23753243846294936) - rollout_corr/log_ppl_diff:np.float64(0.00101516539871227) - rollout_corr/log_ppl_abs_diff:np.float64(0.002944408348412253) - rollout_corr/log_ppl_diff_max:np.float64(0.019116565585136414) - rollout_corr/log_ppl_diff_min:np.float64(-0.01233530044555664) - rollout_corr/ppl_ratio:np.float64(1.0010242883581668) - rollout_corr/chi2_token:np.float64(0.0029916237108409405) - rollout_corr/chi2_seq:np.float64(0.7363117965869606) - actor/pg_loss:np.float64(0.0054677382577210665) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017338892685074825) - actor/ppo_kl:np.float64(0.00015598068775091178) - actor/pg_clipfrac_lower:np.float64(1.3706140634894837e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.60546875) - self_distillation/token_reweight_w_mean:np.float64(478414.26378723653) - self_distillation/token_reweight_w_std:np.float64(5065154.901935935) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991468349471688) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12116794451139867) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.6787499040365219) - perf/mfu/actor:np.float64(0.03271239188344616) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.80411529541016) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:4 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003213214920833707 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003213214920833707 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:263.1953125 - response_length/max:1198.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:263.1953125 - response_length_non_aborted/max:1198.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:335.09375 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001463424414396286 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1881604846566916) - timing_s/agent_loop/generate_sequences/max:np.float64(7.47804525308311) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.5504021373853902) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.47804525308311) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:1198 - timing_s/gen:13.364666033536196 - timing_s/reward:0.055124446749687195 - timing_s/old_log_prob:2.247605225071311 - timing_s/adv:0.5140605103224516 - timing_s/update_actor:16.550543075427413 - timing_s/step:32.820558559149504 - timing_s/stop_profile:0.00011437758803367615 - timing_per_token_ms/gen:0.19835355803876928 - timing_per_token_ms/update_actor:0.10805906866864766 - timing_per_token_ms/adv:0.00335631886709792 - perf/total_num_tokens:153162 - perf/time_per_step:32.820558559149504 - perf/throughput:583.3310230079193 (TaskRunner pid=2122883) Training Progress: 63%|██████▎ | 63/100 [51:48<25:37, 41.55s/it] (TaskRunner pid=2122883) step:64 - global_seqlen/min:16199 - global_seqlen/max:26881 - global_seqlen/minmax_diff:10682 - global_seqlen/balanced_min:21490 - global_seqlen/balanced_max:21502 - global_seqlen/mean:21497.75 - actor/entropy:0.21458002924919128 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3945135474205017 - training/rollout_probs_diff_mean:0.0047240303829312325 - training/rollout_probs_diff_std:0.01327318325638771 - training/rollout_actor_probs_pearson_corr:0.9981720447540283 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.4765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.0000271797180176 - rollout_corr/rollout_is_ratio_fraction_high:5.3534622566075996e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010706924513215199 - rollout_corr/rollout_is_max:6.235465049743652 - rollout_corr/rollout_is_min:0.25675758719444275 - rollout_corr/rollout_is_std:0.04021582379937172 - rollout_corr/rollout_is_eff_sample_size:0.9983852990445926 - rollout_corr/rollout_is_seq_mean:1.000148892402649 - rollout_corr/rollout_is_seq_std:0.0032988900784403086 - rollout_corr/rollout_is_seq_max:1.0242486000061035 - rollout_corr/rollout_is_seq_min:0.9885872006416321 - rollout_corr/rollout_is_seq_max_deviation:0.024248600006103516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.268503988161683) - rollout_corr/training_log_ppl:np.float64(0.2344126374082407) - rollout_corr/kl:np.float64(0.0013722575135601822) - rollout_corr/k3_kl:np.float64(0.0013257689138868045) - rollout_corr/rollout_ppl:np.float64(1.266730026807636) - rollout_corr/rollout_log_ppl:np.float64(0.23304037815250922) - rollout_corr/log_ppl_diff:np.float64(0.0013722592557314783) - rollout_corr/log_ppl_abs_diff:np.float64(0.002840419387212023) - rollout_corr/log_ppl_diff_max:np.float64(0.015095949172973633) - rollout_corr/log_ppl_diff_min:np.float64(-0.010050714015960693) - rollout_corr/ppl_ratio:np.float64(1.0013799467124045) - rollout_corr/chi2_token:np.float64(0.00259414897300303) - rollout_corr/chi2_seq:np.float64(0.5995087241753936) - actor/pg_loss:np.float64(0.007810348994098604) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017692785711460601) - actor/ppo_kl:np.float64(0.0005061967070645323) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4765625) - self_distillation/token_reweight_w_mean:np.float64(587268.0798286705) - self_distillation/token_reweight_w_std:np.float64(7362155.60160229) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012550125829875) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1439109018247109) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.7157184034585953) - perf/mfu/actor:np.float64(0.036492575998587316) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.8278350830078) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:4 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02531183883547783 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02531183883547783 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:291.8671875 - response_length/max:1070.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:291.8671875 - response_length_non_aborted/max:1070.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:379.9375 - prompt_length/max:499.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.023211896419525e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.191472725942731) - timing_s/agent_loop/generate_sequences/max:np.float64(7.287185866385698) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9950591388187604) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.287185866385698) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:415 - timing_s/agent_loop/slowest/response_length:1070 - timing_s/gen:13.135733580216765 - timing_s/reward:0.06018144264817238 - timing_s/old_log_prob:2.2917927093803883 - timing_s/adv:0.5165592208504677 - timing_s/update_actor:16.707958148792386 - timing_s/step:32.80109519511461 - timing_s/stop_profile:0.00012754648923873901 - timing_per_token_ms/gen:0.17580413796162592 - timing_per_token_ms/update_actor:0.09714945836652897 - timing_per_token_ms/adv:0.0030035656106480193 - perf/total_num_tokens:171982 - perf/time_per_step:32.80109519511461 - perf/throughput:655.3973235382052 (TaskRunner pid=2122883) Training Progress: 64%|██████▍ | 64/100 [52:20<23:21, 38.94s/it] (TaskRunner pid=2122883) step:65 - global_seqlen/min:13134 - global_seqlen/max:22137 - global_seqlen/minmax_diff:9003 - global_seqlen/balanced_min:17936 - global_seqlen/balanced_max:18093 - global_seqlen/mean:17977.125 - actor/entropy:0.21280328929424286 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.407850444316864 - training/rollout_probs_diff_mean:0.004871057812124491 - training/rollout_probs_diff_std:0.013702865689992905 - training/rollout_actor_probs_pearson_corr:0.9980465769767761 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.765625 - self_distillation/correct_sample_fraction:0.5234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.765625 - rollout_corr/rollout_is_mean:1.0000638961791992 - rollout_corr/rollout_is_ratio_fraction_high:1.5778594388393685e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012622875510714948 - rollout_corr/rollout_is_max:2.3150124549865723 - rollout_corr/rollout_is_min:0.2893199920654297 - rollout_corr/rollout_is_std:0.041086770594120026 - rollout_corr/rollout_is_eff_sample_size:0.9983149598221633 - rollout_corr/rollout_is_seq_mean:1.0001049041748047 - rollout_corr/rollout_is_seq_std:0.0032439520582556725 - rollout_corr/rollout_is_seq_max:1.013520359992981 - rollout_corr/rollout_is_seq_min:0.9887478947639465 - rollout_corr/rollout_is_seq_max_deviation:0.013520359992980957 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.256556692533195) - rollout_corr/training_log_ppl:np.float64(0.22500694784685038) - rollout_corr/kl:np.float64(0.001361132893192174) - rollout_corr/k3_kl:np.float64(0.0013893124849744254) - rollout_corr/rollout_ppl:np.float64(1.25478261616081) - rollout_corr/rollout_log_ppl:np.float64(0.22364581374858972) - rollout_corr/log_ppl_diff:np.float64(0.0013611340982606634) - rollout_corr/log_ppl_abs_diff:np.float64(0.003076999870245345) - rollout_corr/log_ppl_diff_max:np.float64(0.015015125274658203) - rollout_corr/log_ppl_diff_min:np.float64(-0.010976061224937439) - rollout_corr/ppl_ratio:np.float64(1.001369426259771) - rollout_corr/chi2_token:np.float64(0.0027954219840466976) - rollout_corr/chi2_seq:np.float64(0.498077409574762) - actor/pg_loss:np.float64(0.0047087157145142555) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002266843486495418) - actor/ppo_kl:np.float64(0.0005259367158743089) - actor/pg_clipfrac_lower:np.float64(2.0451570890145376e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.765625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5234375) - self_distillation/token_reweight_w_mean:np.float64(575063.0486043207) - self_distillation/token_reweight_w_std:np.float64(6029142.028431675) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999974900856614) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1101971018651966) - self_distillation/empty_target_batch:np.float64(0.234375) - actor/grad_norm:np.float64(0.6160394102334976) - perf/mfu/actor:np.float64(0.03089012319613719) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.76618576049805) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:4 - critic/score/mean:0.5234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02519841492176056 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02519841492176056 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:247.56640625 - response_length/max:1273.0 - response_length/min:109.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:247.56640625 - response_length_non_aborted/max:1273.0 - response_length_non_aborted/min:109.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.21875 - prompt_length/max:498.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014909543097019196 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2503177151083946) - timing_s/agent_loop/generate_sequences/max:np.float64(7.673106657341123) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.481383962018299) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.673106657341123) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1218 - timing_s/gen:13.46312802657485 - timing_s/reward:0.06146645545959473 - timing_s/old_log_prob:2.3153125289827585 - timing_s/adv:0.553697656840086 - timing_s/update_actor:16.608258245512843 - timing_s/step:33.08947719074786 - timing_s/stop_profile:0.00012203864753246307 - timing_per_token_ms/gen:0.21242924131111995 - timing_per_token_ms/update_actor:0.11548188493372023 - timing_per_token_ms/adv:0.0038500153447790317 - perf/total_num_tokens:143817 - perf/time_per_step:33.08947719074786 - perf/throughput:543.2882754952254 (TaskRunner pid=2122883) Training Progress: 65%|██████▌ | 65/100 [52:54<21:41, 37.20s/it] (TaskRunner pid=2122883) step:66 - global_seqlen/min:12651 - global_seqlen/max:23559 - global_seqlen/minmax_diff:10908 - global_seqlen/balanced_min:16718 - global_seqlen/balanced_max:16741 - global_seqlen/mean:16734.625 - actor/entropy:0.19700400531291962 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3154793083667755 - training/rollout_probs_diff_mean:0.00472779106348753 - training/rollout_probs_diff_std:0.013478272594511509 - training/rollout_actor_probs_pearson_corr:0.9979892373085022 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.53515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:1.0000770092010498 - rollout_corr/rollout_is_ratio_fraction_high:3.4337710530962795e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.867542106192559e-05 - rollout_corr/rollout_is_max:2.0585169792175293 - rollout_corr/rollout_is_min:0.3337665796279907 - rollout_corr/rollout_is_std:0.04114910215139389 - rollout_corr/rollout_is_eff_sample_size:0.9983098511093312 - rollout_corr/rollout_is_seq_mean:1.0000991821289062 - rollout_corr/rollout_is_seq_std:0.0033235810697078705 - rollout_corr/rollout_is_seq_max:1.010388731956482 - rollout_corr/rollout_is_seq_min:0.9896685481071472 - rollout_corr/rollout_is_seq_max_deviation:0.010388731956481934 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.230994427576661) - rollout_corr/training_log_ppl:np.float64(0.2043733007449191) - rollout_corr/kl:np.float64(0.0012404392970850608) - rollout_corr/k3_kl:np.float64(0.0012966901686013443) - rollout_corr/rollout_ppl:np.float64(1.229399942792952) - rollout_corr/rollout_log_ppl:np.float64(0.20313286200689618) - rollout_corr/log_ppl_diff:np.float64(0.001240438738022931) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031714067881694064) - rollout_corr/log_ppl_diff_max:np.float64(0.016345471143722534) - rollout_corr/log_ppl_diff_min:np.float64(-0.008899062871932983) - rollout_corr/ppl_ratio:np.float64(1.0012497496791184) - rollout_corr/chi2_token:np.float64(0.0026735947467386723) - rollout_corr/chi2_seq:np.float64(1.3038442207034677) - actor/pg_loss:np.float64(0.004394229501485825) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015602343514729) - actor/ppo_kl:np.float64(0.0004241243286600138) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53515625) - self_distillation/token_reweight_w_mean:np.float64(540545.633164251) - self_distillation/token_reweight_w_std:np.float64(5716813.33122617) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003452564124018) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07450340743525885) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.5204346254467964) - perf/mfu/actor:np.float64(0.028829480161249834) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.7582664489746) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:4 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005770881660282612 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.005770881660282612 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:227.51953125 - response_length/max:1157.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:227.51953125 - response_length_non_aborted/max:1157.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.4375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.591877460479736e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6880578771233559) - timing_s/agent_loop/generate_sequences/max:np.float64(5.9939240626990795) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.0637885364340036) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.9939240626990795) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:435 - timing_s/agent_loop/slowest/response_length:1157 - timing_s/gen:11.854472966864705 - timing_s/reward:0.05460910499095917 - timing_s/old_log_prob:2.2059535086154938 - timing_s/adv:0.5360113754868507 - timing_s/update_actor:16.550181152299047 - timing_s/step:31.287991248071194 - timing_s/stop_profile:3.69735062122345e-05 - timing_per_token_ms/gen:0.20352773571748142 - timing_per_token_ms/update_actor:0.12362228875982466 - timing_per_token_ms/adv:0.004003759984813305 - perf/total_num_tokens:133877 - perf/time_per_step:31.287991248071194 - perf/throughput:534.8577627536774 (TaskRunner pid=2122883) Training Progress: 66%|██████▌ | 66/100 [53:25<20:04, 35.43s/it] (TaskRunner pid=2122883) step:67 - global_seqlen/min:14763 - global_seqlen/max:22066 - global_seqlen/minmax_diff:7303 - global_seqlen/balanced_min:18043 - global_seqlen/balanced_max:18191 - global_seqlen/mean:18069.125 - actor/entropy:0.21443374454975128 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3140978515148163 - training/rollout_probs_diff_mean:0.0051235631108284 - training/rollout_probs_diff_std:0.013876632787287235 - training/rollout_actor_probs_pearson_corr:0.9979842305183411 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.66796875 - self_distillation/correct_sample_fraction:0.43359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.66796875 - rollout_corr/rollout_is_mean:1.0000861883163452 - rollout_corr/rollout_is_ratio_fraction_high:1.738979153742548e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.2169376431265846e-05 - rollout_corr/rollout_is_max:2.3516223430633545 - rollout_corr/rollout_is_min:0.3772035539150238 - rollout_corr/rollout_is_std:0.042313169687986374 - rollout_corr/rollout_is_eff_sample_size:0.9982130330660876 - rollout_corr/rollout_is_seq_mean:1.0001331567764282 - rollout_corr/rollout_is_seq_std:0.003116749692708254 - rollout_corr/rollout_is_seq_max:1.0102242231369019 - rollout_corr/rollout_is_seq_min:0.9922493100166321 - rollout_corr/rollout_is_seq_max_deviation:0.010224223136901855 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2491217982023954) - rollout_corr/training_log_ppl:np.float64(0.21858850728312973) - rollout_corr/kl:np.float64(0.0013292867849905576) - rollout_corr/k3_kl:np.float64(0.0013373246251262572) - rollout_corr/rollout_ppl:np.float64(1.247418281622231) - rollout_corr/rollout_log_ppl:np.float64(0.21725921976030804) - rollout_corr/log_ppl_diff:np.float64(0.0013292875228216872) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031616015621693805) - rollout_corr/log_ppl_diff_max:np.float64(0.029362037777900696) - rollout_corr/log_ppl_diff_min:np.float64(-0.00867721438407898) - rollout_corr/ppl_ratio:np.float64(1.0013392963446677) - rollout_corr/chi2_token:np.float64(0.0027240454219281673) - rollout_corr/chi2_seq:np.float64(0.4304043233860284) - actor/pg_loss:np.float64(0.003972164820879698) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017918323310368578) - actor/ppo_kl:np.float64(0.0005243421803386994) - actor/pg_clipfrac_lower:np.float64(1.6344141840818338e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.66796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.43359375) - self_distillation/token_reweight_w_mean:np.float64(706441.2849378223) - self_distillation/token_reweight_w_std:np.float64(6269580.666031567) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0021861053537577) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09675114441779442) - self_distillation/empty_target_batch:np.float64(0.33203125) - actor/grad_norm:np.float64(0.5632684454321861) - perf/mfu/actor:np.float64(0.031125756559966692) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.7952880859375) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:4 - critic/score/mean:0.43359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.43359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004525693599134684 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004525693599134684 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:224.62890625 - response_length/max:1078.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:224.62890625 - response_length_non_aborted/max:1078.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:340.03125 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.785880446434021e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1052221357822418) - timing_s/agent_loop/generate_sequences/max:np.float64(6.942589655518532) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.4225184994356823) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.942589655518532) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:1078 - timing_s/gen:12.853008408099413 - timing_s/reward:0.05389631725847721 - timing_s/old_log_prob:2.25601764023304 - timing_s/adv:0.5043847281485796 - timing_s/update_actor:16.623241163790226 - timing_s/step:32.37739512138069 - timing_s/stop_profile:0.00012384913861751556 - timing_per_token_ms/gen:0.22351114525866295 - timing_per_token_ms/update_actor:0.11499755220431417 - timing_per_token_ms/adv:0.0034892719497248733 - perf/total_num_tokens:144553 - perf/time_per_step:32.37739512138069 - perf/throughput:558.0784041538876 (TaskRunner pid=2122883) Training Progress: 67%|██████▋ | 67/100 [53:57<18:59, 34.53s/it] (TaskRunner pid=2122883) step:68 - global_seqlen/min:9746 - global_seqlen/max:20472 - global_seqlen/minmax_diff:10726 - global_seqlen/balanced_min:14882 - global_seqlen/balanced_max:14892 - global_seqlen/mean:14888.375 - actor/entropy:0.19889450073242188 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27853599190711975 - training/rollout_probs_diff_mean:0.0050133708864450455 - training/rollout_probs_diff_std:0.013850436545908451 - training/rollout_actor_probs_pearson_corr:0.9978574514389038 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.52734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:1.0000970363616943 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.141498983604833e-05 - rollout_corr/rollout_is_max:1.8664900064468384 - rollout_corr/rollout_is_min:0.3767696022987366 - rollout_corr/rollout_is_std:0.04154696688055992 - rollout_corr/rollout_is_eff_sample_size:0.9982771803450756 - rollout_corr/rollout_is_seq_mean:1.0001392364501953 - rollout_corr/rollout_is_seq_std:0.003491344628855586 - rollout_corr/rollout_is_seq_max:1.0124340057373047 - rollout_corr/rollout_is_seq_min:0.9904762506484985 - rollout_corr/rollout_is_seq_max_deviation:0.012434005737304688 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2253266391344368) - rollout_corr/training_log_ppl:np.float64(0.19990111159859225) - rollout_corr/kl:np.float64(0.0012003272438523283) - rollout_corr/k3_kl:np.float64(0.0011535957297610366) - rollout_corr/rollout_ppl:np.float64(1.2238059635274112) - rollout_corr/rollout_log_ppl:np.float64(0.19870078393432777) - rollout_corr/log_ppl_diff:np.float64(0.0012003276642644778) - rollout_corr/log_ppl_abs_diff:np.float64(0.003384005030966364) - rollout_corr/log_ppl_diff_max:np.float64(0.01639336347579956) - rollout_corr/log_ppl_diff_min:np.float64(-0.012207046151161194) - rollout_corr/ppl_ratio:np.float64(1.0012101489119232) - rollout_corr/chi2_token:np.float64(0.0021951296366751194) - rollout_corr/chi2_seq:np.float64(0.5914839308243245) - actor/pg_loss:np.float64(0.004780508461408317) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013941395154688507) - actor/ppo_kl:np.float64(0.00043155668406047454) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.52734375) - self_distillation/token_reweight_w_mean:np.float64(868609.1069036205) - self_distillation/token_reweight_w_std:np.float64(7623714.479851294) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011716361623257) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09727005992317572) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.7055970579385757) - perf/mfu/actor:np.float64(0.02586255147692835) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.74933242797852) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:4 - critic/score/mean:0.52734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.52734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005760110914707184 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005760110914707184 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:191.91796875 - response_length/max:889.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:191.91796875 - response_length_non_aborted/max:889.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.34375 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014156848192214966 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0743454471230507) - timing_s/agent_loop/generate_sequences/max:np.float64(5.634837629273534) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.979868456000986) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.634837629273534) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:889 - timing_s/gen:11.382458966225386 - timing_s/reward:0.04163476265966892 - timing_s/old_log_prob:2.2224585078656673 - timing_s/adv:0.5000829771161079 - timing_s/update_actor:16.325130246579647 - timing_s/step:30.537147894501686 - timing_s/stop_profile:0.00011890381574630737 - timing_per_token_ms/gen:0.23167570304340204 - timing_per_token_ms/update_actor:0.13706272718295018 - timing_per_token_ms/adv:0.004198602744726237 - perf/total_num_tokens:119107 - perf/time_per_step:30.537147894501686 - perf/throughput:487.54962485153044 (TaskRunner pid=2122883) Training Progress: 68%|██████▊ | 68/100 [54:28<17:47, 33.35s/it] (TaskRunner pid=2122883) step:69 - global_seqlen/min:15468 - global_seqlen/max:22188 - global_seqlen/minmax_diff:6720 - global_seqlen/balanced_min:18204 - global_seqlen/balanced_max:18567 - global_seqlen/mean:18252.5 - actor/entropy:0.19695989787578583 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3735440969467163 - training/rollout_probs_diff_mean:0.004728264175355434 - training/rollout_probs_diff_std:0.013597971759736538 - training/rollout_actor_probs_pearson_corr:0.9979573488235474 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.58984375 - self_distillation/correct_sample_fraction:0.46484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.58984375 - rollout_corr/rollout_is_mean:1.000138759613037 - rollout_corr/rollout_is_ratio_fraction_high:3.193867814843543e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.5969339074217714e-05 - rollout_corr/rollout_is_max:2.512885093688965 - rollout_corr/rollout_is_min:0.2698124945163727 - rollout_corr/rollout_is_std:0.040441952645778656 - rollout_corr/rollout_is_eff_sample_size:0.9983677133097597 - rollout_corr/rollout_is_seq_mean:1.0000832080841064 - rollout_corr/rollout_is_seq_std:0.0029280909802764654 - rollout_corr/rollout_is_seq_max:1.0104328393936157 - rollout_corr/rollout_is_seq_min:0.9879934787750244 - rollout_corr/rollout_is_seq_max_deviation:0.012006521224975586 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2370114037767053) - rollout_corr/training_log_ppl:np.float64(0.2087440811592387) - rollout_corr/kl:np.float64(0.0012408357466497932) - rollout_corr/k3_kl:np.float64(0.0011961429782729738) - rollout_corr/rollout_ppl:np.float64(1.2353982599452138) - rollout_corr/rollout_log_ppl:np.float64(0.20750324439723045) - rollout_corr/log_ppl_diff:np.float64(0.0012408367620082572) - rollout_corr/log_ppl_abs_diff:np.float64(0.002828049662639387) - rollout_corr/log_ppl_diff_max:np.float64(0.013582497835159302) - rollout_corr/log_ppl_diff_min:np.float64(-0.010901011526584625) - rollout_corr/ppl_ratio:np.float64(1.001248066779226) - rollout_corr/chi2_token:np.float64(0.002271374687552452) - rollout_corr/chi2_seq:np.float64(0.7177229449152946) - actor/pg_loss:np.float64(0.004629563074558973) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001182094197247352) - actor/ppo_kl:np.float64(0.0004323655259614867) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.58984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.46484375) - self_distillation/token_reweight_w_mean:np.float64(164067.67598723504) - self_distillation/token_reweight_w_std:np.float64(2016421.0568843854) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9979195562191308) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08755953633226454) - self_distillation/empty_target_batch:np.float64(0.41015625) - actor/grad_norm:np.float64(0.5471128672361374) - perf/mfu/actor:np.float64(0.03103007215439442) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.71195602416992) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:4 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02166440524160862 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02166440524160862 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:244.609375 - response_length/max:1402.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:244.609375 - response_length_non_aborted/max:1402.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:325.78125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011947937309741974 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1571588460355997) - timing_s/agent_loop/generate_sequences/max:np.float64(8.519991947337985) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.450770689632918) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.519991947337985) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:425 - timing_s/agent_loop/slowest/response_length:1402 - timing_s/gen:14.34707204811275 - timing_s/reward:0.05564185231924057 - timing_s/old_log_prob:2.2638629749417305 - timing_s/adv:0.49436523206532 - timing_s/update_actor:16.777688885107636 - timing_s/step:34.02742971852422 - timing_s/stop_profile:0.00012484006583690643 - timing_per_token_ms/gen:0.2291132553195904 - timing_per_token_ms/update_actor:0.11489993757778137 - timing_per_token_ms/adv:0.0033855994525771814 - perf/total_num_tokens:146020 - perf/time_per_step:34.02742971852422 - perf/throughput:536.40548672013 (TaskRunner pid=2122883) Training Progress: 69%|██████▉ | 69/100 [55:02<17:20, 33.56s/it] (TaskRunner pid=2122883) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2122883) validation generation end (TaskRunner pid=2122883) [prompt] system (TaskRunner pid=2122883) (TaskRunner pid=2122883) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2122883) (TaskRunner pid=2122883) A (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) user (TaskRunner pid=2122883) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2122883) (TaskRunner pid=2122883) A: -0.01 (TaskRunner pid=2122883) B: 1.69 (TaskRunner pid=2122883) C: 2.49 (TaskRunner pid=2122883) D: 0.45 (TaskRunner pid=2122883) Please reason step by step. (TaskRunner pid=2122883) assistant (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) [response] (TaskRunner pid=2122883) The folding stability score of a protein sequence is typically determined through computational methods or experimental techniques, which are not provided in the question. Since no specific method or data is given, the score cannot be calculated directly from the sequence alone. However, the question provides four possible scores, and without additional context, the most reasonable assumption is that the score is a typical value within the range of possible stability scores. Among the options, 1.69 is a plausible value for a folding stability score, as it is within the typical range of stability scores for proteins. Therefore, the most reasonable answer is 1.69. (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) B (TaskRunner pid=2122883) (TaskRunner pid=2122883) [ground_truth] B (TaskRunner pid=2122883) [score] 1.0 (TaskRunner pid=2122883) [acc] 1.0 (TaskRunner pid=2122883) [pred] B (TaskRunner pid=2122883) [incorrect_format] 1 (TaskRunner pid=2122883) [feedback] (TaskRunner pid=2122883) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (WorkerDict pid=2123259) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60/actor (TaskRunner pid=2122883) step:70 - global_seqlen/min:10998 - global_seqlen/max:22982 - global_seqlen/minmax_diff:11984 - global_seqlen/balanced_min:17336 - global_seqlen/balanced_max:17343 - global_seqlen/mean:17340.25 - actor/entropy:0.199662446975708 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34822899103164673 - training/rollout_probs_diff_mean:0.004988952539861202 - training/rollout_probs_diff_std:0.014087354764342308 - training/rollout_actor_probs_pearson_corr:0.9978179335594177 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.49609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:0.999821662902832 - rollout_corr/rollout_is_ratio_fraction_high:1.793078627088107e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.793078627088107e-05 - rollout_corr/rollout_is_max:2.048713207244873 - rollout_corr/rollout_is_min:0.19275815784931183 - rollout_corr/rollout_is_std:0.04096545651555061 - rollout_corr/rollout_is_eff_sample_size:0.9983241081129971 - rollout_corr/rollout_is_seq_mean:0.9997748732566833 - rollout_corr/rollout_is_seq_std:0.00317583535797894 - rollout_corr/rollout_is_seq_max:1.0109431743621826 - rollout_corr/rollout_is_seq_min:0.9912058115005493 - rollout_corr/rollout_is_seq_max_deviation:0.010943174362182617 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.235567000694573) - rollout_corr/training_log_ppl:np.float64(0.20803631153830793) - rollout_corr/kl:np.float64(0.0014276731483562344) - rollout_corr/k3_kl:np.float64(0.0011787888146841397) - rollout_corr/rollout_ppl:np.float64(1.233756847679615) - rollout_corr/rollout_log_ppl:np.float64(0.20660863688681275) - rollout_corr/log_ppl_diff:np.float64(0.001427674651495181) - rollout_corr/log_ppl_abs_diff:np.float64(0.003025784491910599) - rollout_corr/log_ppl_diff_max:np.float64(0.014226198196411133) - rollout_corr/log_ppl_diff_min:np.float64(-0.008730649948120117) - rollout_corr/ppl_ratio:np.float64(1.0014357783365995) - rollout_corr/chi2_token:np.float64(0.0017687426880002022) - rollout_corr/chi2_seq:np.float64(0.5918401211965829) - actor/pg_loss:np.float64(0.0053908772533759475) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016089424170786515) - actor/ppo_kl:np.float64(0.0003056624431962973) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.49609375) - self_distillation/token_reweight_w_mean:np.float64(589973.9683744449) - self_distillation/token_reweight_w_std:np.float64(5563161.509655834) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001278449082747) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10386245278641582) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.6108624935150146) - perf/mfu/actor:np.float64(0.029783237475419166) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.7671127319336) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.42) - val-aux/sciknoweval/reward/std@16:np.float64(0.23113610685340358) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.51528) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.21100947357072017) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.32514000000000004) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.18040222263814762) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.41974000000000006) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.22844473537910776) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.59892) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.16675057279932926) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.25292000000000003) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11798816134028049) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.41842) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.18877181836586193) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.6667599999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.1245056095938742) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.20698) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.06619895954904491) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.41896000000000005) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13423860559458853) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7202199999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.07823132973605224) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.18576) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.026280958267470154) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.42282) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.09698912402724233) - val-aux/sciknoweval/score/mean@16:np.float64(0.42) - val-aux/sciknoweval/score/std@16:np.float64(0.23113610685340358) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.51528) - val-aux/sciknoweval/score/best@2/std:np.float64(0.21100947357072017) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.32514000000000004) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.18040222263814762) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.41974000000000006) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.22844473537910776) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.59892) - val-aux/sciknoweval/score/best@4/std:np.float64(0.16675057279932926) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.25292000000000003) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11798816134028049) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.41842) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.18877181836586193) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.6667599999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.1245056095938742) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.20698) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.06619895954904491) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.41896000000000005) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.13423860559458853) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7202199999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.07823132973605224) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.18576) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.026280958267470154) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.42282) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.09698912402724233) - val-core/sciknoweval/acc/mean@16:np.float64(0.42) - val-aux/sciknoweval/acc/std@16:np.float64(0.23113610685340358) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.51528) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.21100947357072017) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.32514000000000004) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.18040222263814762) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.41974000000000006) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.22844473537910776) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.59892) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.16675057279932926) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.25292000000000003) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11798816134028049) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.41842) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.18877181836586193) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.6667599999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.1245056095938742) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.20698) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.06619895954904491) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.41896000000000005) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.13423860559458853) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7202199999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.07823132973605224) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.18576) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.026280958267470154) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.42282) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.09698912402724233) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.004841229182759271) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99982) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.001888809148643663) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9976999999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.006380438856379709) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9986799999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.004965641952456903) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.99564) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.008257747876994065) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9997) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.002431049156228644) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.99176) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.00984390166549829) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9868600000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.009494229826584146) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:4 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01323067955672741 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01323067955672741 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:217.8515625 - response_length/max:860.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:217.8515625 - response_length_non_aborted/max:860.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:324.03125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.492171764373779e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1253771558403969) - timing_s/agent_loop/generate_sequences/max:np.float64(5.687919491901994) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.208337127987761) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.687919491901994) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:860 - timing_s/gen:11.549281872808933 - timing_s/reward:0.05496336705982685 - timing_s/old_log_prob:2.313216272741556 - timing_s/adv:0.5179897584021091 - timing_s/update_actor:16.506855254992843 - timing_s/step:31.032124953344464 - timing_s/testing:66.01478779688478 - timing_s/save_checkpoint:6.830698998644948 - timing_s/stop_profile:9.629130363464355e-05 - timing_per_token_ms/gen:0.2070877151301584 - timing_per_token_ms/update_actor:0.11899233903052756 - timing_per_token_ms/adv:0.003734013050576759 - perf/total_num_tokens:138722 - perf/time_per_step:31.032124953344464 - perf/throughput:558.7838417791356 (TaskRunner pid=2122883) Training Progress: 70%|███████ | 70/100 [56:46<27:20, 54.67s/it] (TaskRunner pid=2122883) step:71 - global_seqlen/min:9804 - global_seqlen/max:21681 - global_seqlen/minmax_diff:11877 - global_seqlen/balanced_min:17495 - global_seqlen/balanced_max:17500 - global_seqlen/mean:17498.375 - actor/entropy:0.19690172374248505 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3381648659706116 - training/rollout_probs_diff_mean:0.004845378454774618 - training/rollout_probs_diff_std:0.013365901075303555 - training/rollout_actor_probs_pearson_corr:0.9980552792549133 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65234375 - self_distillation/correct_sample_fraction:0.48046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65234375 - rollout_corr/rollout_is_mean:1.0001564025878906 - rollout_corr/rollout_is_ratio_fraction_high:3.4841385058825836e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.968277011765167e-05 - rollout_corr/rollout_is_max:3.150454521179199 - rollout_corr/rollout_is_min:0.3398284912109375 - rollout_corr/rollout_is_std:0.04114330932497978 - rollout_corr/rollout_is_eff_sample_size:0.9983106827566944 - rollout_corr/rollout_is_seq_mean:1.0002278089523315 - rollout_corr/rollout_is_seq_std:0.003083531279116869 - rollout_corr/rollout_is_seq_max:1.010101556777954 - rollout_corr/rollout_is_seq_min:0.9893363118171692 - rollout_corr/rollout_is_seq_max_deviation:0.01066368818283081 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2388491262681782) - rollout_corr/training_log_ppl:np.float64(0.2105677341169212) - rollout_corr/kl:np.float64(0.0012890833140133395) - rollout_corr/k3_kl:np.float64(0.0013068942296285968) - rollout_corr/rollout_ppl:np.float64(1.2372192768380046) - rollout_corr/rollout_log_ppl:np.float64(0.2092786498251371) - rollout_corr/log_ppl_diff:np.float64(0.0012890842917840928) - rollout_corr/log_ppl_abs_diff:np.float64(0.003031075408216566) - rollout_corr/log_ppl_diff_max:np.float64(0.01689492166042328) - rollout_corr/log_ppl_diff_min:np.float64(-0.01494649052619934) - rollout_corr/ppl_ratio:np.float64(1.0012982161715627) - rollout_corr/chi2_token:np.float64(0.002663188148289919) - rollout_corr/chi2_seq:np.float64(0.42635189229622483) - actor/pg_loss:np.float64(0.006174354115501046) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00172065351762285) - actor/ppo_kl:np.float64(0.0006120751639642208) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48046875) - self_distillation/token_reweight_w_mean:np.float64(469061.6761071717) - self_distillation/token_reweight_w_std:np.float64(4943925.522530494) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003047524951398) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08837462472729385) - self_distillation/empty_target_batch:np.float64(0.34765625) - actor/grad_norm:np.float64(0.6535552144050598) - perf/mfu/actor:np.float64(0.030004138816374676) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.94376373291016) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:5 - critic/score/mean:0.48046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0025042244233191013 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0025042244233191013 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:224.23046875 - response_length/max:788.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:224.23046875 - response_length_non_aborted/max:788.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:322.59375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002894662320613861 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0912248101085424) - timing_s/agent_loop/generate_sequences/max:np.float64(5.649241955950856) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.27755354999681) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.649241955950856) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:788 - timing_s/gen:11.400874027982354 - timing_s/reward:0.06556447967886925 - timing_s/old_log_prob:2.415134770795703 - timing_s/adv:0.5144570916891098 - timing_s/update_actor:16.434071181342006 - timing_s/step:30.942065130919218 - timing_s/stop_profile:5.480460822582245e-05 - timing_per_token_ms/gen:0.19861111837329676 - timing_per_token_ms/update_actor:0.11739712388537511 - timing_per_token_ms/adv:0.003675034765293276 - perf/total_num_tokens:139987 - perf/time_per_step:30.942065130919218 - perf/throughput:565.5205923057329 (TaskRunner pid=2122883) Training Progress: 71%|███████ | 71/100 [57:17<23:02, 47.68s/it] (TaskRunner pid=2122883) step:72 - global_seqlen/min:13771 - global_seqlen/max:21717 - global_seqlen/minmax_diff:7946 - global_seqlen/balanced_min:18281 - global_seqlen/balanced_max:18449 - global_seqlen/mean:18338.0 - actor/entropy:0.19176998734474182 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37767118215560913 - training/rollout_probs_diff_mean:0.004621072206646204 - training/rollout_probs_diff_std:0.01323121227324009 - training/rollout_actor_probs_pearson_corr:0.9980506896972656 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.66796875 - self_distillation/correct_sample_fraction:0.421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.66796875 - rollout_corr/rollout_is_mean:1.0001182556152344 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.93550061644055e-05 - rollout_corr/rollout_is_max:1.8492262363433838 - rollout_corr/rollout_is_min:0.13725660741329193 - rollout_corr/rollout_is_std:0.03996754437685013 - rollout_corr/rollout_is_eff_sample_size:0.9984054996396693 - rollout_corr/rollout_is_seq_mean:1.0001261234283447 - rollout_corr/rollout_is_seq_std:0.0031107664108276367 - rollout_corr/rollout_is_seq_max:1.010321021080017 - rollout_corr/rollout_is_seq_min:0.9898220896720886 - rollout_corr/rollout_is_seq_max_deviation:0.01032102108001709 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.239236116874963) - rollout_corr/training_log_ppl:np.float64(0.2116152676026104) - rollout_corr/kl:np.float64(0.0011325802346102698) - rollout_corr/k3_kl:np.float64(0.001299673892788178) - rollout_corr/rollout_ppl:np.float64(1.2377676661126316) - rollout_corr/rollout_log_ppl:np.float64(0.21048268360027578) - rollout_corr/log_ppl_diff:np.float64(0.0011325840023346245) - rollout_corr/log_ppl_abs_diff:np.float64(0.003151193872326985) - rollout_corr/log_ppl_diff_max:np.float64(0.0144386887550354) - rollout_corr/log_ppl_diff_min:np.float64(-0.009770482778549194) - rollout_corr/ppl_ratio:np.float64(1.0011412110179663) - rollout_corr/chi2_token:np.float64(0.0029832087457180023) - rollout_corr/chi2_seq:np.float64(0.8706808432471007) - actor/pg_loss:np.float64(0.0043286209693178535) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016793236786725174) - actor/ppo_kl:np.float64(0.0003311673048500552) - actor/pg_clipfrac_lower:np.float64(3.354357795615215e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.66796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.421875) - self_distillation/token_reweight_w_mean:np.float64(500642.83659131406) - self_distillation/token_reweight_w_std:np.float64(5350166.238907624) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019122541416436) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09215450899500865) - self_distillation/empty_target_batch:np.float64(0.33203125) - actor/grad_norm:np.float64(0.5711423642933369) - perf/mfu/actor:np.float64(0.031247951307443515) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.2710838317871) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:5 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.018253633752465248 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.018253633752465248 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:246.125 - response_length/max:1318.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:246.125 - response_length_non_aborted/max:1318.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:326.9375 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010432861745357513 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9296897631138563) - timing_s/agent_loop/generate_sequences/max:np.float64(8.103419492021203) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3991422778126434) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.103419492021203) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:436 - timing_s/agent_loop/slowest/response_length:1318 - timing_s/gen:13.928170634433627 - timing_s/reward:0.4275554921478033 - timing_s/old_log_prob:2.5062245801091194 - timing_s/adv:0.598910516127944 - timing_s/update_actor:16.280453188344836 - timing_s/step:33.830064464360476 - timing_s/stop_profile:0.00017435476183891296 - timing_per_token_ms/gen:0.2210540032128242 - timing_per_token_ms/update_actor:0.11097484177898922 - timing_per_token_ms/adv:0.004082441624822391 - perf/total_num_tokens:146704 - perf/time_per_step:33.830064464360476 - perf/throughput:542.0622245434631 (TaskRunner pid=2122883) Training Progress: 72%|███████▏ | 72/100 [57:51<20:19, 43.55s/it] (TaskRunner pid=2122883) step:73 - global_seqlen/min:10239 - global_seqlen/max:17583 - global_seqlen/minmax_diff:7344 - global_seqlen/balanced_min:14427 - global_seqlen/balanced_max:14431 - global_seqlen/mean:14428.5 - actor/entropy:0.19701342284679413 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3973785638809204 - training/rollout_probs_diff_mean:0.004991715308278799 - training/rollout_probs_diff_std:0.013931449502706528 - training/rollout_actor_probs_pearson_corr:0.9978450536727905 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0000314712524414 - rollout_corr/rollout_is_ratio_fraction_high:4.2892683268291876e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00019301706925034523 - rollout_corr/rollout_is_max:2.332307815551758 - rollout_corr/rollout_is_min:0.33360418677330017 - rollout_corr/rollout_is_std:0.0416143424808979 - rollout_corr/rollout_is_eff_sample_size:0.9982712404358611 - rollout_corr/rollout_is_seq_mean:1.0000600814819336 - rollout_corr/rollout_is_seq_std:0.0033438869286328554 - rollout_corr/rollout_is_seq_max:1.0109838247299194 - rollout_corr/rollout_is_seq_min:0.9896968007087708 - rollout_corr/rollout_is_seq_max_deviation:0.010983824729919434 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2266183192841709) - rollout_corr/training_log_ppl:np.float64(0.20162912894738838) - rollout_corr/kl:np.float64(0.0010475357565766785) - rollout_corr/k3_kl:np.float64(0.0011981667771010507) - rollout_corr/rollout_ppl:np.float64(1.2252924079075456) - rollout_corr/rollout_log_ppl:np.float64(0.20058159226027783) - rollout_corr/log_ppl_diff:np.float64(0.0010475366871105507) - rollout_corr/log_ppl_abs_diff:np.float64(0.003063296157051809) - rollout_corr/log_ppl_diff_max:np.float64(0.01853451132774353) - rollout_corr/log_ppl_diff_min:np.float64(-0.009390532970428467) - rollout_corr/ppl_ratio:np.float64(1.0010553821921349) - rollout_corr/chi2_token:np.float64(0.0026172325015068054) - rollout_corr/chi2_seq:np.float64(0.2984912055544555) - actor/pg_loss:np.float64(0.005422829883173108) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0018017217225860804) - actor/ppo_kl:np.float64(0.00022023160384776475) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(578319.4778499373) - self_distillation/token_reweight_w_std:np.float64(6073846.084833732) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9982847378123552) - self_distillation/token_reweight_w_clip_frac:np.float64(0.116363640059717) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.679996520280838) - perf/mfu/actor:np.float64(0.02498840296640397) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.2462501525879) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:5 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005806596949696541 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005806596949696541 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:182.140625 - response_length/max:722.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:182.140625 - response_length_non_aborted/max:722.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.75 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014832429587841034 - timing_s/agent_loop/generate_sequences/min:np.float64(1.21829747967422) - timing_s/agent_loop/generate_sequences/max:np.float64(4.604781422764063) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.9372474718547892) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.604781422764063) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:722 - timing_s/gen:11.13407732360065 - timing_s/reward:0.05069064348936081 - timing_s/old_log_prob:2.2318159490823746 - timing_s/adv:0.49003932252526283 - timing_s/update_actor:15.927147168666124 - timing_s/step:29.924155635759234 - timing_s/stop_profile:0.0001366306096315384 - timing_per_token_ms/gen:0.23878522183238934 - timing_per_token_ms/update_actor:0.13798339370573973 - timing_per_token_ms/adv:0.004245411187279194 - perf/total_num_tokens:115428 - perf/time_per_step:29.924155635759234 - perf/throughput:482.1689933585964 (TaskRunner pid=2122883) Training Progress: 73%|███████▎ | 73/100 [58:21<17:45, 39.47s/it] (TaskRunner pid=2122883) step:74 - global_seqlen/min:12820 - global_seqlen/max:22635 - global_seqlen/minmax_diff:9815 - global_seqlen/balanced_min:16510 - global_seqlen/balanced_max:16523 - global_seqlen/mean:16520.75 - actor/entropy:0.1998564451932907 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40099841356277466 - training/rollout_probs_diff_mean:0.005098035093396902 - training/rollout_probs_diff_std:0.014329318888485432 - training/rollout_actor_probs_pearson_corr:0.997765839099884 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65234375 - self_distillation/correct_sample_fraction:0.50390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65234375 - rollout_corr/rollout_is_mean:1.0000135898590088 - rollout_corr/rollout_is_ratio_fraction_high:1.871047425083816e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.484189700335264e-05 - rollout_corr/rollout_is_max:2.0350987911224365 - rollout_corr/rollout_is_min:0.29247722029685974 - rollout_corr/rollout_is_std:0.04207443445920944 - rollout_corr/rollout_is_eff_sample_size:0.9982329891138849 - rollout_corr/rollout_is_seq_mean:1.000016212463379 - rollout_corr/rollout_is_seq_std:0.0031045295763760805 - rollout_corr/rollout_is_seq_max:1.0087541341781616 - rollout_corr/rollout_is_seq_min:0.9893158674240112 - rollout_corr/rollout_is_seq_max_deviation:0.01068413257598877 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2355217128060758) - rollout_corr/training_log_ppl:np.float64(0.20864850541693158) - rollout_corr/kl:np.float64(0.0012986420607177251) - rollout_corr/k3_kl:np.float64(0.001190482557717587) - rollout_corr/rollout_ppl:np.float64(1.233883251901716) - rollout_corr/rollout_log_ppl:np.float64(0.20734986252500676) - rollout_corr/log_ppl_diff:np.float64(0.0012986428919248283) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032230869110208005) - rollout_corr/log_ppl_diff_max:np.float64(0.015039652585983276) - rollout_corr/log_ppl_diff_min:np.float64(-0.009414374828338623) - rollout_corr/ppl_ratio:np.float64(1.0013071070425212) - rollout_corr/chi2_token:np.float64(0.002133794594556093) - rollout_corr/chi2_seq:np.float64(0.6196947358548641) - actor/pg_loss:np.float64(0.004897095495834947) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011714055831362202) - actor/ppo_kl:np.float64(0.00039855819784406776) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.50390625) - self_distillation/token_reweight_w_mean:np.float64(341515.478634842) - self_distillation/token_reweight_w_std:np.float64(3745655.63619092) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9992651597131044) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08639141090679914) - self_distillation/empty_target_batch:np.float64(0.34765625) - actor/grad_norm:np.float64(0.5267659574747086) - perf/mfu/actor:np.float64(0.02834496970975933) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.1980857849121) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:5 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009532986208796501 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009532986208796501 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:208.7734375 - response_length/max:963.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:208.7734375 - response_length_non_aborted/max:963.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:307.5 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.530914783477783e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0088061578571796) - timing_s/agent_loop/generate_sequences/max:np.float64(6.282911816611886) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.159521869587479) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.282911816611886) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:415 - timing_s/agent_loop/slowest/response_length:963 - timing_s/gen:12.101818397641182 - timing_s/reward:0.05504104122519493 - timing_s/old_log_prob:2.226160455495119 - timing_s/adv:0.4680979307740927 - timing_s/update_actor:16.286116683855653 - timing_s/step:31.225198931992054 - timing_s/stop_profile:0.00012497790157794952 - timing_per_token_ms/gen:0.2264307599753243 - timing_per_token_ms/update_actor:0.12322470744257716 - timing_per_token_ms/adv:0.003541742435831399 - perf/total_num_tokens:132166 - perf/time_per_step:31.225198931992054 - perf/throughput:529.0838990644033 (TaskRunner pid=2122883) Training Progress: 74%|███████▍ | 74/100 [58:52<16:02, 37.01s/it] (TaskRunner pid=2122883) step:75 - global_seqlen/min:12126 - global_seqlen/max:21222 - global_seqlen/minmax_diff:9096 - global_seqlen/balanced_min:18203 - global_seqlen/balanced_max:18392 - global_seqlen/mean:18236.375 - actor/entropy:0.2161518633365631 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3713438808917999 - training/rollout_probs_diff_mean:0.005161375738680363 - training/rollout_probs_diff_std:0.013850989751517773 - training/rollout_actor_probs_pearson_corr:0.9980016350746155 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:0.999866247177124 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.835853855591267e-05 - rollout_corr/rollout_is_max:1.6655349731445312 - rollout_corr/rollout_is_min:0.44153663516044617 - rollout_corr/rollout_is_std:0.04145936295390129 - rollout_corr/rollout_is_eff_sample_size:0.9982835955264238 - rollout_corr/rollout_is_seq_mean:0.9998180270195007 - rollout_corr/rollout_is_seq_std:0.002778091700747609 - rollout_corr/rollout_is_seq_max:1.0106350183486938 - rollout_corr/rollout_is_seq_min:0.9903151392936707 - rollout_corr/rollout_is_seq_max_deviation:0.010635018348693848 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2521987650543451) - rollout_corr/training_log_ppl:np.float64(0.2221425136376638) - rollout_corr/kl:np.float64(0.001645305610473713) - rollout_corr/k3_kl:np.float64(0.0012188858009380965) - rollout_corr/rollout_ppl:np.float64(1.2501694383099675) - rollout_corr/rollout_log_ppl:np.float64(0.2204972050822107) - rollout_corr/log_ppl_diff:np.float64(0.0016453085554530844) - rollout_corr/log_ppl_abs_diff:np.float64(0.0030864700238453224) - rollout_corr/log_ppl_diff_max:np.float64(0.012482985854148865) - rollout_corr/log_ppl_diff_min:np.float64(-0.006725788116455078) - rollout_corr/ppl_ratio:np.float64(1.0016529564745724) - rollout_corr/chi2_token:np.float64(0.0015822243876755238) - rollout_corr/chi2_seq:np.float64(0.3376803044229746) - actor/pg_loss:np.float64(0.004625871777534485) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014200286204868462) - actor/ppo_kl:np.float64(0.0005539787339348123) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.484375) - self_distillation/token_reweight_w_mean:np.float64(701245.2609014655) - self_distillation/token_reweight_w_std:np.float64(7589330.247221366) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001619327813387) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10732259399082977) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.6458503305912018) - perf/mfu/actor:np.float64(0.0315188448474388) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.17803955078125) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:5 - critic/score/mean:0.484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006162949837744236 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006162949837744236 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:228.57421875 - response_length/max:1278.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:228.57421875 - response_length_non_aborted/max:1278.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:341.3125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015654414892196655 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3382554687559605) - timing_s/agent_loop/generate_sequences/max:np.float64(7.776741812005639) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.4855196749194874) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.776741812005639) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:1278 - timing_s/gen:13.650750419124961 - timing_s/reward:0.04579509235918522 - timing_s/old_log_prob:2.2064243238419294 - timing_s/adv:0.47517586685717106 - timing_s/update_actor:16.09972064755857 - timing_s/step:32.5119928792119 - timing_s/stop_profile:3.0023977160453796e-05 - timing_per_token_ms/gen:0.2332863439994012 - timing_per_token_ms/update_actor:0.11035444713901865 - timing_per_token_ms/adv:0.0032570608663808668 - perf/total_num_tokens:145891 - perf/time_per_step:32.5119928792119 - perf/throughput:560.9122476051075 (TaskRunner pid=2122883) Training Progress: 75%|███████▌ | 75/100 [59:25<14:51, 35.67s/it] (TaskRunner pid=2122883) step:76 - global_seqlen/min:11565 - global_seqlen/max:20025 - global_seqlen/minmax_diff:8460 - global_seqlen/balanced_min:15676 - global_seqlen/balanced_max:15680 - global_seqlen/mean:15677.5 - actor/entropy:0.20906737446784973 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4323614835739136 - training/rollout_probs_diff_mean:0.005196167156100273 - training/rollout_probs_diff_std:0.014064192771911621 - training/rollout_actor_probs_pearson_corr:0.9979039430618286 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.64453125 - self_distillation/correct_sample_fraction:0.46484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.64453125 - rollout_corr/rollout_is_mean:0.9995588064193726 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.81677445047535e-05 - rollout_corr/rollout_is_max:1.5994808673858643 - rollout_corr/rollout_is_min:0.36089852452278137 - rollout_corr/rollout_is_std:0.041405413299798965 - rollout_corr/rollout_is_eff_sample_size:0.9982870407504366 - rollout_corr/rollout_is_seq_mean:0.9995691776275635 - rollout_corr/rollout_is_seq_std:0.003029718529433012 - rollout_corr/rollout_is_seq_max:1.0080506801605225 - rollout_corr/rollout_is_seq_min:0.9909569621086121 - rollout_corr/rollout_is_seq_max_deviation:0.00904303789138794 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2378678624518216) - rollout_corr/training_log_ppl:np.float64(0.2106911562732421) - rollout_corr/kl:np.float64(0.001780368542952715) - rollout_corr/k3_kl:np.float64(0.001284653402535696) - rollout_corr/rollout_ppl:np.float64(1.2356635369360447) - rollout_corr/rollout_log_ppl:np.float64(0.20891078899148852) - rollout_corr/log_ppl_diff:np.float64(0.0017803672817535698) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032782099151518196) - rollout_corr/log_ppl_diff_max:np.float64(0.015327438712120056) - rollout_corr/log_ppl_diff_min:np.float64(-0.010779500007629395) - rollout_corr/ppl_ratio:np.float64(1.0017898133955896) - rollout_corr/chi2_token:np.float64(0.001633466687053442) - rollout_corr/chi2_seq:np.float64(0.331364804180339) - actor/pg_loss:np.float64(0.0036399068776518106) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001190648112242343) - actor/ppo_kl:np.float64(0.00045568517619187787) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.64453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.46484375) - self_distillation/token_reweight_w_mean:np.float64(496084.83509531594) - self_distillation/token_reweight_w_std:np.float64(5315430.144841015) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001681232592091) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07428947099833749) - self_distillation/empty_target_batch:np.float64(0.35546875) - actor/grad_norm:np.float64(0.5466546416282654) - perf/mfu/actor:np.float64(0.026923698700232014) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.2565155029297) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:5 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004592355340719223 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004592355340719223 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:199.890625 - response_length/max:674.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:199.890625 - response_length_non_aborted/max:674.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.03125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.350844442844391e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1626148615032434) - timing_s/agent_loop/generate_sequences/max:np.float64(4.779665656387806) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.1255838525030413) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.779665656387806) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:412 - timing_s/agent_loop/slowest/response_length:674 - timing_s/gen:10.666637863963842 - timing_s/reward:0.053339336067438126 - timing_s/old_log_prob:2.288154808804393 - timing_s/adv:0.4929979033768177 - timing_s/update_actor:16.330744432285428 - timing_s/step:29.92172957211733 - timing_s/stop_profile:0.00012400373816490173 - timing_per_token_ms/gen:0.2084467651052107 - timing_per_token_ms/update_actor:0.13020845504931772 - timing_per_token_ms/adv:0.003930775820258473 - perf/total_num_tokens:125420 - perf/time_per_step:29.92172957211733 - perf/throughput:523.9503272099997 (TaskRunner pid=2122883) Training Progress: 76%|███████▌ | 76/100 [59:55<13:34, 33.96s/it] (TaskRunner pid=2122883) step:77 - global_seqlen/min:15406 - global_seqlen/max:21723 - global_seqlen/minmax_diff:6317 - global_seqlen/balanced_min:17270 - global_seqlen/balanced_max:17277 - global_seqlen/mean:17272.75 - actor/entropy:0.22195269167423248 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3425203263759613 - training/rollout_probs_diff_mean:0.005554586183279753 - training/rollout_probs_diff_std:0.014829994179308414 - training/rollout_actor_probs_pearson_corr:0.9977590441703796 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9998087286949158 - rollout_corr/rollout_is_ratio_fraction_high:3.536693111527711e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.073386223055422e-05 - rollout_corr/rollout_is_max:6.2831339836120605 - rollout_corr/rollout_is_min:0.22866636514663696 - rollout_corr/rollout_is_std:0.043667059391736984 - rollout_corr/rollout_is_eff_sample_size:0.9980959261529178 - rollout_corr/rollout_is_seq_mean:0.9998674392700195 - rollout_corr/rollout_is_seq_std:0.0035912925377488136 - rollout_corr/rollout_is_seq_max:1.0285019874572754 - rollout_corr/rollout_is_seq_min:0.9909583330154419 - rollout_corr/rollout_is_seq_max_deviation:0.02850198745727539 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2535131475888193) - rollout_corr/training_log_ppl:np.float64(0.2237606028211303) - rollout_corr/kl:np.float64(0.0013143475887176237) - rollout_corr/k3_kl:np.float64(0.0014206709299742215) - rollout_corr/rollout_ppl:np.float64(1.2518676673062146) - rollout_corr/rollout_log_ppl:np.float64(0.22244625524035655) - rollout_corr/log_ppl_diff:np.float64(0.001314347580773756) - rollout_corr/log_ppl_abs_diff:np.float64(0.002847455471055582) - rollout_corr/log_ppl_diff_max:np.float64(0.017824441194534302) - rollout_corr/log_ppl_diff_min:np.float64(-0.01252920925617218) - rollout_corr/ppl_ratio:np.float64(1.00132129737176) - rollout_corr/chi2_token:np.float64(0.004566720454022288) - rollout_corr/chi2_seq:np.float64(1.2924350867979228) - actor/pg_loss:np.float64(0.0070024499436840415) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012036074376737815) - actor/ppo_kl:np.float64(0.00015001226471511586) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(294809.86716741277) - self_distillation/token_reweight_w_std:np.float64(3251414.243952886) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996227624360472) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10142341829487123) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.8022593855857849) - perf/mfu/actor:np.float64(0.029566345858852098) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.2310447692871) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:5 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00545755960047245 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00545755960047245 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:220.8984375 - response_length/max:607.0 - response_length/min:106.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:220.8984375 - response_length_non_aborted/max:607.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:318.875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.929388761520386e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.813449727371335) - timing_s/agent_loop/generate_sequences/max:np.float64(4.553269684314728) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.197502375282056) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.553269684314728) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:607 - timing_s/gen:10.332580428570509 - timing_s/reward:0.05444280430674553 - timing_s/old_log_prob:2.2673390675336123 - timing_s/adv:0.5042030327022076 - timing_s/update_actor:16.172887090593576 - timing_s/step:29.42025664076209 - timing_s/stop_profile:0.00011752359569072723 - timing_per_token_ms/gen:0.1827158342806456 - timing_per_token_ms/update_actor:0.11704047626024791 - timing_per_token_ms/adv:0.003648832935564745 - perf/total_num_tokens:138182 - perf/time_per_step:29.42025664076209 - perf/throughput:587.1039879396706 (TaskRunner pid=2122883) Training Progress: 77%|███████▋ | 77/100 [1:00:24<12:30, 32.61s/it] (TaskRunner pid=2122883) step:78 - global_seqlen/min:14370 - global_seqlen/max:23360 - global_seqlen/minmax_diff:8990 - global_seqlen/balanced_min:18531 - global_seqlen/balanced_max:18674 - global_seqlen/mean:18562.25 - actor/entropy:0.20804251730442047 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.41408851742744446 - training/rollout_probs_diff_mean:0.004874464590102434 - training/rollout_probs_diff_std:0.013253279961645603 - training/rollout_actor_probs_pearson_corr:0.9981226921081543 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.58984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0000364780426025 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.655927114072256e-05 - rollout_corr/rollout_is_max:1.8679466247558594 - rollout_corr/rollout_is_min:0.29214736819267273 - rollout_corr/rollout_is_std:0.04069319739937782 - rollout_corr/rollout_is_eff_sample_size:0.9983468013504078 - rollout_corr/rollout_is_seq_mean:1.0000871419906616 - rollout_corr/rollout_is_seq_std:0.002924885367974639 - rollout_corr/rollout_is_seq_max:1.0078153610229492 - rollout_corr/rollout_is_seq_min:0.9887160658836365 - rollout_corr/rollout_is_seq_max_deviation:0.011283934116363525 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2604153575375676) - rollout_corr/training_log_ppl:np.float64(0.2286913382849889) - rollout_corr/kl:np.float64(0.00165782988653973) - rollout_corr/k3_kl:np.float64(0.0014850372907062592) - rollout_corr/rollout_ppl:np.float64(1.258268583100289) - rollout_corr/rollout_log_ppl:np.float64(0.22703350719530135) - rollout_corr/log_ppl_diff:np.float64(0.0016578310896875337) - rollout_corr/log_ppl_abs_diff:np.float64(0.003049075006856583) - rollout_corr/log_ppl_diff_max:np.float64(0.027070119976997375) - rollout_corr/log_ppl_diff_min:np.float64(-0.00748780369758606) - rollout_corr/ppl_ratio:np.float64(1.0016673954669386) - rollout_corr/chi2_token:np.float64(0.0024911961518228054) - rollout_corr/chi2_seq:np.float64(0.23507609497755766) - actor/pg_loss:np.float64(0.0077421864261850715) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020995262079850363) - actor/ppo_kl:np.float64(0.0008263197890130414) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_w_mean:np.float64(516053.4604536833) - self_distillation/token_reweight_w_std:np.float64(6114605.236582575) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.00137659907341) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12383084223256446) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.6829441338777542) - perf/mfu/actor:np.float64(0.03151336998012594) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.21386337280273) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:5 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009750287048518658 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009750287048518658 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:251.6953125 - response_length/max:1327.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:251.6953125 - response_length_non_aborted/max:1327.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:328.375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.014535367488861e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2290885895490646) - timing_s/agent_loop/generate_sequences/max:np.float64(8.05762130022049) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.65355857443501) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.05762130022049) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:1327 - timing_s/gen:13.922783764079213 - timing_s/reward:0.055119847878813744 - timing_s/old_log_prob:2.244201695546508 - timing_s/adv:0.5187073163688183 - timing_s/update_actor:16.548424838110805 - timing_s/step:33.38240977190435 - timing_s/stop_profile:0.00013656727969646454 - timing_per_token_ms/gen:0.2160782159120839 - timing_per_token_ms/update_actor:0.11143870515502434 - timing_per_token_ms/adv:0.003493025605522083 - perf/total_num_tokens:148498 - perf/time_per_step:33.38240977190435 - perf/throughput:556.0488331079847 (TaskRunner pid=2122883) Training Progress: 78%|███████▊ | 78/100 [1:00:58<12:02, 32.86s/it] (TaskRunner pid=2122883) step:79 - global_seqlen/min:12701 - global_seqlen/max:19610 - global_seqlen/minmax_diff:6909 - global_seqlen/balanced_min:17271 - global_seqlen/balanced_max:17283 - global_seqlen/mean:17280.5 - actor/entropy:0.21525056660175323 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7200708985328674 - training/rollout_probs_diff_mean:0.005195227451622486 - training/rollout_probs_diff_std:0.014740017242729664 - training/rollout_actor_probs_pearson_corr:0.9977768659591675 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.50390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:0.999945342540741 - rollout_corr/rollout_is_ratio_fraction_high:1.6934227460296825e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015240804350469261 - rollout_corr/rollout_is_max:3.029350996017456 - rollout_corr/rollout_is_min:0.015028837136924267 - rollout_corr/rollout_is_std:0.04251272976398468 - rollout_corr/rollout_is_eff_sample_size:0.9981956909582614 - rollout_corr/rollout_is_seq_mean:0.9999911785125732 - rollout_corr/rollout_is_seq_std:0.0029612616635859013 - rollout_corr/rollout_is_seq_max:1.0107840299606323 - rollout_corr/rollout_is_seq_min:0.9903897047042847 - rollout_corr/rollout_is_seq_max_deviation:0.010784029960632324 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2490549199283123) - rollout_corr/training_log_ppl:np.float64(0.2196736648620572) - rollout_corr/kl:np.float64(0.001692611791760612) - rollout_corr/k3_kl:np.float64(0.0015163807295834886) - rollout_corr/rollout_ppl:np.float64(1.246921275742352) - rollout_corr/rollout_log_ppl:np.float64(0.21798105229390785) - rollout_corr/log_ppl_diff:np.float64(0.0016926125681493431) - rollout_corr/log_ppl_abs_diff:np.float64(0.003021725657163188) - rollout_corr/log_ppl_diff_max:np.float64(0.033561140298843384) - rollout_corr/log_ppl_diff_min:np.float64(-0.007312938570976257) - rollout_corr/ppl_ratio:np.float64(1.00170242250897) - rollout_corr/chi2_token:np.float64(0.002288506831973791) - rollout_corr/chi2_seq:np.float64(0.5073147353250533) - actor/pg_loss:np.float64(0.006525568896904588) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014381463884092227) - actor/ppo_kl:np.float64(0.0006511699603919396) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.50390625) - self_distillation/token_reweight_w_mean:np.float64(377157.56747266906) - self_distillation/token_reweight_w_std:np.float64(4524650.8311734535) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991592520382255) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11779780086362734) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.6401597559452057) - perf/mfu/actor:np.float64(0.0296049883148389) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.26931762695312) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:5 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0018056120024994016 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0018056120024994016 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:230.671875 - response_length/max:890.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:230.671875 - response_length_non_aborted/max:890.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:309.34375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012810342013835907 - timing_s/agent_loop/generate_sequences/min:np.float64(1.271697724238038) - timing_s/agent_loop/generate_sequences/max:np.float64(5.964426325634122) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.5344342128373682) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.964426325634122) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:890 - timing_s/gen:11.82465054281056 - timing_s/reward:0.06565968319773674 - timing_s/old_log_prob:2.223105376586318 - timing_s/adv:0.4894870724529028 - timing_s/update_actor:16.2752229552716 - timing_s/step:30.97196850553155 - timing_s/stop_profile:0.00010983459651470184 - timing_per_token_ms/gen:0.20024132193339023 - timing_per_token_ms/update_actor:0.11772824104678396 - timing_per_token_ms/adv:0.003540747319615338 - perf/total_num_tokens:138244 - perf/time_per_step:30.97196850553155 - perf/throughput:557.939996513742 (TaskRunner pid=2122883) Training Progress: 79%|███████▉ | 79/100 [1:01:29<11:18, 32.30s/it] (TaskRunner pid=2122883) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2122883) validation generation end (TaskRunner pid=2122883) [prompt] system (TaskRunner pid=2122883) (TaskRunner pid=2122883) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2122883) (TaskRunner pid=2122883) A (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) user (TaskRunner pid=2122883) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2122883) (TaskRunner pid=2122883) A: -0.01 (TaskRunner pid=2122883) B: 1.69 (TaskRunner pid=2122883) C: 2.49 (TaskRunner pid=2122883) D: 0.45 (TaskRunner pid=2122883) Please reason step by step. (TaskRunner pid=2122883) assistant (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) [response] (TaskRunner pid=2122883) The folding stability score of a protein sequence is typically determined through computational methods or experimental techniques, which are not provided in the question. Without access to specific computational tools or experimental data, it is not possible to calculate the exact folding stability score for the given sequence. However, the question provides four possible scores: -0.01, 1.69, 2.49, and 0.45. Among these, 1.69 is a reasonable value for a folding stability score, as it is positive and within a typical range for such scores. Therefore, the most plausible answer is 1.69. (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) B (TaskRunner pid=2122883) (TaskRunner pid=2122883) [ground_truth] B (TaskRunner pid=2122883) [score] 1.0 (TaskRunner pid=2122883) [acc] 1.0 (TaskRunner pid=2122883) [pred] B (TaskRunner pid=2122883) [incorrect_format] 1 (TaskRunner pid=2122883) [feedback] (TaskRunner pid=2122883) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (WorkerDict pid=2123259) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70/actor (TaskRunner pid=2122883) step:80 - global_seqlen/min:12832 - global_seqlen/max:21001 - global_seqlen/minmax_diff:8169 - global_seqlen/balanced_min:16760 - global_seqlen/balanced_max:16771 - global_seqlen/mean:16769.125 - actor/entropy:0.21624459326267242 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6118733286857605 - training/rollout_probs_diff_mean:0.005380045156925917 - training/rollout_probs_diff_std:0.014546380378305912 - training/rollout_actor_probs_pearson_corr:0.9978134632110596 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.5546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:0.9999927282333374 - rollout_corr/rollout_is_ratio_fraction_high:3.5480494261719286e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.096098852343857e-05 - rollout_corr/rollout_is_max:3.9595537185668945 - rollout_corr/rollout_is_min:0.3052869737148285 - rollout_corr/rollout_is_std:0.04341449216008186 - rollout_corr/rollout_is_eff_sample_size:0.9981184902564971 - rollout_corr/rollout_is_seq_mean:0.9999736547470093 - rollout_corr/rollout_is_seq_std:0.003248573513701558 - rollout_corr/rollout_is_seq_max:1.0156736373901367 - rollout_corr/rollout_is_seq_min:0.990421712398529 - rollout_corr/rollout_is_seq_max_deviation:0.01567363739013672 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2467931918799877) - rollout_corr/training_log_ppl:np.float64(0.2184930287185125) - rollout_corr/kl:np.float64(0.0014139293027497501) - rollout_corr/k3_kl:np.float64(0.0013873711505993924) - rollout_corr/rollout_ppl:np.float64(1.2450102255679667) - rollout_corr/rollout_log_ppl:np.float64(0.2170790982490871) - rollout_corr/log_ppl_diff:np.float64(0.0014139304694253951) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031045038776937872) - rollout_corr/log_ppl_diff_max:np.float64(0.015252634882926941) - rollout_corr/log_ppl_diff_min:np.float64(-0.009716227650642395) - rollout_corr/ppl_ratio:np.float64(1.001422454835847) - rollout_corr/chi2_token:np.float64(0.0026412673760205507) - rollout_corr/chi2_seq:np.float64(0.6153503498062491) - actor/pg_loss:np.float64(0.005080795381218195) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020866218960691185) - actor/ppo_kl:np.float64(0.00041786666335319467) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5546875) - self_distillation/token_reweight_w_mean:np.float64(607829.0349954877) - self_distillation/token_reweight_w_std:np.float64(6253481.655585986) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0039331214502454) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09547229323652573) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.5971960201859474) - perf/mfu/actor:np.float64(0.028664539556949087) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.34428787231445) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.41875) - val-aux/sciknoweval/reward/std@16:np.float64(0.2211101008128732) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5120999999999999) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.21421579131747706) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.32600000000000007) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15530304468803183) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.41967999999999994) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.22011101001975603) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.60526) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.18121537035652593) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.27272) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.09037255406946425) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.40104) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.18007103708696343) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.68068) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.1255793418054099) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.24458) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.05154055880198174) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.38406000000000007) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13114827437959176) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7286199999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.06724812379449635) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.22526) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03666116598309839) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.37333999999999995) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.10244541245169461) - val-aux/sciknoweval/score/mean@16:np.float64(0.41875) - val-aux/sciknoweval/score/std@16:np.float64(0.2211101008128732) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.5120999999999999) - val-aux/sciknoweval/score/best@2/std:np.float64(0.21421579131747706) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.32600000000000007) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15530304468803183) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.41967999999999994) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.22011101001975603) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.60526) - val-aux/sciknoweval/score/best@4/std:np.float64(0.18121537035652593) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.27272) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.09037255406946425) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.40104) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.18007103708696343) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.68068) - val-aux/sciknoweval/score/best@8/std:np.float64(0.1255793418054099) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.24458) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.05154055880198174) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.38406000000000007) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.13114827437959176) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7286199999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.06724812379449635) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.22526) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03666116598309839) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.37333999999999995) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.10244541245169461) - val-core/sciknoweval/acc/mean@16:np.float64(0.41875) - val-aux/sciknoweval/acc/std@16:np.float64(0.2211101008128732) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.5120999999999999) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.21421579131747706) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.32600000000000007) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15530304468803183) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.41967999999999994) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.22011101001975603) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.60526) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.18121537035652593) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.27272) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.09037255406946425) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.40104) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.18007103708696343) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.68068) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.1255793418054099) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.24458) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.05154055880198174) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.38406000000000007) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.13114827437959176) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7286199999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.06724812379449635) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.22526) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03666116598309839) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.37333999999999995) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.10244541245169461) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9975) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.009682458365518542) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9998600000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0023042059141141357) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.99542) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.01273201501325385) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.99774) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.009234474228192518) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.99092) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.016755431405841622) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99928) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.00529873114696611) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.98448) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.019491610038772544) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99994) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0015256715413187116) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9750200000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.019365214541818408) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:5 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009311412461102009 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009311412461102009 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:220.19140625 - response_length/max:872.0 - response_length/min:106.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:220.19140625 - response_length_non_aborted/max:872.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:303.84375 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014083273708820343 - timing_s/agent_loop/generate_sequences/min:np.float64(1.258571656420827) - timing_s/agent_loop/generate_sequences/max:np.float64(5.787867229431868) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.4260267040735926) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.787867229431868) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:414 - timing_s/agent_loop/slowest/response_length:872 - timing_s/gen:11.63143178448081 - timing_s/reward:0.053980035707354546 - timing_s/old_log_prob:2.230306738987565 - timing_s/adv:0.5261808633804321 - timing_s/update_actor:16.25501248985529 - timing_s/step:30.786901891231537 - timing_s/testing:70.0108205396682 - timing_s/save_checkpoint:7.13819195330143 - timing_s/stop_profile:8.743070065975189e-05 - timing_per_token_ms/gen:0.20634447629868918 - timing_per_token_ms/update_actor:0.12116771514506042 - timing_per_token_ms/adv:0.003922244477428251 - perf/total_num_tokens:134153 - perf/time_per_step:30.786901891231537 - perf/throughput:544.6837443807894 (TaskRunner pid=2122883) Training Progress: 80%|████████ | 80/100 [1:03:17<18:20, 55.01s/it] (TaskRunner pid=2122883) step:81 - global_seqlen/min:15445 - global_seqlen/max:22337 - global_seqlen/minmax_diff:6892 - global_seqlen/balanced_min:18825 - global_seqlen/balanced_max:18829 - global_seqlen/mean:18827.5 - actor/entropy:0.20360738039016724 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4294052720069885 - training/rollout_probs_diff_mean:0.005188526585698128 - training/rollout_probs_diff_std:0.014588381163775921 - training/rollout_actor_probs_pearson_corr:0.9977055191993713 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0000635385513306 - rollout_corr/rollout_is_ratio_fraction_high:6.641429354203865e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.3207146771019325e-05 - rollout_corr/rollout_is_max:2.8543355464935303 - rollout_corr/rollout_is_min:0.44702544808387756 - rollout_corr/rollout_is_std:0.0434611439704895 - rollout_corr/rollout_is_eff_sample_size:0.9981148086752116 - rollout_corr/rollout_is_seq_mean:1.000145435333252 - rollout_corr/rollout_is_seq_std:0.0028170603327453136 - rollout_corr/rollout_is_seq_max:1.0175964832305908 - rollout_corr/rollout_is_seq_min:0.9926503300666809 - rollout_corr/rollout_is_seq_max_deviation:0.01759648323059082 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2413170905783772) - rollout_corr/training_log_ppl:np.float64(0.21387588272045832) - rollout_corr/kl:np.float64(0.0007690461568188312) - rollout_corr/k3_kl:np.float64(0.0012659859104928728) - rollout_corr/rollout_ppl:np.float64(1.2403291380032897) - rollout_corr/rollout_log_ppl:np.float64(0.21310683467891067) - rollout_corr/log_ppl_diff:np.float64(0.0007690480415476486) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026080097450176254) - rollout_corr/log_ppl_diff_max:np.float64(0.010331928730010986) - rollout_corr/log_ppl_diff_min:np.float64(-0.011125713586807251) - rollout_corr/ppl_ratio:np.float64(1.0007748361676931) - rollout_corr/chi2_token:np.float64(0.004497667541727424) - rollout_corr/chi2_seq:np.float64(1.3236201561521739) - actor/pg_loss:np.float64(0.005069697159342468) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008628873297311657) - actor/ppo_kl:np.float64(-7.736450616491197e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(291387.3354909057) - self_distillation/token_reweight_w_std:np.float64(3353028.3190113385) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000224343733862) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06953210350184236) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.531838670372963) - perf/mfu/actor:np.float64(0.03232239126793497) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.74435806274414) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:5 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002285066759213805 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.002285066759213805 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:235.265625 - response_length/max:684.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:235.265625 - response_length_non_aborted/max:684.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:353.09375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.15045428276062e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9042285159230232) - timing_s/agent_loop/generate_sequences/max:np.float64(4.9942101035267115) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3118886794691207) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.9942101035267115) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:424 - timing_s/agent_loop/slowest/response_length:684 - timing_s/gen:10.56088761612773 - timing_s/reward:0.052522532641887665 - timing_s/old_log_prob:2.370040809735656 - timing_s/adv:0.5358876064419746 - timing_s/update_actor:16.438335929065943 - timing_s/step:30.05812031030655 - timing_s/stop_profile:0.00012102536857128143 - timing_per_token_ms/gen:0.17534846941833915 - timing_per_token_ms/update_actor:0.1091378032735755 - timing_per_token_ms/adv:0.0035578781466071878 - perf/total_num_tokens:150620 - perf/time_per_step:30.05812031030655 - perf/throughput:626.3698396850281 (TaskRunner pid=2122883) Training Progress: 81%|████████ | 81/100 [1:03:47<15:03, 47.54s/it] (TaskRunner pid=2122883) step:82 - global_seqlen/min:12582 - global_seqlen/max:21105 - global_seqlen/minmax_diff:8523 - global_seqlen/balanced_min:16604 - global_seqlen/balanced_max:16611 - global_seqlen/mean:16609.0 - actor/entropy:0.21828316152095795 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311316251754761 - training/rollout_probs_diff_mean:0.005371991079300642 - training/rollout_probs_diff_std:0.014509456232190132 - training/rollout_actor_probs_pearson_corr:0.9978258609771729 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.59375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9998401999473572 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.5041833547875285e-05 - rollout_corr/rollout_is_max:1.7522820234298706 - rollout_corr/rollout_is_min:0.2179795652627945 - rollout_corr/rollout_is_std:0.04170876741409302 - rollout_corr/rollout_is_eff_sample_size:0.9982629246817137 - rollout_corr/rollout_is_seq_mean:0.9997910857200623 - rollout_corr/rollout_is_seq_std:0.0029545440338552 - rollout_corr/rollout_is_seq_max:1.0073826313018799 - rollout_corr/rollout_is_seq_min:0.9908558130264282 - rollout_corr/rollout_is_seq_max_deviation:0.009144186973571777 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.250613727606833) - rollout_corr/training_log_ppl:np.float64(0.2206964968208922) - rollout_corr/kl:np.float64(0.0011574556569300398) - rollout_corr/k3_kl:np.float64(0.001129698138868207) - rollout_corr/rollout_ppl:np.float64(1.2491793450899422) - rollout_corr/rollout_log_ppl:np.float64(0.21953904036490712) - rollout_corr/log_ppl_diff:np.float64(0.0011574564559850842) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027988772781100124) - rollout_corr/log_ppl_diff_max:np.float64(0.010943189263343811) - rollout_corr/log_ppl_diff_min:np.float64(-0.008710235357284546) - rollout_corr/ppl_ratio:np.float64(1.0011638076975942) - rollout_corr/chi2_token:np.float64(0.0022077024914324284) - rollout_corr/chi2_seq:np.float64(0.42010653484612703) - actor/pg_loss:np.float64(0.0061133516719564795) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015116771783141303) - actor/ppo_kl:np.float64(1.9086709578175487e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59375) - self_distillation/token_reweight_w_mean:np.float64(585352.785423923) - self_distillation/token_reweight_w_std:np.float64(5886800.302368791) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995204512961209) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11297732920502312) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.6384810656309128) - perf/mfu/actor:np.float64(0.02862514647106433) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.69329833984375) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:5 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005249614827334881 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005249614827334881 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:212.90625 - response_length/max:694.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:212.90625 - response_length_non_aborted/max:694.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:306.125 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016188248991966248 - timing_s/agent_loop/generate_sequences/min:np.float64(1.191643027588725) - timing_s/agent_loop/generate_sequences/max:np.float64(4.798683322966099) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.316618954995647) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.798683322966099) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:694 - timing_s/gen:10.743289474397898 - timing_s/reward:0.0457973200827837 - timing_s/old_log_prob:2.278144048526883 - timing_s/adv:0.49894460663199425 - timing_s/update_actor:16.217082366347313 - timing_s/step:29.820195823907852 - timing_s/stop_profile:0.00011817924678325653 - timing_per_token_ms/gen:0.19711011071477136 - timing_per_token_ms/update_actor:0.12205041217372593 - timing_per_token_ms/adv:0.003755077116563266 - perf/total_num_tokens:132872 - perf/time_per_step:29.820195823907852 - perf/throughput:556.9715268832678 (TaskRunner pid=2122883) Training Progress: 82%|████████▏ | 82/100 [1:04:17<12:40, 42.24s/it] (TaskRunner pid=2122883) step:83 - global_seqlen/min:12171 - global_seqlen/max:22671 - global_seqlen/minmax_diff:10500 - global_seqlen/balanced_min:16335 - global_seqlen/balanced_max:16338 - global_seqlen/mean:16336.875 - actor/entropy:0.21306166052818298 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3262504041194916 - training/rollout_probs_diff_mean:0.005117357242852449 - training/rollout_probs_diff_std:0.013851228170096874 - training/rollout_actor_probs_pearson_corr:0.997997522354126 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:0.9997509717941284 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.668184530804865e-05 - rollout_corr/rollout_is_max:1.9102360010147095 - rollout_corr/rollout_is_min:0.4035520553588867 - rollout_corr/rollout_is_std:0.04130885377526283 - rollout_corr/rollout_is_eff_sample_size:0.9982955945128663 - rollout_corr/rollout_is_seq_mean:0.9998254776000977 - rollout_corr/rollout_is_seq_std:0.003131643170490861 - rollout_corr/rollout_is_seq_max:1.0085992813110352 - rollout_corr/rollout_is_seq_min:0.9895705580711365 - rollout_corr/rollout_is_seq_max_deviation:0.010429441928863525 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2474157405085862) - rollout_corr/training_log_ppl:np.float64(0.21835851964715403) - rollout_corr/kl:np.float64(0.0012202397511362761) - rollout_corr/k3_kl:np.float64(0.0013064491049590288) - rollout_corr/rollout_ppl:np.float64(1.2458532857708633) - rollout_corr/rollout_log_ppl:np.float64(0.21713827828352805) - rollout_corr/log_ppl_diff:np.float64(0.0012202413636259735) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031315083615481853) - rollout_corr/log_ppl_diff_max:np.float64(0.012149259448051453) - rollout_corr/log_ppl_diff_min:np.float64(-0.011597350239753723) - rollout_corr/ppl_ratio:np.float64(1.0012280030641705) - rollout_corr/chi2_token:np.float64(0.0028531376738101244) - rollout_corr/chi2_seq:np.float64(0.5433944647666067) - actor/pg_loss:np.float64(0.0065868336241692305) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0018259112548548728) - actor/ppo_kl:np.float64(0.00016781720140102152) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(497521.2222965432) - self_distillation/token_reweight_w_std:np.float64(5335673.747744209) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009391447529197) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10771104626473971) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.696645587682724) - perf/mfu/actor:np.float64(0.02794429017338062) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.5435447692871) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:5 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006303493399173021 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006303493399173021 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:206.74609375 - response_length/max:518.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:206.74609375 - response_length_non_aborted/max:518.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:303.78125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013703666627407074 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2017349246889353) - timing_s/agent_loop/generate_sequences/max:np.float64(4.064564555883408) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.2316053204122) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.064564555883408) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:518 - timing_s/gen:9.984482478350401 - timing_s/reward:0.052736496552824974 - timing_s/old_log_prob:2.2688260599970818 - timing_s/adv:0.47870176285505295 - timing_s/update_actor:16.14213657937944 - timing_s/step:29.018089989200234 - timing_s/stop_profile:0.00011230818927288055 - timing_per_token_ms/gen:0.18864629543239558 - timing_per_token_ms/update_actor:0.12350997803572776 - timing_per_token_ms/adv:0.003662739682888044 - perf/total_num_tokens:130695 - perf/time_per_step:29.018089989200234 - perf/throughput:562.98932859055 (TaskRunner pid=2122883) Training Progress: 83%|████████▎ | 83/100 [1:04:46<10:50, 38.29s/it] (TaskRunner pid=2122883) step:84 - global_seqlen/min:13107 - global_seqlen/max:21342 - global_seqlen/minmax_diff:8235 - global_seqlen/balanced_min:17248 - global_seqlen/balanced_max:17255 - global_seqlen/mean:17251.75 - actor/entropy:0.2078612893819809 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.41484442353248596 - training/rollout_probs_diff_mean:0.00501943938434124 - training/rollout_probs_diff_std:0.013952094130218029 - training/rollout_actor_probs_pearson_corr:0.9979199767112732 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6796875 - self_distillation/correct_sample_fraction:0.48046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6796875 - rollout_corr/rollout_is_mean:0.999704897403717 - rollout_corr/rollout_is_ratio_fraction_high:1.8811844711308368e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.64355323149357e-05 - rollout_corr/rollout_is_max:3.0805070400238037 - rollout_corr/rollout_is_min:0.05248083919286728 - rollout_corr/rollout_is_std:0.040534697473049164 - rollout_corr/rollout_is_eff_sample_size:0.9983585642197468 - rollout_corr/rollout_is_seq_mean:0.9996325969696045 - rollout_corr/rollout_is_seq_std:0.0031430358067154884 - rollout_corr/rollout_is_seq_max:1.0117285251617432 - rollout_corr/rollout_is_seq_min:0.9894457459449768 - rollout_corr/rollout_is_seq_max_deviation:0.011728525161743164 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.242810423951596) - rollout_corr/training_log_ppl:np.float64(0.21454886808351148) - rollout_corr/kl:np.float64(0.001670566343833002) - rollout_corr/k3_kl:np.float64(0.0014176062694559732) - rollout_corr/rollout_ppl:np.float64(1.240722066257149) - rollout_corr/rollout_log_ppl:np.float64(0.2128783009829931) - rollout_corr/log_ppl_diff:np.float64(0.001670567100518383) - rollout_corr/log_ppl_abs_diff:np.float64(0.003282473553554155) - rollout_corr/log_ppl_diff_max:np.float64(0.01952880620956421) - rollout_corr/log_ppl_diff_min:np.float64(-0.013135656714439392) - rollout_corr/ppl_ratio:np.float64(1.0016799475997686) - rollout_corr/chi2_token:np.float64(0.002261385088786483) - rollout_corr/chi2_seq:np.float64(1.0813027566764504) - actor/pg_loss:np.float64(0.0058089441154152155) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014233192905521719) - actor/ppo_kl:np.float64(0.0003872091626533347) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48046875) - self_distillation/token_reweight_w_mean:np.float64(483881.97194674145) - self_distillation/token_reweight_w_std:np.float64(5088252.776621332) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999582192394882) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1075905231264187) - self_distillation/empty_target_batch:np.float64(0.3203125) - actor/grad_norm:np.float64(0.6635904759168625) - perf/mfu/actor:np.float64(0.02916947468869229) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.59191131591797) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:5 - critic/score/mean:0.48046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.013916061259806156 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.013916061259806156 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:207.6484375 - response_length/max:750.0 - response_length/min:100.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:207.6484375 - response_length_non_aborted/max:750.0 - response_length_non_aborted/min:100.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:331.46875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.707920253276825e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1966691762208939) - timing_s/agent_loop/generate_sequences/max:np.float64(4.983985688537359) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.2186228269565618) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.983985688537359) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:425 - timing_s/agent_loop/slowest/response_length:750 - timing_s/gen:10.676102293655276 - timing_s/reward:0.05822275020182133 - timing_s/old_log_prob:2.3356290757656097 - timing_s/adv:0.47087168879806995 - timing_s/update_actor:16.726612301543355 - timing_s/step:30.356266921386123 - timing_s/stop_profile:0.00011728145182132721 - timing_per_token_ms/gen:0.20083717020307906 - timing_per_token_ms/update_actor:0.12119504036940712 - timing_per_token_ms/adv:0.0034117675655953015 - perf/total_num_tokens:138014 - perf/time_per_step:30.356266921386123 - perf/throughput:568.3093393755233 (TaskRunner pid=2122883) Training Progress: 84%|████████▍ | 84/100 [1:05:16<09:34, 35.92s/it] (TaskRunner pid=2122883) step:85 - global_seqlen/min:12069 - global_seqlen/max:20047 - global_seqlen/minmax_diff:7978 - global_seqlen/balanced_min:16063 - global_seqlen/balanced_max:16314 - global_seqlen/mean:16095.0 - actor/entropy:0.2045360505580902 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7543164491653442 - training/rollout_probs_diff_mean:0.005047583021223545 - training/rollout_probs_diff_std:0.01423710584640503 - training/rollout_actor_probs_pearson_corr:0.9978507161140442 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.68359375 - self_distillation/correct_sample_fraction:0.48828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.68359375 - rollout_corr/rollout_is_mean:0.9999943375587463 - rollout_corr/rollout_is_ratio_fraction_high:3.8331800169544294e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.9165900084772147e-05 - rollout_corr/rollout_is_max:4.387180328369141 - rollout_corr/rollout_is_min:0.49413782358169556 - rollout_corr/rollout_is_std:0.04079852253198624 - rollout_corr/rollout_is_eff_sample_size:0.9983380090836461 - rollout_corr/rollout_is_seq_mean:1.0000009536743164 - rollout_corr/rollout_is_seq_std:0.0031892997212707996 - rollout_corr/rollout_is_seq_max:1.0172944068908691 - rollout_corr/rollout_is_seq_min:0.9912353754043579 - rollout_corr/rollout_is_seq_max_deviation:0.01729440689086914 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.241943544242531) - rollout_corr/training_log_ppl:np.float64(0.2141833759524161) - rollout_corr/kl:np.float64(0.0017040160140098237) - rollout_corr/k3_kl:np.float64(0.0013909810037660009) - rollout_corr/rollout_ppl:np.float64(1.2398182358592749) - rollout_corr/rollout_log_ppl:np.float64(0.21247935874271207) - rollout_corr/log_ppl_diff:np.float64(0.001704017209704034) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031017301516840234) - rollout_corr/log_ppl_diff_max:np.float64(0.01839999109506607) - rollout_corr/log_ppl_diff_min:np.float64(-0.006871715188026428) - rollout_corr/ppl_ratio:np.float64(1.0017130349297076) - rollout_corr/chi2_token:np.float64(0.002452671527862549) - rollout_corr/chi2_seq:np.float64(0.1490656854584813) - actor/pg_loss:np.float64(0.0054126320173963904) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001665425015744404) - actor/ppo_kl:np.float64(0.0007683562842331071) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.68359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48828125) - self_distillation/token_reweight_w_mean:np.float64(373973.35678189015) - self_distillation/token_reweight_w_std:np.float64(3960859.6874851873) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998681743163615) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10099843548960052) - self_distillation/empty_target_batch:np.float64(0.31640625) - actor/grad_norm:np.float64(0.6237643733620644) - perf/mfu/actor:np.float64(0.02787586130815002) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.19374084472656) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:6 - critic/score/mean:0.48828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011717552319169044 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011717552319169044 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:203.8125 - response_length/max:1094.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:203.8125 - response_length_non_aborted/max:1094.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.15625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002668313682079315 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2239922620356083) - timing_s/agent_loop/generate_sequences/max:np.float64(6.708017874509096) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.2544095085613662) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.708017874509096) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:1094 - timing_s/gen:12.766073238104582 - timing_s/reward:0.051905008032917976 - timing_s/old_log_prob:2.5455926675349474 - timing_s/adv:0.4925172030925751 - timing_s/update_actor:16.307795433327556 - timing_s/step:32.2666488904506 - timing_s/stop_profile:0.00014187954366207123 - timing_per_token_ms/gen:0.24467328346566586 - timing_per_token_ms/update_actor:0.1266526517033827 - timing_per_token_ms/adv:0.0038250792411663177 - perf/total_num_tokens:128760 - perf/time_per_step:32.2666488904506 - perf/throughput:498.81225827462237 (TaskRunner pid=2122883) Training Progress: 85%|████████▌ | 85/100 [1:05:49<08:44, 34.97s/it] (TaskRunner pid=2122883) step:86 - global_seqlen/min:11119 - global_seqlen/max:22626 - global_seqlen/minmax_diff:11507 - global_seqlen/balanced_min:16850 - global_seqlen/balanced_max:16852 - global_seqlen/mean:16851.25 - actor/entropy:0.22577793896198273 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311284065246582 - training/rollout_probs_diff_mean:0.005342109594494104 - training/rollout_probs_diff_std:0.014018124900758266 - training/rollout_actor_probs_pearson_corr:0.9979961514472961 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.56640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:1.0000851154327393 - rollout_corr/rollout_is_ratio_fraction_high:1.8723085304372944e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.744617060874589e-05 - rollout_corr/rollout_is_max:2.115358352661133 - rollout_corr/rollout_is_min:0.21797996759414673 - rollout_corr/rollout_is_std:0.04185427725315094 - rollout_corr/rollout_is_eff_sample_size:0.9982517580297928 - rollout_corr/rollout_is_seq_mean:1.0001436471939087 - rollout_corr/rollout_is_seq_std:0.003316018031910062 - rollout_corr/rollout_is_seq_max:1.0115185976028442 - rollout_corr/rollout_is_seq_min:0.9917247891426086 - rollout_corr/rollout_is_seq_max_deviation:0.011518597602844238 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2645139200612903) - rollout_corr/training_log_ppl:np.float64(0.23199850719538517) - rollout_corr/kl:np.float64(0.001092571028761924) - rollout_corr/k3_kl:np.float64(0.001342841520084903) - rollout_corr/rollout_ppl:np.float64(1.2630862044170499) - rollout_corr/rollout_log_ppl:np.float64(0.2309059344988782) - rollout_corr/log_ppl_diff:np.float64(0.001092572696506977) - rollout_corr/log_ppl_abs_diff:np.float64(0.003120153967756778) - rollout_corr/log_ppl_diff_max:np.float64(0.03792935609817505) - rollout_corr/log_ppl_diff_min:np.float64(-0.010497242212295532) - rollout_corr/ppl_ratio:np.float64(1.0011034170165658) - rollout_corr/chi2_token:np.float64(0.002893829019740224) - rollout_corr/chi2_seq:np.float64(1.1480373400263488) - actor/pg_loss:np.float64(0.005574887851253152) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017103734035117668) - actor/ppo_kl:np.float64(0.0003303216700523137) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.56640625) - self_distillation/token_reweight_w_mean:np.float64(513197.1782494064) - self_distillation/token_reweight_w_std:np.float64(5438486.467104235) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009826868772507) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10202174982987344) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.5823733061552048) - perf/mfu/actor:np.float64(0.02872267604930534) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.13408279418945) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:6 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0024995319545269012 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0024995319545269012 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:208.6328125 - response_length/max:542.0 - response_length/min:106.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:208.6328125 - response_length_non_aborted/max:542.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:317.96875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010875426232814789 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3264490310102701) - timing_s/agent_loop/generate_sequences/max:np.float64(4.1357088424265385) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3096187077389914) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.1357088424265385) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:542 - timing_s/gen:9.943797947838902 - timing_s/reward:0.04411541484296322 - timing_s/old_log_prob:2.2926547210663557 - timing_s/adv:0.47092016600072384 - timing_s/update_actor:16.34498623199761 - timing_s/step:29.13274565897882 - timing_s/stop_profile:3.097020089626312e-05 - timing_per_token_ms/gen:0.18617857981349753 - timing_per_token_ms/update_actor:0.12124461265482983 - timing_per_token_ms/adv:0.003493213901051286 - perf/total_num_tokens:134810 - perf/time_per_step:29.13274565897882 - perf/throughput:578.4298602423827 (TaskRunner pid=2122883) Training Progress: 86%|████████▌ | 86/100 [1:06:18<07:45, 33.22s/it] (TaskRunner pid=2122883) step:87 - global_seqlen/min:13699 - global_seqlen/max:20034 - global_seqlen/minmax_diff:6335 - global_seqlen/balanced_min:16196 - global_seqlen/balanced_max:16205 - global_seqlen/mean:16202.875 - actor/entropy:0.22808320820331573 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4770777225494385 - training/rollout_probs_diff_mean:0.0052826725877821445 - training/rollout_probs_diff_std:0.014059673063457012 - training/rollout_actor_probs_pearson_corr:0.998014509677887 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.69921875 - self_distillation/correct_sample_fraction:0.42578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.69921875 - rollout_corr/rollout_is_mean:0.9995889067649841 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9377215721760876e-05 - rollout_corr/rollout_is_max:1.8165900707244873 - rollout_corr/rollout_is_min:0.13343973457813263 - rollout_corr/rollout_is_std:0.04224056005477905 - rollout_corr/rollout_is_eff_sample_size:0.998217546864214 - rollout_corr/rollout_is_seq_mean:0.9996139407157898 - rollout_corr/rollout_is_seq_std:0.0032783818896859884 - rollout_corr/rollout_is_seq_max:1.0129916667938232 - rollout_corr/rollout_is_seq_min:0.9896868467330933 - rollout_corr/rollout_is_seq_max_deviation:0.012991666793823242 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2670027003623545) - rollout_corr/training_log_ppl:np.float64(0.23412725170783233) - rollout_corr/kl:np.float64(0.0013061097060056426) - rollout_corr/k3_kl:np.float64(0.0011628404061525544) - rollout_corr/rollout_ppl:np.float64(1.2652938198298216) - rollout_corr/rollout_log_ppl:np.float64(0.2328211401036242) - rollout_corr/log_ppl_diff:np.float64(0.0013061116042081267) - rollout_corr/log_ppl_abs_diff:np.float64(0.00299576812540181) - rollout_corr/log_ppl_diff_max:np.float64(0.020071491599082947) - rollout_corr/log_ppl_diff_min:np.float64(-0.008261770009994507) - rollout_corr/ppl_ratio:np.float64(1.0013143399264663) - rollout_corr/chi2_token:np.float64(0.0019626670982688665) - rollout_corr/chi2_seq:np.float64(0.6271165478974581) - actor/pg_loss:np.float64(0.005690383608452976) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016033555998546944) - actor/ppo_kl:np.float64(-2.0786709365339107e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.69921875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.42578125) - self_distillation/token_reweight_w_mean:np.float64(740095.8522027656) - self_distillation/token_reweight_w_std:np.float64(7703203.598722717) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0025711173657328) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11661678130622022) - self_distillation/empty_target_batch:np.float64(0.30078125) - actor/grad_norm:np.float64(0.6615225672721863) - perf/mfu/actor:np.float64(0.027610136886118748) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.15999221801758) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:6 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002850872930139303 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002850872930139303 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:201.58984375 - response_length/max:720.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:201.58984375 - response_length_non_aborted/max:720.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:304.75 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.329904913902283e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2359547521919012) - timing_s/agent_loop/generate_sequences/max:np.float64(4.947052989155054) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.24328743157821) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.947052989155054) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:720 - timing_s/gen:10.758620804175735 - timing_s/reward:0.0536897424608469 - timing_s/old_log_prob:2.2264872901141644 - timing_s/adv:0.4786405097693205 - timing_s/update_actor:16.508141493424773 - timing_s/step:30.114757101982832 - timing_s/stop_profile:0.00012442097067832947 - timing_per_token_ms/gen:0.20847212207986773 - timing_per_token_ms/update_actor:0.12735503339241316 - timing_per_token_ms/adv:0.0036925584947834914 - perf/total_num_tokens:129623 - perf/time_per_step:30.114757101982832 - perf/throughput:538.0377117148709 (TaskRunner pid=2122883) Training Progress: 87%|████████▋ | 87/100 [1:06:48<06:59, 32.30s/it] (TaskRunner pid=2122883) step:88 - global_seqlen/min:12296 - global_seqlen/max:20557 - global_seqlen/minmax_diff:8261 - global_seqlen/balanced_min:17022 - global_seqlen/balanced_max:17025 - global_seqlen/mean:17023.375 - actor/entropy:0.22276757657527924 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6303352117538452 - training/rollout_probs_diff_mean:0.005168667994439602 - training/rollout_probs_diff_std:0.01409972459077835 - training/rollout_actor_probs_pearson_corr:0.9979792833328247 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.703125 - self_distillation/correct_sample_fraction:0.4921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.703125 - rollout_corr/rollout_is_mean:1.0003288984298706 - rollout_corr/rollout_is_ratio_fraction_high:5.5634885939070955e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.27248111111112e-05 - rollout_corr/rollout_is_max:3.249321699142456 - rollout_corr/rollout_is_min:0.06875494867563248 - rollout_corr/rollout_is_std:0.04171448200941086 - rollout_corr/rollout_is_eff_sample_size:0.9982639938423429 - rollout_corr/rollout_is_seq_mean:1.000346064567566 - rollout_corr/rollout_is_seq_std:0.0032712207175791264 - rollout_corr/rollout_is_seq_max:1.0110112428665161 - rollout_corr/rollout_is_seq_min:0.9913240671157837 - rollout_corr/rollout_is_seq_max_deviation:0.011011242866516113 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2595614441670477) - rollout_corr/training_log_ppl:np.float64(0.22807635995559394) - rollout_corr/kl:np.float64(0.001190670725655707) - rollout_corr/k3_kl:np.float64(0.001538011453078525) - rollout_corr/rollout_ppl:np.float64(1.257966611534357) - rollout_corr/rollout_log_ppl:np.float64(0.22688568860758096) - rollout_corr/log_ppl_diff:np.float64(0.0011906713480129838) - rollout_corr/log_ppl_abs_diff:np.float64(0.0034372524241916835) - rollout_corr/log_ppl_diff_max:np.float64(0.021827280521392822) - rollout_corr/log_ppl_diff_min:np.float64(-0.014090389013290405) - rollout_corr/ppl_ratio:np.float64(1.0012007460463792) - rollout_corr/chi2_token:np.float64(0.004329530056566) - rollout_corr/chi2_seq:np.float64(1.7654117406345904) - actor/pg_loss:np.float64(0.0044011540012434125) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0018381910886091646) - actor/ppo_kl:np.float64(0.0006000098311353952) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4921875) - self_distillation/token_reweight_w_mean:np.float64(616948.9494683831) - self_distillation/token_reweight_w_std:np.float64(6217318.756260075) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004455572925508) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10639075534709264) - self_distillation/empty_target_batch:np.float64(0.296875) - actor/grad_norm:np.float64(0.5419183969497681) - perf/mfu/actor:np.float64(0.028824236807819127) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.21399307250977) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:6 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0034400904551148415 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0034400904551148415 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:210.63671875 - response_length/max:547.0 - response_length/min:106.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:210.63671875 - response_length_non_aborted/max:547.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:321.34375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016992166638374329 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9193975105881691) - timing_s/agent_loop/generate_sequences/max:np.float64(4.1702308394014835) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.1946431579781347) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.1702308394014835) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:539 - timing_s/gen:9.90405653603375 - timing_s/reward:0.054226161912083626 - timing_s/old_log_prob:2.2448495719581842 - timing_s/adv:0.4909304156899452 - timing_s/update_actor:16.746504409238696 - timing_s/step:29.53132553398609 - timing_s/stop_profile:0.00011452659964561462 - timing_per_token_ms/gen:0.18367035469157408 - timing_per_token_ms/update_actor:0.12296698223206838 - timing_per_token_ms/adv:0.003604825832788337 - perf/total_num_tokens:136187 - perf/time_per_step:29.53132553398609 - perf/throughput:576.4514356258295 (TaskRunner pid=2122883) Training Progress: 88%|████████▊ | 88/100 [1:07:18<06:17, 31.49s/it] (TaskRunner pid=2122883) step:89 - global_seqlen/min:15072 - global_seqlen/max:22879 - global_seqlen/minmax_diff:7807 - global_seqlen/balanced_min:18089 - global_seqlen/balanced_max:18166 - global_seqlen/mean:18099.375 - actor/entropy:0.20623931288719177 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7721544504165649 - training/rollout_probs_diff_mean:0.005006530787795782 - training/rollout_probs_diff_std:0.014681383036077023 - training/rollout_actor_probs_pearson_corr:0.997765302658081 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.0000381469726562 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00016286940081045032 - rollout_corr/rollout_is_max:1.7593015432357788 - rollout_corr/rollout_is_min:0.009484831243753433 - rollout_corr/rollout_is_std:0.0410359650850296 - rollout_corr/rollout_is_eff_sample_size:0.9983191181153939 - rollout_corr/rollout_is_seq_mean:1.000022530555725 - rollout_corr/rollout_is_seq_std:0.002867963397875428 - rollout_corr/rollout_is_seq_max:1.0078660249710083 - rollout_corr/rollout_is_seq_min:0.9917351603507996 - rollout_corr/rollout_is_seq_max_deviation:0.00826483964920044 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2432606685906649) - rollout_corr/training_log_ppl:np.float64(0.2151060424221214) - rollout_corr/kl:np.float64(0.001223206437249047) - rollout_corr/k3_kl:np.float64(0.0014821982162231961) - rollout_corr/rollout_ppl:np.float64(1.241701312828809) - rollout_corr/rollout_log_ppl:np.float64(0.21388283489795867) - rollout_corr/log_ppl_diff:np.float64(0.0012232075241627172) - rollout_corr/log_ppl_abs_diff:np.float64(0.003019628828042187) - rollout_corr/log_ppl_diff_max:np.float64(0.012458458542823792) - rollout_corr/log_ppl_diff_min:np.float64(-0.009425677359104156) - rollout_corr/ppl_ratio:np.float64(1.001230750232935) - rollout_corr/chi2_token:np.float64(0.0032338621094822884) - rollout_corr/chi2_seq:np.float64(0.811216983012855) - actor/pg_loss:np.float64(0.005030228639952838) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001661099498051044) - actor/ppo_kl:np.float64(0.00023485885652974048) - actor/pg_clipfrac_lower:np.float64(3.753240162041038e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(451622.0916177754) - self_distillation/token_reweight_w_std:np.float64(4642653.067052306) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9989522572141141) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08174082619370893) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.5944810956716537) - perf/mfu/actor:np.float64(0.03101760938380116) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.39448928833008) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:6 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0021557575091719627 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0021557575091719627 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:215.85546875 - response_length/max:907.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:215.85546875 - response_length_non_aborted/max:907.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:349.75 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000102972611784935 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3130767196416855) - timing_s/agent_loop/generate_sequences/max:np.float64(5.7279154025018215) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.337402616853069) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.7279154025018215) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:907 - timing_s/gen:11.618766691535711 - timing_s/reward:0.05591783672571182 - timing_s/old_log_prob:2.1951356530189514 - timing_s/adv:0.5187619235366583 - timing_s/update_actor:16.619402937591076 - timing_s/step:31.09566144272685 - timing_s/stop_profile:3.01264226436615e-05 - timing_per_token_ms/gen:0.21026016923099786 - timing_per_token_ms/update_actor:0.11477884552360977 - timing_per_token_ms/adv:0.0035827336823554563 - perf/total_num_tokens:144795 - perf/time_per_step:31.09566144272685 - perf/throughput:582.0546713031367 (TaskRunner pid=2122883) Training Progress: 89%|████████▉ | 89/100 [1:07:49<05:45, 31.37s/it] (TaskRunner pid=2122883) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2122883) validation generation end (TaskRunner pid=2122883) [prompt] system (TaskRunner pid=2122883) (TaskRunner pid=2122883) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2122883) (TaskRunner pid=2122883) A (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) user (TaskRunner pid=2122883) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2122883) (TaskRunner pid=2122883) A: -0.01 (TaskRunner pid=2122883) B: 1.69 (TaskRunner pid=2122883) C: 2.49 (TaskRunner pid=2122883) D: 0.45 (TaskRunner pid=2122883) Please reason step by step. (TaskRunner pid=2122883) assistant (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) [response] (TaskRunner pid=2122883) The question asks for the folding stability score of a specific protein sequence. Without access to the actual computational tool or database that calculates this score, it is not possible to determine the exact value. However, the options provided are numerical scores that are typically used in protein folding stability assessments. Among the options, 1.69 is a reasonable value for a folding stability score, as it falls within the typical range of scores used to indicate the stability of a protein structure. Therefore, the most plausible answer is 1.69. (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) B (TaskRunner pid=2122883) (TaskRunner pid=2122883) [ground_truth] B (TaskRunner pid=2122883) [score] 1.0 (TaskRunner pid=2122883) [acc] 1.0 (TaskRunner pid=2122883) [pred] B (TaskRunner pid=2122883) [incorrect_format] 1 (TaskRunner pid=2122883) [feedback] (TaskRunner pid=2122883) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (WorkerDict pid=2123259) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80/actor (TaskRunner pid=2122883) step:90 - global_seqlen/min:13820 - global_seqlen/max:20696 - global_seqlen/minmax_diff:6876 - global_seqlen/balanced_min:16907 - global_seqlen/balanced_max:16910 - global_seqlen/mean:16908.375 - actor/entropy:0.19698114693164825 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6937196254730225 - training/rollout_probs_diff_mean:0.004946956876665354 - training/rollout_probs_diff_std:0.014074870385229588 - training/rollout_actor_probs_pearson_corr:0.9977890253067017 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.6875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:0.9999498724937439 - rollout_corr/rollout_is_ratio_fraction_high:1.8322065443499014e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010993239266099408 - rollout_corr/rollout_is_max:3.0224504470825195 - rollout_corr/rollout_is_min:0.00395188108086586 - rollout_corr/rollout_is_std:0.04045668616890907 - rollout_corr/rollout_is_eff_sample_size:0.9983656933663717 - rollout_corr/rollout_is_seq_mean:0.9999589920043945 - rollout_corr/rollout_is_seq_std:0.0028546203393489122 - rollout_corr/rollout_is_seq_max:1.0088039636611938 - rollout_corr/rollout_is_seq_min:0.99063640832901 - rollout_corr/rollout_is_seq_max_deviation:0.00936359167098999 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2240748791955411) - rollout_corr/training_log_ppl:np.float64(0.199976087838877) - rollout_corr/kl:np.float64(0.0012380354552803396) - rollout_corr/k3_kl:np.float64(0.0013727707471673511) - rollout_corr/rollout_ppl:np.float64(1.2225207276642323) - rollout_corr/rollout_log_ppl:np.float64(0.19873805253882892) - rollout_corr/log_ppl_diff:np.float64(0.0012380353000480682) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029817646427545696) - rollout_corr/log_ppl_diff_max:np.float64(0.014439418911933899) - rollout_corr/log_ppl_diff_min:np.float64(-0.009225308895111084) - rollout_corr/ppl_ratio:np.float64(1.0012455948162824) - rollout_corr/chi2_token:np.float64(0.0027791301254183054) - rollout_corr/chi2_seq:np.float64(0.5106464147102088) - actor/pg_loss:np.float64(0.007799276732839644) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020063032634425326) - actor/ppo_kl:np.float64(0.00030233424429404465) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_w_mean:np.float64(428857.22522661276) - self_distillation/token_reweight_w_std:np.float64(5102986.168928434) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9979278270620853) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11433671996928751) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.6825058907270432) - perf/mfu/actor:np.float64(0.02891552234793523) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.10916900634766) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.485) - val-aux/sciknoweval/reward/std@16:np.float64(0.1826775697855999) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.55618) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.14939994447654767) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.41332) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15483870511911316) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.4847600000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.18236506835118788) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.61418) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11578798271076449) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.35528) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1237239235875391) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.48751999999999995) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13784588373033047) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.66276) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08530648056930885) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.3057) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09776880099258937) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.49102) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.08840624761811913) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.6975800000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.05027353975557356) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.26202) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06976567401063935) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.49556) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.058256449010532264) - val-aux/sciknoweval/score/mean@16:np.float64(0.485) - val-aux/sciknoweval/score/std@16:np.float64(0.1826775697855999) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.55618) - val-aux/sciknoweval/score/best@2/std:np.float64(0.14939994447654767) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.41332) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15483870511911316) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.4847600000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.18236506835118788) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.61418) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11578798271076449) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.35528) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1237239235875391) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.48751999999999995) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13784588373033047) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.66276) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08530648056930885) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.3057) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09776880099258937) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.49102) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.08840624761811913) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.6975800000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.05027353975557356) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.26202) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06976567401063935) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.49556) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.058256449010532264) - val-core/sciknoweval/acc/mean@16:np.float64(0.485) - val-aux/sciknoweval/acc/std@16:np.float64(0.1826775697855999) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.55618) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.14939994447654767) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.41332) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15483870511911316) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.4847600000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.18236506835118788) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.61418) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11578798271076449) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.35528) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1237239235875391) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.48751999999999995) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13784588373033047) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.66276) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08530648056930885) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.3057) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09776880099258937) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.49102) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.08840624761811913) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.6975800000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.05027353975557356) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.26202) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06976567401063935) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.49556) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.058256449010532264) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:6 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0017451767344027758 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0017451767344027758 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:213.19921875 - response_length/max:599.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:213.19921875 - response_length_non_aborted/max:599.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.1875 - prompt_length/max:499.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.5389716029167175e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3019966781139374) - timing_s/agent_loop/generate_sequences/max:np.float64(4.4793809447437525) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3214208512436016) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.4793809447437525) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:421 - timing_s/agent_loop/slowest/response_length:599 - timing_s/gen:10.459255415946245 - timing_s/reward:0.0545695535838604 - timing_s/old_log_prob:2.228510320186615 - timing_s/adv:0.479594849050045 - timing_s/update_actor:16.379232734441757 - timing_s/step:29.687829732894897 - timing_s/testing:21.612145015969872 - timing_s/save_checkpoint:6.945257265120745 - timing_s/stop_profile:0.00011808052659034729 - timing_per_token_ms/gen:0.19163516033540823 - timing_per_token_ms/update_actor:0.12108816440404353 - timing_per_token_ms/adv:0.0035455421429472452 - perf/total_num_tokens:135267 - perf/time_per_step:29.687829732894897 - perf/throughput:569.5389374072391 (TaskRunner pid=2122883) Training Progress: 90%|█████████ | 90/100 [1:08:47<06:34, 39.46s/it] (TaskRunner pid=2122883) step:91 - global_seqlen/min:12564 - global_seqlen/max:21781 - global_seqlen/minmax_diff:9217 - global_seqlen/balanced_min:17651 - global_seqlen/balanced_max:17661 - global_seqlen/mean:17654.0 - actor/entropy:0.1927318572998047 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102362394332886 - training/rollout_probs_diff_mean:0.0049341288395226 - training/rollout_probs_diff_std:0.013917961157858372 - training/rollout_actor_probs_pearson_corr:0.9978261590003967 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:1.0002244710922241 - rollout_corr/rollout_is_ratio_fraction_high:1.766285095072817e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.065140380291268e-05 - rollout_corr/rollout_is_max:2.8016161918640137 - rollout_corr/rollout_is_min:0.35093075037002563 - rollout_corr/rollout_is_std:0.04169304296374321 - rollout_corr/rollout_is_eff_sample_size:0.9982654193934108 - rollout_corr/rollout_is_seq_mean:1.0002824068069458 - rollout_corr/rollout_is_seq_std:0.003071394981816411 - rollout_corr/rollout_is_seq_max:1.009526014328003 - rollout_corr/rollout_is_seq_min:0.9910033941268921 - rollout_corr/rollout_is_seq_max_deviation:0.00952601432800293 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2255855221301317) - rollout_corr/training_log_ppl:np.float64(0.20155303589854157) - rollout_corr/kl:np.float64(0.0010046164143293623) - rollout_corr/k3_kl:np.float64(0.0012420805354622644) - rollout_corr/rollout_ppl:np.float64(1.2243025470525026) - rollout_corr/rollout_log_ppl:np.float64(0.20054841847741045) - rollout_corr/log_ppl_diff:np.float64(0.0010046174211311154) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029608946133521385) - rollout_corr/log_ppl_diff_max:np.float64(0.015554353594779968) - rollout_corr/log_ppl_diff_min:np.float64(-0.006916940212249756) - rollout_corr/ppl_ratio:np.float64(1.0010123364627361) - rollout_corr/chi2_token:np.float64(0.003038359573110938) - rollout_corr/chi2_seq:np.float64(0.7696262006647885) - actor/pg_loss:np.float64(0.005795407225377858) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001670339756856265) - actor/ppo_kl:np.float64(0.0003866044301399363) - actor/pg_clipfrac_lower:np.float64(3.50374812114751e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.578125) - self_distillation/token_reweight_w_mean:np.float64(411195.32281198865) - self_distillation/token_reweight_w_std:np.float64(4754109.018892683) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010015049483627) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08646299615793396) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.5738260895013809) - perf/mfu/actor:np.float64(0.029971853940342928) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(145.91704177856445) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:6 - critic/score/mean:0.578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0023601986467838287 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0023601986467838287 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:221.15625 - response_length/max:894.0 - response_length/min:132.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:221.15625 - response_length_non_aborted/max:894.0 - response_length_non_aborted/min:132.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.53125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.253699004650116e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5865748934447765) - timing_s/agent_loop/generate_sequences/max:np.float64(6.002601191401482) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.461126289665117) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.002601191401482) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:415 - timing_s/agent_loop/slowest/response_length:894 - timing_s/gen:11.83268902078271 - timing_s/reward:0.051725566387176514 - timing_s/old_log_prob:2.437912594527006 - timing_s/adv:0.4970137644559145 - timing_s/update_actor:16.579867849126458 - timing_s/step:31.503263341262937 - timing_s/stop_profile:2.8558075428009033e-05 - timing_per_token_ms/gen:0.2089990289102499 - timing_per_token_ms/update_actor:0.11739455540618599 - timing_per_token_ms/adv:0.0035191299737730437 - perf/total_num_tokens:141232 - perf/time_per_step:31.503263341262937 - perf/throughput:560.3863894594314 (TaskRunner pid=2122883) Training Progress: 91%|█████████ | 91/100 [1:09:19<05:33, 37.08s/it] (TaskRunner pid=2122883) step:92 - global_seqlen/min:15984 - global_seqlen/max:22004 - global_seqlen/minmax_diff:6020 - global_seqlen/balanced_min:18842 - global_seqlen/balanced_max:18845 - global_seqlen/mean:18843.5 - actor/entropy:0.20552825927734375 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8458645343780518 - training/rollout_probs_diff_mean:0.0050735012628138065 - training/rollout_probs_diff_std:0.01476234756410122 - training/rollout_actor_probs_pearson_corr:0.9976810216903687 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73046875 - self_distillation/correct_sample_fraction:0.515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73046875 - rollout_corr/rollout_is_mean:1.0001269578933716 - rollout_corr/rollout_is_ratio_fraction_high:6.938180740689859e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.938180740689859e-05 - rollout_corr/rollout_is_max:3.0634891986846924 - rollout_corr/rollout_is_min:0.11200299113988876 - rollout_corr/rollout_is_std:0.04258697107434273 - rollout_corr/rollout_is_eff_sample_size:0.9981899895749777 - rollout_corr/rollout_is_seq_mean:1.0000636577606201 - rollout_corr/rollout_is_seq_std:0.0031039409805089235 - rollout_corr/rollout_is_seq_max:1.0127332210540771 - rollout_corr/rollout_is_seq_min:0.991115391254425 - rollout_corr/rollout_is_seq_max_deviation:0.012733221054077148 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.235127116087824) - rollout_corr/training_log_ppl:np.float64(0.20900695942691527) - rollout_corr/kl:np.float64(0.0012782263143336081) - rollout_corr/k3_kl:np.float64(0.0013150399036589988) - rollout_corr/rollout_ppl:np.float64(1.2335282661952078) - rollout_corr/rollout_log_ppl:np.float64(0.20772873278474435) - rollout_corr/log_ppl_diff:np.float64(0.001278226642170921) - rollout_corr/log_ppl_abs_diff:np.float64(0.002990237466292456) - rollout_corr/log_ppl_diff_max:np.float64(0.012402921915054321) - rollout_corr/log_ppl_diff_min:np.float64(-0.011149108409881592) - rollout_corr/ppl_ratio:np.float64(1.0012855627574027) - rollout_corr/chi2_token:np.float64(0.002650786191225052) - rollout_corr/chi2_seq:np.float64(0.7602505653630942) - actor/pg_loss:np.float64(0.004597345599904656) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014476663550340163) - actor/ppo_kl:np.float64(0.0003580096706059521) - actor/pg_clipfrac_lower:np.float64(2.2321428332361393e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_w_mean:np.float64(553014.0691977837) - self_distillation/token_reweight_w_std:np.float64(5880688.194196964) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0030879531987011) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08305959589779377) - self_distillation/empty_target_batch:np.float64(0.26953125) - actor/grad_norm:np.float64(0.5153934583067894) - perf/mfu/actor:np.float64(0.03218116200910849) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(145.88633728027344) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:6 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007577794138342142 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007577794138342142 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:225.203125 - response_length/max:638.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:225.203125 - response_length_non_aborted/max:638.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:363.65625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.5453045964241028e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4698635023087263) - timing_s/agent_loop/generate_sequences/max:np.float64(4.719432448968291) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.4977113083878066) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.719432448968291) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:630 - timing_s/gen:10.855900479480624 - timing_s/reward:0.046163953840732574 - timing_s/old_log_prob:2.2303929030895233 - timing_s/adv:0.48601470328867435 - timing_s/update_actor:16.612354155629873 - timing_s/step:30.2675437964499 - timing_s/stop_profile:0.00011699646711349487 - timing_per_token_ms/gen:0.1883005009276456 - timing_per_token_ms/update_actor:0.11019949953319363 - timing_per_token_ms/adv:0.0032240209043481465 - perf/total_num_tokens:150748 - perf/time_per_step:30.2675437964499 - perf/throughput:622.5645571613964 (TaskRunner pid=2122883) Training Progress: 92%|█████████▏| 92/100 [1:09:49<04:40, 35.05s/it] (TaskRunner pid=2122883) step:93 - global_seqlen/min:14142 - global_seqlen/max:22580 - global_seqlen/minmax_diff:8438 - global_seqlen/balanced_min:17968 - global_seqlen/balanced_max:19159 - global_seqlen/mean:18118.25 - actor/entropy:0.2024667114019394 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3206155598163605 - training/rollout_probs_diff_mean:0.004839439410716295 - training/rollout_probs_diff_std:0.013173987157642841 - training/rollout_actor_probs_pearson_corr:0.9981030821800232 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.55078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:0.9996916651725769 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.3325748012866825e-05 - rollout_corr/rollout_is_max:1.922481656074524 - rollout_corr/rollout_is_min:0.0006340847467072308 - rollout_corr/rollout_is_std:0.0401468463242054 - rollout_corr/rollout_is_eff_sample_size:0.9983899332265034 - rollout_corr/rollout_is_seq_mean:0.9997001886367798 - rollout_corr/rollout_is_seq_std:0.0026396142784506083 - rollout_corr/rollout_is_seq_max:1.0092061758041382 - rollout_corr/rollout_is_seq_min:0.991854190826416 - rollout_corr/rollout_is_seq_max_deviation:0.009206175804138184 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2384646688587964) - rollout_corr/training_log_ppl:np.float64(0.21190124790882692) - rollout_corr/kl:np.float64(0.001526296815164585) - rollout_corr/k3_kl:np.float64(0.0011333852019390633) - rollout_corr/rollout_ppl:np.float64(1.2365091615356505) - rollout_corr/rollout_log_ppl:np.float64(0.21037495086784475) - rollout_corr/log_ppl_diff:np.float64(0.001526297040982172) - rollout_corr/log_ppl_abs_diff:np.float64(0.002726003556745127) - rollout_corr/log_ppl_diff_max:np.float64(0.024714887142181396) - rollout_corr/log_ppl_diff_min:np.float64(-0.008926495909690857) - rollout_corr/ppl_ratio:np.float64(1.0015337793156505) - rollout_corr/chi2_token:np.float64(0.0012352506164461374) - rollout_corr/chi2_seq:np.float64(0.21279096603393555) - actor/pg_loss:np.float64(0.003292814362794161) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011469970941107022) - actor/ppo_kl:np.float64(0.0002611581061984225) - actor/pg_clipfrac_lower:np.float64(2.1701389414374717e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.55078125) - self_distillation/token_reweight_w_mean:np.float64(562577.1132855881) - self_distillation/token_reweight_w_std:np.float64(5476452.781596074) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991126488894224) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08483730700390879) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.5270627364516258) - perf/mfu/actor:np.float64(0.030530903588503785) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.33957290649414) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:6 - critic/score/mean:0.55078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0026640654541552067 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0026640654541552067 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:219.7578125 - response_length/max:1916.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:219.7578125 - response_length_non_aborted/max:1916.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:346.4375 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001993868499994278 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4296222887933254) - timing_s/agent_loop/generate_sequences/max:np.float64(10.560190688818693) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3909345842766925) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.560190688818693) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1916 - timing_s/gen:16.355938605964184 - timing_s/reward:0.056257134303450584 - timing_s/old_log_prob:2.2599684596061707 - timing_s/adv:0.6293122116476297 - timing_s/update_actor:16.973596734926105 - timing_s/step:36.360969342291355 - timing_s/stop_profile:3.0001625418663025e-05 - timing_per_token_ms/gen:0.2907308934900669 - timing_per_token_ms/update_actor:0.11710289856171335 - timing_per_token_ms/adv:0.004341701127644983 - perf/total_num_tokens:144946 - perf/time_per_step:36.360969342291355 - perf/throughput:498.2884210110072 (TaskRunner pid=2122883) Training Progress: 93%|█████████▎| 93/100 [1:10:26<04:08, 35.45s/it] (TaskRunner pid=2122883) step:94 - global_seqlen/min:12367 - global_seqlen/max:21204 - global_seqlen/minmax_diff:8837 - global_seqlen/balanced_min:15481 - global_seqlen/balanced_max:15486 - global_seqlen/mean:15484.25 - actor/entropy:0.18188761174678802 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26072946190834045 - training/rollout_probs_diff_mean:0.004726748447865248 - training/rollout_probs_diff_std:0.013647111132740974 - training/rollout_actor_probs_pearson_corr:0.9978026747703552 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.68359375 - self_distillation/correct_sample_fraction:0.515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.68359375 - rollout_corr/rollout_is_mean:0.9999868273735046 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.8747504226630554e-05 - rollout_corr/rollout_is_max:1.9659937620162964 - rollout_corr/rollout_is_min:0.06315071135759354 - rollout_corr/rollout_is_std:0.03998841717839241 - rollout_corr/rollout_is_eff_sample_size:0.9984032418855266 - rollout_corr/rollout_is_seq_mean:1.0000756978988647 - rollout_corr/rollout_is_seq_std:0.003055985551327467 - rollout_corr/rollout_is_seq_max:1.0124971866607666 - rollout_corr/rollout_is_seq_min:0.9919678568840027 - rollout_corr/rollout_is_seq_max_deviation:0.012497186660766602 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.208876115269959) - rollout_corr/training_log_ppl:np.float64(0.1878806567110587) - rollout_corr/kl:np.float64(0.001109215704858002) - rollout_corr/k3_kl:np.float64(0.0013251136373924055) - rollout_corr/rollout_ppl:np.float64(1.2075008023530245) - rollout_corr/rollout_log_ppl:np.float64(0.1867714399995748) - rollout_corr/log_ppl_diff:np.float64(0.0011092167114838958) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029893830069340765) - rollout_corr/log_ppl_diff_max:np.float64(0.017061367630958557) - rollout_corr/log_ppl_diff_min:np.float64(-0.011037014424800873) - rollout_corr/ppl_ratio:np.float64(1.0011169675271958) - rollout_corr/chi2_token:np.float64(0.0031342231668531895) - rollout_corr/chi2_seq:np.float64(0.7604034042451531) - actor/pg_loss:np.float64(0.0038497489877045155) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0018158378306907252) - actor/ppo_kl:np.float64(0.0003219292054641443) - actor/pg_clipfrac_lower:np.float64(2.0559211407089606e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.68359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_w_mean:np.float64(568107.0682997338) - self_distillation/token_reweight_w_std:np.float64(5762513.378730269) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993454308714718) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07923695075442083) - self_distillation/empty_target_batch:np.float64(0.31640625) - actor/grad_norm:np.float64(0.571109913289547) - perf/mfu/actor:np.float64(0.02667527205188385) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.28662490844727) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:6 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0017134688096120954 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0017134688096120954 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:199.4765625 - response_length/max:551.0 - response_length/min:122.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:199.4765625 - response_length_non_aborted/max:551.0 - response_length_non_aborted/min:122.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:284.40625 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010585226118564606 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4458684269338846) - timing_s/agent_loop/generate_sequences/max:np.float64(4.342792140319943) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.248531175951939) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.342792140319943) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:551 - timing_s/gen:10.355708943679929 - timing_s/reward:0.052927035838365555 - timing_s/old_log_prob:2.2116340044885874 - timing_s/adv:0.4717257060110569 - timing_s/update_actor:16.339361615478992 - timing_s/step:29.518693948164582 - timing_s/stop_profile:0.00011059083044528961 - timing_per_token_ms/gen:0.20279068154309968 - timing_per_token_ms/update_actor:0.13190307583091684 - timing_per_token_ms/adv:0.0038081090948145447 - perf/total_num_tokens:123874 - perf/time_per_step:29.518693948164582 - perf/throughput:524.5574220590739 (TaskRunner pid=2122883) Training Progress: 94%|█████████▍| 94/100 [1:10:55<03:22, 33.68s/it] (TaskRunner pid=2122883) step:95 - global_seqlen/min:12550 - global_seqlen/max:19516 - global_seqlen/minmax_diff:6966 - global_seqlen/balanced_min:15669 - global_seqlen/balanced_max:15675 - global_seqlen/mean:15672.625 - actor/entropy:0.1723659634590149 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.42921364307403564 - training/rollout_probs_diff_mean:0.004754330962896347 - training/rollout_probs_diff_std:0.01394499558955431 - training/rollout_actor_probs_pearson_corr:0.997614324092865 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.60546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9999802708625793 - rollout_corr/rollout_is_ratio_fraction_high:7.880686462158337e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.9105146647198126e-05 - rollout_corr/rollout_is_max:3.042388916015625 - rollout_corr/rollout_is_min:0.31485360860824585 - rollout_corr/rollout_is_std:0.04127493128180504 - rollout_corr/rollout_is_eff_sample_size:0.9982989210156206 - rollout_corr/rollout_is_seq_mean:0.9999309778213501 - rollout_corr/rollout_is_seq_std:0.0030664117075502872 - rollout_corr/rollout_is_seq_max:1.0131924152374268 - rollout_corr/rollout_is_seq_min:0.9899721145629883 - rollout_corr/rollout_is_seq_max_deviation:0.013192415237426758 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.194142870604992) - rollout_corr/training_log_ppl:np.float64(0.17597143551392946) - rollout_corr/kl:np.float64(0.0014179566909646724) - rollout_corr/k3_kl:np.float64(0.001468020923283575) - rollout_corr/rollout_ppl:np.float64(1.1923948260955513) - rollout_corr/rollout_log_ppl:np.float64(0.1745534775982378) - rollout_corr/log_ppl_diff:np.float64(0.0014179579156916589) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032777310407254845) - rollout_corr/log_ppl_diff_max:np.float64(0.028223499655723572) - rollout_corr/log_ppl_diff_min:np.float64(-0.0074967145919799805) - rollout_corr/ppl_ratio:np.float64(1.0014282818883657) - rollout_corr/chi2_token:np.float64(0.0027583797927945852) - rollout_corr/chi2_seq:np.float64(0.5167441787198186) - actor/pg_loss:np.float64(0.0040054775308817625) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016601379911662661) - actor/ppo_kl:np.float64(0.0004547775269259091) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.60546875) - self_distillation/token_reweight_w_mean:np.float64(408989.465876037) - self_distillation/token_reweight_w_std:np.float64(4584876.381394251) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0018037902191281) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06804537688731216) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.555400162935257) - perf/mfu/actor:np.float64(0.026826703716015997) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.3315315246582) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:6 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007434935308992863 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007434935308992863 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:198.26953125 - response_length/max:567.0 - response_length/min:126.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:198.26953125 - response_length_non_aborted/max:567.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.5 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013350322842597961 - timing_s/agent_loop/generate_sequences/min:np.float64(1.496675157919526) - timing_s/agent_loop/generate_sequences/max:np.float64(4.148937383666635) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.279242074866488) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.148937383666635) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:567 - timing_s/gen:9.941075744107366 - timing_s/reward:0.05165614001452923 - timing_s/old_log_prob:2.20885256677866 - timing_s/adv:0.4986916873604059 - timing_s/update_actor:16.47932105138898 - timing_s/step:29.26573464088142 - timing_s/stop_profile:0.00011342950165271759 - timing_per_token_ms/gen:0.19585625123839798 - timing_per_token_ms/update_actor:0.1314339577080178 - timing_per_token_ms/adv:0.003977410352129955 - perf/total_num_tokens:125381 - perf/time_per_step:29.26573464088142 - perf/throughput:535.528159204548 (TaskRunner pid=2122883) Training Progress: 95%|█████████▌| 95/100 [1:11:24<02:41, 32.37s/it] (TaskRunner pid=2122883) step:96 - global_seqlen/min:11753 - global_seqlen/max:19731 - global_seqlen/minmax_diff:7978 - global_seqlen/balanced_min:15977 - global_seqlen/balanced_max:16380 - global_seqlen/mean:16066.125 - actor/entropy:0.18051546812057495 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5147483348846436 - training/rollout_probs_diff_mean:0.005008666310459375 - training/rollout_probs_diff_std:0.014469319954514503 - training/rollout_actor_probs_pearson_corr:0.9975249171257019 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.53515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9997128248214722 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.8940052036195993e-05 - rollout_corr/rollout_is_max:1.883395791053772 - rollout_corr/rollout_is_min:0.0843614861369133 - rollout_corr/rollout_is_std:0.0420609749853611 - rollout_corr/rollout_is_eff_sample_size:0.9982329891138849 - rollout_corr/rollout_is_seq_mean:0.9996994733810425 - rollout_corr/rollout_is_seq_std:0.0030340866651386023 - rollout_corr/rollout_is_seq_max:1.0078694820404053 - rollout_corr/rollout_is_seq_min:0.9882951378822327 - rollout_corr/rollout_is_seq_max_deviation:0.011704862117767334 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2039276850409806) - rollout_corr/training_log_ppl:np.float64(0.1833886240492575) - rollout_corr/kl:np.float64(0.0019757910764717224) - rollout_corr/k3_kl:np.float64(0.0021378641860110292) - rollout_corr/rollout_ppl:np.float64(1.2014858215115964) - rollout_corr/rollout_log_ppl:np.float64(0.18141283292789012) - rollout_corr/log_ppl_diff:np.float64(0.001975791121367365) - rollout_corr/log_ppl_abs_diff:np.float64(0.004001759923994541) - rollout_corr/log_ppl_diff_max:np.float64(0.03609415888786316) - rollout_corr/log_ppl_diff_min:np.float64(-0.013453051447868347) - rollout_corr/ppl_ratio:np.float64(1.001994721358642) - rollout_corr/chi2_token:np.float64(0.0038919271901249886) - rollout_corr/chi2_seq:np.float64(1.5032801111228764) - actor/pg_loss:np.float64(0.008404427906498313) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0034021124438368133) - actor/ppo_kl:np.float64(0.0007476098585712521) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53515625) - self_distillation/token_reweight_w_mean:np.float64(604270.4651900218) - self_distillation/token_reweight_w_std:np.float64(6237989.264843742) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000877343583852) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1141639132401906) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.7972646504640579) - perf/mfu/actor:np.float64(0.02722053833241182) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.33768463134766) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:6 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006266914773732424 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.006266914773732424 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:200.62890625 - response_length/max:991.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:200.62890625 - response_length_non_aborted/max:991.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:301.4375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001253075897693634 - timing_s/agent_loop/generate_sequences/min:np.float64(1.300127375870943) - timing_s/agent_loop/generate_sequences/max:np.float64(6.019844556227326) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.2045263498512213) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.019844556227326) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:991 - timing_s/gen:11.907243676483631 - timing_s/reward:0.051400572061538696 - timing_s/old_log_prob:2.1595465298742056 - timing_s/adv:0.5041366368532181 - timing_s/update_actor:16.78061268851161 - timing_s/step:31.484902096912265 - timing_s/stop_profile:0.00011850707232952118 - timing_per_token_ms/gen:0.2318343427208121 - timing_per_token_ms/update_actor:0.1305589609233061 - timing_per_token_ms/adv:0.003922357108926531 - perf/total_num_tokens:128529 - perf/time_per_step:31.484902096912265 - perf/throughput:510.28029086917854 (TaskRunner pid=2122883) Training Progress: 96%|█████████▌| 96/100 [1:11:56<02:08, 32.12s/it] (TaskRunner pid=2122883) step:97 - global_seqlen/min:10953 - global_seqlen/max:18766 - global_seqlen/minmax_diff:7813 - global_seqlen/balanced_min:15532 - global_seqlen/balanced_max:15577 - global_seqlen/mean:15539.5 - actor/entropy:0.16762813925743103 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4621080160140991 - training/rollout_probs_diff_mean:0.0049015823751688 - training/rollout_probs_diff_std:0.014530841261148453 - training/rollout_actor_probs_pearson_corr:0.9972710013389587 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.62109375 - self_distillation/correct_sample_fraction:0.515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.62109375 - rollout_corr/rollout_is_mean:1.0001134872436523 - rollout_corr/rollout_is_ratio_fraction_high:2.096787648042664e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.290362944127992e-05 - rollout_corr/rollout_is_max:2.718266487121582 - rollout_corr/rollout_is_min:0.27240684628486633 - rollout_corr/rollout_is_std:0.04075905680656433 - rollout_corr/rollout_is_eff_sample_size:0.9983419299402324 - rollout_corr/rollout_is_seq_mean:1.0001165866851807 - rollout_corr/rollout_is_seq_std:0.0031043454073369503 - rollout_corr/rollout_is_seq_max:1.0120599269866943 - rollout_corr/rollout_is_seq_min:0.9912907481193542 - rollout_corr/rollout_is_seq_max_deviation:0.012059926986694336 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.180992522276938) - rollout_corr/training_log_ppl:np.float64(0.16471268837631214) - rollout_corr/kl:np.float64(0.0010781966158646128) - rollout_corr/k3_kl:np.float64(0.00135383781866949) - rollout_corr/rollout_ppl:np.float64(1.179666185285896) - rollout_corr/rollout_log_ppl:np.float64(0.16363449190976098) - rollout_corr/log_ppl_diff:np.float64(0.0010781964665511623) - rollout_corr/log_ppl_abs_diff:np.float64(0.002958475160994567) - rollout_corr/log_ppl_diff_max:np.float64(0.040684640407562256) - rollout_corr/log_ppl_diff_min:np.float64(-0.00859902799129486) - rollout_corr/ppl_ratio:np.float64(1.0010882455389947) - rollout_corr/chi2_token:np.float64(0.0030083218589425087) - rollout_corr/chi2_seq:np.float64(0.46528330561704934) - actor/pg_loss:np.float64(0.0045534297823905945) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000986409560937318) - actor/ppo_kl:np.float64(0.00034714695744098023) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.62109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_w_mean:np.float64(418781.28417293704) - self_distillation/token_reweight_w_std:np.float64(4127527.157774403) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994778318796307) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05477544415043667) - self_distillation/empty_target_batch:np.float64(0.37890625) - actor/grad_norm:np.float64(0.47031004168093204) - perf/mfu/actor:np.float64(0.026484544790594824) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.26701736450195) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:6 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0008989977068267763 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0008989977068267763 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:186.296875 - response_length/max:521.0 - response_length/min:117.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:186.296875 - response_length_non_aborted/max:521.0 - response_length_non_aborted/min:117.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.3125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.993595838546753e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3136495407670736) - timing_s/agent_loop/generate_sequences/max:np.float64(3.662762999534607) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.092001289151085) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.662762999534607) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:521 - timing_s/gen:9.951778454706073 - timing_s/reward:0.05435767211019993 - timing_s/old_log_prob:2.201238000765443 - timing_s/adv:0.47390690073370934 - timing_s/update_actor:16.638015927746892 - timing_s/step:29.408174956217408 - timing_s/stop_profile:0.00012020952999591827 - timing_per_token_ms/gen:0.20866766868040915 - timing_per_token_ms/update_actor:0.13383648064405942 - timing_per_token_ms/adv:0.003812115099695207 - perf/total_num_tokens:124316 - perf/time_per_step:29.408174956217408 - perf/throughput:528.4074929211027 (TaskRunner pid=2122883) Training Progress: 97%|█████████▋| 97/100 [1:12:25<01:33, 31.32s/it] (TaskRunner pid=2122883) step:98 - global_seqlen/min:10609 - global_seqlen/max:19875 - global_seqlen/minmax_diff:9266 - global_seqlen/balanced_min:15622 - global_seqlen/balanced_max:15661 - global_seqlen/mean:15639.625 - actor/entropy:0.15689386427402496 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29895806312561035 - training/rollout_probs_diff_mean:0.004634734708815813 - training/rollout_probs_diff_std:0.014165589585900307 - training/rollout_actor_probs_pearson_corr:0.9973638653755188 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.6171875 - self_distillation/correct_sample_fraction:0.47265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6171875 - rollout_corr/rollout_is_mean:0.9999297857284546 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00020426088303793222 - rollout_corr/rollout_is_max:1.9731019735336304 - rollout_corr/rollout_is_min:0.3392069637775421 - rollout_corr/rollout_is_std:0.04116937890648842 - rollout_corr/rollout_is_eff_sample_size:0.9983079502062758 - rollout_corr/rollout_is_seq_mean:0.9998367428779602 - rollout_corr/rollout_is_seq_std:0.0030175577849149704 - rollout_corr/rollout_is_seq_max:1.0067346096038818 - rollout_corr/rollout_is_seq_min:0.9912088513374329 - rollout_corr/rollout_is_seq_max_deviation:0.008791148662567139 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1760511882603168) - rollout_corr/training_log_ppl:np.float64(0.1607860552467173) - rollout_corr/kl:np.float64(0.0013580936059085502) - rollout_corr/k3_kl:np.float64(0.0012840381255614375) - rollout_corr/rollout_ppl:np.float64(1.1744432025589049) - rollout_corr/rollout_log_ppl:np.float64(0.15942796152376104) - rollout_corr/log_ppl_diff:np.float64(0.001358093722956255) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031231173197738826) - rollout_corr/log_ppl_diff_max:np.float64(0.01717827469110489) - rollout_corr/log_ppl_diff_min:np.float64(-0.00918780267238617) - rollout_corr/ppl_ratio:np.float64(1.0013662145938724) - rollout_corr/chi2_token:np.float64(0.002376826712861657) - rollout_corr/chi2_seq:np.float64(0.4572152856271714) - actor/pg_loss:np.float64(0.005094089079648256) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014830221407464705) - actor/ppo_kl:np.float64(0.00030168789419349196) - actor/pg_clipfrac_lower:np.float64(2.0777924873982556e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6171875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.47265625) - self_distillation/token_reweight_w_mean:np.float64(541957.2093263853) - self_distillation/token_reweight_w_std:np.float64(5558886.31772876) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006034357938915) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06585665716556832) - self_distillation/empty_target_batch:np.float64(0.3828125) - actor/grad_norm:np.float64(0.5607671290636063) - perf/mfu/actor:np.float64(0.02655483396971807) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.38301467895508) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:6 - critic/score/mean:0.47265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.47265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006357620004564524 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006357620004564524 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:191.23828125 - response_length/max:619.0 - response_length/min:122.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:191.23828125 - response_length_non_aborted/max:619.0 - response_length_non_aborted/min:122.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:297.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.298697113990784e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1248416900634766) - timing_s/agent_loop/generate_sequences/max:np.float64(4.359633307904005) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.9661425068916287) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.359633307904005) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:619 - timing_s/gen:10.306377716362476 - timing_s/reward:0.053195398300886154 - timing_s/old_log_prob:2.2464902866631746 - timing_s/adv:0.48585028015077114 - timing_s/update_actor:16.741798777133226 - timing_s/step:29.920553406700492 - timing_s/stop_profile:0.0001219678670167923 - timing_per_token_ms/gen:0.2105189802553767 - timing_per_token_ms/update_actor:0.13380914485747922 - timing_per_token_ms/adv:0.0038831675963359987 - perf/total_num_tokens:125117 - perf/time_per_step:29.920553406700492 - perf/throughput:522.7050712403474 (TaskRunner pid=2122883) Training Progress: 98%|█████████▊| 98/100 [1:12:55<01:01, 30.91s/it] (TaskRunner pid=2122883) step:99 - global_seqlen/min:11254 - global_seqlen/max:19365 - global_seqlen/minmax_diff:8111 - global_seqlen/balanced_min:14846 - global_seqlen/balanced_max:14884 - global_seqlen/mean:14854.875 - actor/entropy:0.1654287576675415 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3766273558139801 - training/rollout_probs_diff_mean:0.004851612728089094 - training/rollout_probs_diff_std:0.01478587370365858 - training/rollout_actor_probs_pearson_corr:0.9972561001777649 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:0.9998522400856018 - rollout_corr/rollout_is_ratio_fraction_high:8.606036863056943e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00017212073726113886 - rollout_corr/rollout_is_max:2.241464614868164 - rollout_corr/rollout_is_min:0.0024131345562636852 - rollout_corr/rollout_is_std:0.04235963150858879 - rollout_corr/rollout_is_eff_sample_size:0.9982082817437392 - rollout_corr/rollout_is_seq_mean:0.9998134970664978 - rollout_corr/rollout_is_seq_std:0.0033946263138204813 - rollout_corr/rollout_is_seq_max:1.0130257606506348 - rollout_corr/rollout_is_seq_min:0.9866660833358765 - rollout_corr/rollout_is_seq_max_deviation:0.013333916664123535 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1866260999813676) - rollout_corr/training_log_ppl:np.float64(0.16976790214539506) - rollout_corr/kl:np.float64(0.0013737595602520969) - rollout_corr/k3_kl:np.float64(0.0014210123007671882) - rollout_corr/rollout_ppl:np.float64(1.1849938463419676) - rollout_corr/rollout_log_ppl:np.float64(0.1683941402588971) - rollout_corr/log_ppl_diff:np.float64(0.0013737618864979595) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031606208940502256) - rollout_corr/log_ppl_diff_max:np.float64(0.019837364554405212) - rollout_corr/log_ppl_diff_min:np.float64(-0.011420145630836487) - rollout_corr/ppl_ratio:np.float64(1.0013831653632224) - rollout_corr/chi2_token:np.float64(0.002982073463499546) - rollout_corr/chi2_seq:np.float64(0.4137629037722945) - actor/pg_loss:np.float64(0.005757042206823826) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016545631406188477) - actor/ppo_kl:np.float64(0.0001789195605663707) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(424982.73119985964) - self_distillation/token_reweight_w_std:np.float64(4268973.158323454) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9983152812346816) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08449886360904202) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.6708255037665367) - perf/mfu/actor:np.float64(0.025329531080435078) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.14479446411133) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:7 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00973826926201582 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00973826926201582 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:181.55859375 - response_length/max:564.0 - response_length/min:121.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:181.55859375 - response_length_non_aborted/max:564.0 - response_length_non_aborted/min:121.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:282.65625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00039384886622428894 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4838687852025032) - timing_s/agent_loop/generate_sequences/max:np.float64(4.118193060159683) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.065903241862543) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.118193060159683) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:564 - timing_s/gen:9.949486400932074 - timing_s/reward:0.04738493822515011 - timing_s/old_log_prob:2.369191888719797 - timing_s/adv:0.5069726314395666 - timing_s/update_actor:16.423191968351603 - timing_s/step:29.40855737030506 - timing_s/stop_profile:3.053061664104462e-05 - timing_per_token_ms/gen:0.2140641236027469 - timing_per_token_ms/update_actor:0.13819698893756766 - timing_per_token_ms/adv:0.004266045922967769 - perf/total_num_tokens:118839 - perf/time_per_step:29.40855737030506 - perf/throughput:505.1208331286434 (TaskRunner pid=2122883) Training Progress: 99%|█████████▉| 99/100 [1:13:25<00:30, 30.61s/it] (TaskRunner pid=2122883) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2122883) validation generation end (TaskRunner pid=2122883) [prompt] system (TaskRunner pid=2122883) (TaskRunner pid=2122883) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2122883) (TaskRunner pid=2122883) A (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) user (TaskRunner pid=2122883) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2122883) (TaskRunner pid=2122883) A: -0.01 (TaskRunner pid=2122883) B: 1.69 (TaskRunner pid=2122883) C: 2.49 (TaskRunner pid=2122883) D: 0.45 (TaskRunner pid=2122883) Please reason step by step. (TaskRunner pid=2122883) assistant (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) [response] (TaskRunner pid=2122883) The folding stability score of a protein sequence is typically determined through computational methods or experimental techniques, which assess the structural stability based on factors like secondary structure, hydrogen bonding, and van der Waals interactions. Without access to specific computational tools or databases that can calculate this score for the given sequence, it is not possible to determine the exact value. However, the provided options suggest a range of possible scores. Among the options, 1.69 is a reasonable value for a protein sequence that may have moderate stability, as it falls within the typical range of stability scores for proteins. Therefore, the most plausible answer is 1.69. (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) B (TaskRunner pid=2122883) (TaskRunner pid=2122883) [ground_truth] B (TaskRunner pid=2122883) [score] 1.0 (TaskRunner pid=2122883) [acc] 1.0 (TaskRunner pid=2122883) [pred] B (TaskRunner pid=2122883) [incorrect_format] 1 (TaskRunner pid=2122883) [feedback] (TaskRunner pid=2122883) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (WorkerDict pid=2123259) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90/actor (TaskRunner pid=2122883) step:100 - global_seqlen/min:12732 - global_seqlen/max:21577 - global_seqlen/minmax_diff:8845 - global_seqlen/balanced_min:17451 - global_seqlen/balanced_max:17456 - global_seqlen/mean:17454.0 - actor/entropy:0.17713677883148193 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6802612543106079 - training/rollout_probs_diff_mean:0.004852317273616791 - training/rollout_probs_diff_std:0.015326019376516342 - training/rollout_actor_probs_pearson_corr:0.9972407817840576 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.68359375 - self_distillation/correct_sample_fraction:0.51953125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.68359375 - rollout_corr/rollout_is_mean:1.0001729726791382 - rollout_corr/rollout_is_ratio_fraction_high:3.995525185018778e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00019977624469902366 - rollout_corr/rollout_is_max:3.1677703857421875 - rollout_corr/rollout_is_min:0.004871013108640909 - rollout_corr/rollout_is_std:0.04112447053194046 - rollout_corr/rollout_is_eff_sample_size:0.9983123460555775 - rollout_corr/rollout_is_seq_mean:1.0002431869506836 - rollout_corr/rollout_is_seq_std:0.0035587374586611986 - rollout_corr/rollout_is_seq_max:1.0158462524414062 - rollout_corr/rollout_is_seq_min:0.9900786280632019 - rollout_corr/rollout_is_seq_max_deviation:0.01584625244140625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.203045655041933) - rollout_corr/training_log_ppl:np.float64(0.18334656501247082) - rollout_corr/kl:np.float64(0.0016552262983289268) - rollout_corr/k3_kl:np.float64(0.0015354104443474625) - rollout_corr/rollout_ppl:np.float64(1.2010509767569602) - rollout_corr/rollout_log_ppl:np.float64(0.18169133765331935) - rollout_corr/log_ppl_diff:np.float64(0.0016552273591514677) - rollout_corr/log_ppl_abs_diff:np.float64(0.0034326322202105075) - rollout_corr/log_ppl_diff_max:np.float64(0.025318950414657593) - rollout_corr/log_ppl_diff_min:np.float64(-0.009318530559539795) - rollout_corr/ppl_ratio:np.float64(1.001667067874223) - rollout_corr/chi2_token:np.float64(0.0024400001857429743) - rollout_corr/chi2_seq:np.float64(0.4870492776390165) - actor/pg_loss:np.float64(0.004485586890950799) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013001139122934546) - actor/ppo_kl:np.float64(0.0008072886606438345) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.68359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.51953125) - self_distillation/token_reweight_w_mean:np.float64(702577.8307879148) - self_distillation/token_reweight_w_std:np.float64(6412720.680917468) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0022917841561139) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05805497569963336) - self_distillation/empty_target_batch:np.float64(0.31640625) - actor/grad_norm:np.float64(0.551053985953331) - perf/mfu/actor:np.float64(0.030039900561703775) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.17816162109375) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.42875) - val-aux/sciknoweval/reward/std@16:np.float64(0.1440962083400905) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.49282000000000004) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.11460243726780435) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.36628) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.13169012287904225) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.4294) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.14357431870086687) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.53776) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.07143174787390709) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.30992) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.09737417027288235) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.43699999999999994) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.12461646331648187) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.56108) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.03374430360648781) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.27224) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.058556923969906534) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.44081999999999993) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10054251250479307) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.5715600000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.016366340783720925) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.25117999999999996) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.027867680697389763) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.44438) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.08103965668240862) - val-aux/sciknoweval/score/mean@16:np.float64(0.42875) - val-aux/sciknoweval/score/std@16:np.float64(0.1440962083400905) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.49282000000000004) - val-aux/sciknoweval/score/best@2/std:np.float64(0.11460243726780435) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.36628) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.13169012287904225) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.4294) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.14357431870086687) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.53776) - val-aux/sciknoweval/score/best@4/std:np.float64(0.07143174787390709) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.30992) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.09737417027288235) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.43699999999999994) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.12461646331648187) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.56108) - val-aux/sciknoweval/score/best@8/std:np.float64(0.03374430360648781) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.27224) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.058556923969906534) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.44081999999999993) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10054251250479307) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.5715600000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.016366340783720925) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.25117999999999996) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.027867680697389763) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.44438) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.08103965668240862) - val-core/sciknoweval/acc/mean@16:np.float64(0.42875) - val-aux/sciknoweval/acc/std@16:np.float64(0.1440962083400905) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.49282000000000004) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.11460243726780435) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.36628) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.13169012287904225) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.4294) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.14357431870086687) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.53776) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.07143174787390709) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.30992) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.09737417027288235) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.43699999999999994) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.12461646331648187) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.56108) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.03374430360648781) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.27224) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.058556923969906534) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.44081999999999993) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10054251250479307) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.5715600000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.016366340783720925) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.25117999999999996) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.027867680697389763) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.44438) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.08103965668240862) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:7 - critic/score/mean:0.51953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009639204363338649 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0009639204363338649 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:195.53125 - response_length/max:558.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:195.53125 - response_length_non_aborted/max:558.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:349.90625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7704983949661255e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.320506438612938) - timing_s/agent_loop/generate_sequences/max:np.float64(4.294556196779013) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.2014299611910246) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.294556196779013) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:558 - timing_s/gen:10.153399815782905 - timing_s/reward:0.05525374785065651 - timing_s/old_log_prob:2.262016834691167 - timing_s/adv:0.46920773200690746 - timing_s/update_actor:16.347947159782052 - timing_s/step:29.376724991947412 - timing_s/testing:21.447576792910695 - timing_s/save_checkpoint:6.84916939586401 - timing_s/stop_profile:0.00012588128447532654 - timing_per_token_ms/gen:0.2028408146033024 - timing_per_token_ms/update_actor:0.11707880113284957 - timing_per_token_ms/adv:0.0033603166323400613 - perf/total_num_tokens:139632 - perf/time_per_step:29.376724991947412 - perf/throughput:594.1438334186128 (TaskRunner pid=2122883) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2122883) "np.float64(0.42875), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2122883) "np.float64(0.1440962083400905), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2122883) "np.float64(0.49282000000000004), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2122883) "np.float64(0.11460243726780435), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2122883) "np.float64(0.36628), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2122883) "np.float64(0.13169012287904225), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2122883) "np.float64(0.4294), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2122883) "np.float64(0.14357431870086687), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2122883) "np.float64(0.53776), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2122883) "np.float64(0.07143174787390709), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2122883) "np.float64(0.30992), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2122883) Training Progress: 100%|██████████| 100/100 [1:14:23<00:00, 38.74s/it] (TaskRunner pid=2122883) (TaskRunner pid=2122883) "np.float64(0.09737417027288235), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2122883) "np.float64(0.43699999999999994), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2122883) "np.float64(0.12461646331648187), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2122883) "np.float64(0.56108), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2122883) "np.float64(0.03374430360648781), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2122883) "np.float64(0.27224), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2122883) "np.float64(0.058556923969906534), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2122883) "np.float64(0.44081999999999993), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2122883) (TaskRunner pid=2122883) "np.float64(0.10054251250479307), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2122883) "np.float64(0.5715600000000001), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2122883) 'np.float64(0.016366340783720925), ' (TaskRunner pid=2122883) "'val-aux/sciknoweval/reward/worst@16/mean': np.float64(0.25117999999999996), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/reward/worst@16/std': np.float64(0.027867680697389763), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/reward/maj@16/mean': np.float64(0.44438), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/reward/maj@16/std': np.float64(0.08103965668240862), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/mean@16': np.float64(0.42875), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/std@16': np.float64(0.1440962083400905), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/best@2/mean': np.float64(0.49282000000000004), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/best@2/std': np.float64(0.11460243726780435), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/worst@2/mean': np.float64(0.36628), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/worst@2/std': np.float64(0.13169012287904225), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/maj@2/mean': np.float64(0.4294), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/maj@2/std': np.float64(0.14357431870086687), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/best@4/mean': np.float64(0.53776), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/best@4/std': np.float64(0.07143174787390709), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/worst@4/mean': np.float64(0.30992), " (TaskRunner pid=2122883) (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/worst@4/std': np.float64(0.09737417027288235), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/maj@4/mean': np.float64(0.43699999999999994), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/maj@4/std': np.float64(0.12461646331648187), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/best@8/mean': np.float64(0.56108), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/best@8/std': np.float64(0.03374430360648781), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/worst@8/mean': np.float64(0.27224), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/worst@8/std': np.float64(0.058556923969906534), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/maj@8/mean': np.float64(0.44081999999999993), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/maj@8/std': np.float64(0.10054251250479307), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/best@16/mean': np.float64(0.5715600000000001), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/best@16/std': np.float64(0.016366340783720925), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/worst@16/mean': np.float64(0.25117999999999996), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/worst@16/std': np.float64(0.027867680697389763), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/maj@16/mean': np.float64(0.44438), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/maj@16/std': np.float64(0.08103965668240862), " (TaskRunner pid=2122883) "'val-core/sciknoweval/acc/mean@16': np.float64(0.42875), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/std@16': np.float64(0.1440962083400905), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/best@2/mean': np.float64(0.49282000000000004), " (TaskRunner pid=2122883) (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/best@2/std': np.float64(0.11460243726780435), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/worst@2/mean': np.float64(0.36628), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/worst@2/std': np.float64(0.13169012287904225), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/maj@2/mean': np.float64(0.4294), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/maj@2/std': np.float64(0.14357431870086687), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/best@4/mean': np.float64(0.53776), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/best@4/std': np.float64(0.07143174787390709), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/worst@4/mean': np.float64(0.30992), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/worst@4/std': np.float64(0.09737417027288235), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/maj@4/mean': np.float64(0.43699999999999994), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/maj@4/std': np.float64(0.12461646331648187), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/best@8/mean': np.float64(0.56108), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/best@8/std': np.float64(0.03374430360648781), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/worst@8/mean': np.float64(0.27224), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/worst@8/std': np.float64(0.058556923969906534), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/maj@8/mean': np.float64(0.44081999999999993), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/maj@8/std': np.float64(0.10054251250479307), " (TaskRunner pid=2122883) "'val-core/sciknoweval/acc/best@16/mean': np.float64(0.5715600000000001), " (TaskRunner pid=2122883) "'val-core/sciknoweval/acc/best@16/std': np.float64(0.016366340783720925), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/worst@16/mean': np.float64(0.25117999999999996), " (TaskRunner pid=2122883) (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/worst@16/std': np.float64(0.027867680697389763), " (TaskRunner pid=2122883) "'val-core/sciknoweval/acc/maj@16/mean': np.float64(0.44438), " (TaskRunner pid=2122883) "'val-core/sciknoweval/acc/maj@16/std': np.float64(0.08103965668240862), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/mean@16': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/std@16': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/best@2/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/best@2/std': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/worst@2/std': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/maj@2/std': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/best@4/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/best@4/std': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/worst@4/std': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/maj@4/std': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/worst@8/std': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/maj@8/std': np.float64(0.0), " (TaskRunner pid=2122883) (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/worst@16/std': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/maj@16/std': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/num_turns/min': np.int32(2), 'val-aux/num_turns/max': np.int32(2), " (TaskRunner pid=2122883) "'val-aux/num_turns/mean': np.float64(2.0)}") (TaskRunner pid=2122883) Training Progress: 100%|██████████| 100/100 [1:14:23<00:00, 44.64s/it] (TaskRunner pid=2122883) Traceback (most recent call last): (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/workers/default_worker.py", line 322, in (TaskRunner pid=2122883) worker.main_loop() (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py", line 1031, in main_loop (TaskRunner pid=2122883) self.core_worker.run_task_loop() (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/utils/data/_utils/signal_handling.py", line 73, in handler (TaskRunner pid=2122883) _error_if_any_worker_fails() (TaskRunner pid=2122883) RuntimeError: DataLoader worker (pid 2143352) is killed by signal: Killed. (TaskRunner pid=2122883) Traceback (most recent call last): (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/workers/default_worker.py", line 322, in (TaskRunner pid=2122883) worker.main_loop() (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py", line 1031, in main_loop (TaskRunner pid=2122883) self.core_worker.run_task_loop() (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/utils/data/_utils/signal_handling.py", line 73, in handler (TaskRunner pid=2122883) wandb: ERROR Problem finishing run (TaskRunner pid=2122883) Exception ignored in: (TaskRunner pid=2122883) Traceback (most recent call last): (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/verl/utils/tracking.py", line 169, in __del__ (TaskRunner pid=2122883) self.logger["wandb"].finish(exit_code=0) (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 4097, in finish (TaskRunner pid=2122883) wandb.run.finish(exit_code=exit_code, quiet=quiet) (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 400, in wrapper (TaskRunner pid=2122883) return func(self, *args, **kwargs) (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 445, in wrapper (TaskRunner pid=2122883) return func(self, *args, **kwargs) (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 2275, in finish (TaskRunner pid=2122883) return self._finish(exit_code) (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 400, in wrapper (TaskRunner pid=2122883) return func(self, *args, **kwargs) (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 2303, in _finish (TaskRunner pid=2122883) self._atexit_cleanup(exit_code=exit_code) (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 2498, in _atexit_cleanup (TaskRunner pid=2122883) self._on_finish() (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 2711, in _on_finish (TaskRunner pid=2122883) exit_handle = self._backend.interface.deliver_exit(self._exit_code) (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/interface/interface.py", line 1038, in deliver_exit (TaskRunner pid=2122883) return self._deliver_exit(exit_data) (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/interface/interface_shared.py", line 460, in _deliver_exit (TaskRunner pid=2122883) return self._deliver(record) (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/interface/interface_sock.py", line 50, in _deliver (TaskRunner pid=2122883) return self._asyncer.run(lambda: self.deliver_async(record)) (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run (TaskRunner pid=2122883) return future.result() (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result (TaskRunner pid=2122883) return self.__get_result() (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (TaskRunner pid=2122883) raise self._exception (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap (TaskRunner pid=2122883) return await fn() (TaskRunner pid=2122883) ^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/interface/interface_sock.py", line 58, in deliver_async (TaskRunner pid=2122883) handle = await self._client.deliver(request) (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 53, in deliver (TaskRunner pid=2122883) handle = self._mailbox.require_response(request) (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/mailbox/mailbox.py", line 76, in require_response (TaskRunner pid=2122883) raise MailboxClosedError() (TaskRunner pid=2122883) wandb.sdk.mailbox.mailbox.MailboxClosedError: main_ppo exit code: 0 [2026-07-02 12:53:36] Finished biology rlsd_tr [2026-07-02 12:53:36] Starting material grpo Experiment: qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Dataset: material Method: grpo Config: baseline_grpo Model: Qwen/Qwen3-4B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/material/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/material/test.parquet Ray tmp: /tmp/ray_q3g_material_grpo_Qwen3_4B 2026-07-02 12:53:38,592 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Config: baseline_grpo Task: datasets/sciknoweval/material Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/material/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/material/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-GRPO-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_material_grpo_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/material +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.actor.policy_loss.loss_mode=vanilla actor_rollout_ref.actor.kl_loss_coef=0.0 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_material_grpo_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/material', 'EXPERIMENT': 'qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 12:53:48,296 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2146199) TaskRunner hostname: mole-workspace-rw, PID: 2146199 (TaskRunner pid=2146199) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2146199) 'calculate_entropy': False, (TaskRunner pid=2146199) 'calculate_sum_pi_squared': False, (TaskRunner pid=2146199) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2146199) 'async_save': False, (TaskRunner pid=2146199) 'load_contents': ['model', (TaskRunner pid=2146199) 'optimizer', (TaskRunner pid=2146199) 'extra'], (TaskRunner pid=2146199) 'save_contents': ['model', (TaskRunner pid=2146199) 'optimizer', (TaskRunner pid=2146199) 'extra']}, (TaskRunner pid=2146199) 'clip_ratio': 0.2, (TaskRunner pid=2146199) 'clip_ratio_c': 3.0, (TaskRunner pid=2146199) 'clip_ratio_high': 0.28, (TaskRunner pid=2146199) 'clip_ratio_low': 0.2, (TaskRunner pid=2146199) 'data_loader_seed': 42, (TaskRunner pid=2146199) 'entropy_checkpointing': False, (TaskRunner pid=2146199) 'entropy_coeff': 0, (TaskRunner pid=2146199) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2146199) 'freeze_vision_tower': False, (TaskRunner pid=2146199) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2146199) 'dtype': 'bfloat16', (TaskRunner pid=2146199) 'entropy_checkpointing': False, (TaskRunner pid=2146199) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2146199) 'forward_only': False, (TaskRunner pid=2146199) 'forward_prefetch': False, (TaskRunner pid=2146199) 'fsdp_size': -1, (TaskRunner pid=2146199) 'full_determinism': False, (TaskRunner pid=2146199) 'model_dtype': 'fp32', (TaskRunner pid=2146199) 'offload_policy': False, (TaskRunner pid=2146199) 'optimizer_offload': False, (TaskRunner pid=2146199) 'param_offload': False, (TaskRunner pid=2146199) 'reshard_after_forward': True, (TaskRunner pid=2146199) 'seed': 42, (TaskRunner pid=2146199) 'strategy': 'fsdp', (TaskRunner pid=2146199) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2146199) 'use_orig_params': False, (TaskRunner pid=2146199) 'use_torch_compile': True, (TaskRunner pid=2146199) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2146199) 'grad_clip': 1.0, (TaskRunner pid=2146199) 'kl_loss_coef': 0.0, (TaskRunner pid=2146199) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2146199) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2146199) 'loss_scale_factor': None, (TaskRunner pid=2146199) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2146199) 'betas': [0.9, 0.999], (TaskRunner pid=2146199) 'clip_grad': 1.0, (TaskRunner pid=2146199) 'lr': 1e-06, (TaskRunner pid=2146199) 'lr_scheduler_type': 'constant', (TaskRunner pid=2146199) 'lr_warmup_steps': 10, (TaskRunner pid=2146199) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2146199) 'min_lr_ratio': 0.0, (TaskRunner pid=2146199) 'num_cycles': 0.5, (TaskRunner pid=2146199) 'optimizer': 'AdamW', (TaskRunner pid=2146199) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2146199) 'override_optimizer_config': None, (TaskRunner pid=2146199) 'total_training_steps': -1, (TaskRunner pid=2146199) 'warmup_style': None, (TaskRunner pid=2146199) 'weight_decay': 0.01}, (TaskRunner pid=2146199) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2146199) 'clip_cov_lb': 1.0, (TaskRunner pid=2146199) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2146199) 'clip_cov_ub': 5.0, (TaskRunner pid=2146199) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2146199) 'loss_mode': 'vanilla', (TaskRunner pid=2146199) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2146199) 'ppo_epochs': 1, (TaskRunner pid=2146199) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2146199) 'ppo_micro_batch_size': None, (TaskRunner pid=2146199) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2146199) 'ppo_mini_batch_size': 8, (TaskRunner pid=2146199) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2146199) 'all_ranks': False, (TaskRunner pid=2146199) 'enable': False, (TaskRunner pid=2146199) 'ranks': [], (TaskRunner pid=2146199) 'save_path': 'outputs/profile', (TaskRunner pid=2146199) 'tool': None, (TaskRunner pid=2146199) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2146199) 'analysis': True, (TaskRunner pid=2146199) 'contents': [], (TaskRunner pid=2146199) 'discrete': False, (TaskRunner pid=2146199) 'level': 'level0'}, (TaskRunner pid=2146199) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2146199) 'discrete': False}, (TaskRunner pid=2146199) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2146199) 'step_end': None, (TaskRunner pid=2146199) 'step_start': 0}, (TaskRunner pid=2146199) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2146199) 'stack_depth': 32, (TaskRunner pid=2146199) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2146199) 'rollout_n': 8, (TaskRunner pid=2146199) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2146199) 'mode': 'disabled', (TaskRunner pid=2146199) 'record_file': None, (TaskRunner pid=2146199) 'replay_file': None}, (TaskRunner pid=2146199) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2146199) 'alpha': 0.5, (TaskRunner pid=2146199) 'distillation_add_tail': True, (TaskRunner pid=2146199) 'distillation_topk': 100, (TaskRunner pid=2146199) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2146199) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2146199) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2146199) 'enable': False, (TaskRunner pid=2146199) 'loss_weight': 0.0, (TaskRunner pid=2146199) 'mask': 'all'}, (TaskRunner pid=2146199) 'feedback_template': '\n' (TaskRunner pid=2146199) 'The ' (TaskRunner pid=2146199) 'following ' (TaskRunner pid=2146199) 'is ' (TaskRunner pid=2146199) 'feedback ' (TaskRunner pid=2146199) 'from ' (TaskRunner pid=2146199) 'your ' (TaskRunner pid=2146199) 'unsuccessful ' (TaskRunner pid=2146199) 'earlier ' (TaskRunner pid=2146199) 'attempt:\n' (TaskRunner pid=2146199) '\n' (TaskRunner pid=2146199) '{feedback_raw}', (TaskRunner pid=2146199) 'full_logit_distillation': True, (TaskRunner pid=2146199) 'include_environment_feedback': True, (TaskRunner pid=2146199) 'is_clip': 2, (TaskRunner pid=2146199) 'max_reprompt_len': 22528, (TaskRunner pid=2146199) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2146199) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2146199) '\n' (TaskRunner pid=2146199) 'Correctly ' (TaskRunner pid=2146199) 'solve ' (TaskRunner pid=2146199) 'the ' (TaskRunner pid=2146199) 'original ' (TaskRunner pid=2146199) 'question.', (TaskRunner pid=2146199) 'reprompt_truncation': 'right', (TaskRunner pid=2146199) 'solution_template': '\n' (TaskRunner pid=2146199) 'Correct ' (TaskRunner pid=2146199) 'solution:\n' (TaskRunner pid=2146199) '\n' (TaskRunner pid=2146199) '{successful_previous_attempt}', (TaskRunner pid=2146199) 'srpo_dynamic_weighting': False, (TaskRunner pid=2146199) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2146199) 'success_reward_threshold': 0.5, (TaskRunner pid=2146199) 'teacher_regularization': 'ema', (TaskRunner pid=2146199) 'teacher_update_rate': 0.0, (TaskRunner pid=2146199) 'token_reweight_decay_steps': None, (TaskRunner pid=2146199) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2146199) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2146199) 'shuffle': False, (TaskRunner pid=2146199) 'strategy': 'fsdp', (TaskRunner pid=2146199) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2146199) 'tau_neg': 1.05, (TaskRunner pid=2146199) 'tau_pos': 1.0, (TaskRunner pid=2146199) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2146199) 'use_dynamic_bsz': False, (TaskRunner pid=2146199) 'use_fused_kernels': False, (TaskRunner pid=2146199) 'use_kl_loss': False, (TaskRunner pid=2146199) 'use_prefix_grouper': False, (TaskRunner pid=2146199) 'use_remove_padding': True, (TaskRunner pid=2146199) 'use_torch_compile': True}, (TaskRunner pid=2146199) 'hybrid_engine': True, (TaskRunner pid=2146199) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2146199) 'custom_chat_template': None, (TaskRunner pid=2146199) 'enable_activation_offload': False, (TaskRunner pid=2146199) 'enable_gradient_checkpointing': True, (TaskRunner pid=2146199) 'exclude_modules': None, (TaskRunner pid=2146199) 'external_lib': None, (TaskRunner pid=2146199) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2146199) 'hf_config_path': None, (TaskRunner pid=2146199) 'lora_adapter_path': None, (TaskRunner pid=2146199) 'lora_alpha': 16, (TaskRunner pid=2146199) 'lora_rank': 0, (TaskRunner pid=2146199) 'override_config': {}, (TaskRunner pid=2146199) 'path': 'Qwen/Qwen3-4B', (TaskRunner pid=2146199) 'target_modules': 'all-linear', (TaskRunner pid=2146199) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2146199) 'num_shards': 4}, (TaskRunner pid=2146199) 'tokenizer_path': None, (TaskRunner pid=2146199) 'trust_remote_code': True, (TaskRunner pid=2146199) 'use_fused_kernels': False, (TaskRunner pid=2146199) 'use_liger': False, (TaskRunner pid=2146199) 'use_remove_padding': True, (TaskRunner pid=2146199) 'use_shm': False}, (TaskRunner pid=2146199) 'nccl_timeout': 600, (TaskRunner pid=2146199) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2146199) 'entropy_checkpointing': False, (TaskRunner pid=2146199) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2146199) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2146199) 'dtype': 'bfloat16', (TaskRunner pid=2146199) 'entropy_checkpointing': False, (TaskRunner pid=2146199) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2146199) 'forward_only': True, (TaskRunner pid=2146199) 'forward_prefetch': False, (TaskRunner pid=2146199) 'fsdp_size': -1, (TaskRunner pid=2146199) 'full_determinism': False, (TaskRunner pid=2146199) 'model_dtype': 'fp32', (TaskRunner pid=2146199) 'offload_policy': False, (TaskRunner pid=2146199) 'optimizer_offload': False, (TaskRunner pid=2146199) 'param_offload': False, (TaskRunner pid=2146199) 'reshard_after_forward': True, (TaskRunner pid=2146199) 'seed': 42, (TaskRunner pid=2146199) 'strategy': 'fsdp', (TaskRunner pid=2146199) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2146199) 'use_orig_params': False, (TaskRunner pid=2146199) 'use_torch_compile': True, (TaskRunner pid=2146199) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2146199) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2146199) 'log_prob_micro_batch_size': None, (TaskRunner pid=2146199) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2146199) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2146199) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2146199) 'all_ranks': False, (TaskRunner pid=2146199) 'enable': False, (TaskRunner pid=2146199) 'ranks': [], (TaskRunner pid=2146199) 'save_path': 'outputs/profile', (TaskRunner pid=2146199) 'tool': None, (TaskRunner pid=2146199) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2146199) 'analysis': True, (TaskRunner pid=2146199) 'contents': [], (TaskRunner pid=2146199) 'discrete': False, (TaskRunner pid=2146199) 'level': 'level0'}, (TaskRunner pid=2146199) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2146199) 'discrete': False}, (TaskRunner pid=2146199) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2146199) 'step_end': None, (TaskRunner pid=2146199) 'step_start': 0}, (TaskRunner pid=2146199) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2146199) 'stack_depth': 32, (TaskRunner pid=2146199) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2146199) 'rollout_n': 8, (TaskRunner pid=2146199) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2146199) 'mode': 'disabled', (TaskRunner pid=2146199) 'record_file': None, (TaskRunner pid=2146199) 'replay_file': None}, (TaskRunner pid=2146199) 'strategy': 'fsdp', (TaskRunner pid=2146199) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2146199) 'use_torch_compile': True}, (TaskRunner pid=2146199) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2146199) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2146199) 'agent_loop_config_path': None, (TaskRunner pid=2146199) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2146199) 'name': None, (TaskRunner pid=2146199) 'path': None}, (TaskRunner pid=2146199) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2146199) 'num_workers': 8}, (TaskRunner pid=2146199) 'calculate_log_probs': True, (TaskRunner pid=2146199) 'cudagraph_capture_sizes': None, (TaskRunner pid=2146199) 'data_parallel_size': 1, (TaskRunner pid=2146199) 'disable_log_stats': True, (TaskRunner pid=2146199) 'do_sample': True, (TaskRunner pid=2146199) 'dtype': 'bfloat16', (TaskRunner pid=2146199) 'enable_chunked_prefill': True, (TaskRunner pid=2146199) 'enable_prefix_caching': True, (TaskRunner pid=2146199) 'enable_rollout_routing_replay': False, (TaskRunner pid=2146199) 'enforce_eager': False, (TaskRunner pid=2146199) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2146199) 'expert_parallel_size': 1, (TaskRunner pid=2146199) 'free_cache_engine': True, (TaskRunner pid=2146199) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2146199) 'ignore_eos': False, (TaskRunner pid=2146199) 'layered_summon': False, (TaskRunner pid=2146199) 'load_format': 'dummy', (TaskRunner pid=2146199) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2146199) 'log_prob_micro_batch_size': None, (TaskRunner pid=2146199) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2146199) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2146199) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2146199) 'max_model_len': 10240, (TaskRunner pid=2146199) 'max_num_batched_tokens': 10240, (TaskRunner pid=2146199) 'max_num_seqs': 1024, (TaskRunner pid=2146199) 'mode': 'async', (TaskRunner pid=2146199) 'multi_stage_wake_up': False, (TaskRunner pid=2146199) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2146199) 'enable': False, (TaskRunner pid=2146199) 'format': 'hermes', (TaskRunner pid=2146199) 'interaction_config_path': None, (TaskRunner pid=2146199) 'max_assistant_turns': None, (TaskRunner pid=2146199) 'max_parallel_calls': 1, (TaskRunner pid=2146199) 'max_tool_response_length': 256, (TaskRunner pid=2146199) 'max_user_turns': None, (TaskRunner pid=2146199) 'num_repeat_rollouts': None, (TaskRunner pid=2146199) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2146199) 'tool_config_path': None, (TaskRunner pid=2146199) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2146199) 'use_inference_chat_template': False}, (TaskRunner pid=2146199) 'n': 8, (TaskRunner pid=2146199) 'name': 'vllm', (TaskRunner pid=2146199) 'over_sample_rate': 0, (TaskRunner pid=2146199) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2146199) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2146199) 'all_ranks': False, (TaskRunner pid=2146199) 'enable': False, (TaskRunner pid=2146199) 'ranks': [], (TaskRunner pid=2146199) 'save_path': 'outputs/profile', (TaskRunner pid=2146199) 'tool': None, (TaskRunner pid=2146199) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2146199) 'analysis': True, (TaskRunner pid=2146199) 'contents': [], (TaskRunner pid=2146199) 'discrete': False, (TaskRunner pid=2146199) 'level': 'level0'}, (TaskRunner pid=2146199) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2146199) 'discrete': False}, (TaskRunner pid=2146199) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2146199) 'step_end': None, (TaskRunner pid=2146199) 'step_start': 0}, (TaskRunner pid=2146199) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2146199) 'stack_depth': 32, (TaskRunner pid=2146199) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2146199) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2146199) 'enable': False, (TaskRunner pid=2146199) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2146199) 'port': 9090, (TaskRunner pid=2146199) 'served_model_name': 'Qwen/Qwen3-4B'}, (TaskRunner pid=2146199) 'prompt_length': 2048, (TaskRunner pid=2146199) 'quantization': None, (TaskRunner pid=2146199) 'quantization_config_file': None, (TaskRunner pid=2146199) 'response_length': 8192, (TaskRunner pid=2146199) 'scheduling_policy': 'fcfs', (TaskRunner pid=2146199) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2146199) 'skip_rollout': False, (TaskRunner pid=2146199) 'skip_tokenizer_init': True, (TaskRunner pid=2146199) 'temperature': 1.0, (TaskRunner pid=2146199) 'tensor_model_parallel_size': 2, (TaskRunner pid=2146199) 'top_k': -1, (TaskRunner pid=2146199) 'top_p': 1.0, (TaskRunner pid=2146199) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2146199) 'backend': None, (TaskRunner pid=2146199) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2146199) 'token2text': False}, (TaskRunner pid=2146199) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2146199) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2146199) 'do_sample': True, (TaskRunner pid=2146199) 'n': 16, (TaskRunner pid=2146199) 'temperature': 0.6, (TaskRunner pid=2146199) 'top_k': -1, (TaskRunner pid=2146199) 'top_p': 0.95}}}, (TaskRunner pid=2146199) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2146199) 'adv_estimator': 'grpo', (TaskRunner pid=2146199) 'gamma': 1.0, (TaskRunner pid=2146199) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2146199) 'horizon': 10000, (TaskRunner pid=2146199) 'kl_coef': 0.001, (TaskRunner pid=2146199) 'target_kl': 0.1, (TaskRunner pid=2146199) 'type': 'fixed'}, (TaskRunner pid=2146199) 'kl_penalty': 'kl', (TaskRunner pid=2146199) 'lam': 1.0, (TaskRunner pid=2146199) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2146199) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2146199) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2146199) 'loss_type': 'ppo_clip', (TaskRunner pid=2146199) 'rollout_is': 'token', (TaskRunner pid=2146199) 'rollout_is_batch_normalize': False, (TaskRunner pid=2146199) 'rollout_is_threshold': 2.0, (TaskRunner pid=2146199) 'rollout_rs': None, (TaskRunner pid=2146199) 'rollout_rs_threshold': None}, (TaskRunner pid=2146199) 'use_kl_in_reward': False, (TaskRunner pid=2146199) 'use_pf_ppo': False}, (TaskRunner pid=2146199) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2146199) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2146199) 'async_save': False, (TaskRunner pid=2146199) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2146199) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2146199) 'cliprange_value': 0.5, (TaskRunner pid=2146199) 'data_loader_seed': 42, (TaskRunner pid=2146199) 'enable': None, (TaskRunner pid=2146199) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2146199) 'forward_micro_batch_size': None, (TaskRunner pid=2146199) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2146199) 'grad_clip': 1.0, (TaskRunner pid=2146199) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2146199) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2146199) 'enable_activation_offload': False, (TaskRunner pid=2146199) 'enable_gradient_checkpointing': True, (TaskRunner pid=2146199) 'external_lib': None, (TaskRunner pid=2146199) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2146199) 'dtype': 'bfloat16', (TaskRunner pid=2146199) 'entropy_checkpointing': False, (TaskRunner pid=2146199) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2146199) 'forward_only': False, (TaskRunner pid=2146199) 'forward_prefetch': False, (TaskRunner pid=2146199) 'fsdp_size': -1, (TaskRunner pid=2146199) 'full_determinism': False, (TaskRunner pid=2146199) 'model_dtype': 'fp32', (TaskRunner pid=2146199) 'offload_policy': False, (TaskRunner pid=2146199) 'optimizer_offload': False, (TaskRunner pid=2146199) 'param_offload': False, (TaskRunner pid=2146199) 'reshard_after_forward': True, (TaskRunner pid=2146199) 'seed': 42, (TaskRunner pid=2146199) 'strategy': 'fsdp', (TaskRunner pid=2146199) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2146199) 'use_orig_params': False, (TaskRunner pid=2146199) 'use_torch_compile': True, (TaskRunner pid=2146199) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2146199) 'lora_alpha': 16, (TaskRunner pid=2146199) 'lora_rank': 0, (TaskRunner pid=2146199) 'override_config': {}, (TaskRunner pid=2146199) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2146199) 'target_modules': 'all-linear', (TaskRunner pid=2146199) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2146199) 'tokenizer_path': 'Qwen/Qwen3-4B', (TaskRunner pid=2146199) 'trust_remote_code': True, (TaskRunner pid=2146199) 'use_remove_padding': False, (TaskRunner pid=2146199) 'use_shm': False}, (TaskRunner pid=2146199) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2146199) 'betas': [0.9, 0.999], (TaskRunner pid=2146199) 'clip_grad': 1.0, (TaskRunner pid=2146199) 'lr': 1e-05, (TaskRunner pid=2146199) 'lr_scheduler_type': 'constant', (TaskRunner pid=2146199) 'lr_warmup_steps': -1, (TaskRunner pid=2146199) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2146199) 'min_lr_ratio': 0.0, (TaskRunner pid=2146199) 'num_cycles': 0.5, (TaskRunner pid=2146199) 'optimizer': 'AdamW', (TaskRunner pid=2146199) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2146199) 'override_optimizer_config': None, (TaskRunner pid=2146199) 'total_training_steps': -1, (TaskRunner pid=2146199) 'warmup_style': None, (TaskRunner pid=2146199) 'weight_decay': 0.01}, (TaskRunner pid=2146199) 'ppo_epochs': 1, (TaskRunner pid=2146199) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2146199) 'ppo_micro_batch_size': None, (TaskRunner pid=2146199) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2146199) 'ppo_mini_batch_size': 8, (TaskRunner pid=2146199) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2146199) 'all_ranks': False, (TaskRunner pid=2146199) 'enable': False, (TaskRunner pid=2146199) 'ranks': [], (TaskRunner pid=2146199) 'save_path': 'outputs/profile', (TaskRunner pid=2146199) 'tool': None, (TaskRunner pid=2146199) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2146199) 'analysis': True, (TaskRunner pid=2146199) 'contents': [], (TaskRunner pid=2146199) 'discrete': False, (TaskRunner pid=2146199) 'level': 'level0'}, (TaskRunner pid=2146199) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2146199) 'discrete': False}, (TaskRunner pid=2146199) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2146199) 'step_end': None, (TaskRunner pid=2146199) 'step_start': 0}, (TaskRunner pid=2146199) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2146199) 'stack_depth': 32, (TaskRunner pid=2146199) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2146199) 'rollout_n': 8, (TaskRunner pid=2146199) 'shuffle': False, (TaskRunner pid=2146199) 'strategy': 'fsdp', (TaskRunner pid=2146199) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2146199) 'use_dynamic_bsz': False}, (TaskRunner pid=2146199) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2146199) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2146199) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2146199) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2146199) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2146199) 'dataloader_num_workers': 8, (TaskRunner pid=2146199) 'filter_overlong_prompts': True, (TaskRunner pid=2146199) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2146199) 'image_key': 'images', (TaskRunner pid=2146199) 'image_patch_size': 14, (TaskRunner pid=2146199) 'max_prompt_length': 2048, (TaskRunner pid=2146199) 'max_response_length': 8192, (TaskRunner pid=2146199) 'prompt_key': 'prompt', (TaskRunner pid=2146199) 'return_full_prompt': False, (TaskRunner pid=2146199) 'return_multi_modal_inputs': True, (TaskRunner pid=2146199) 'return_raw_chat': True, (TaskRunner pid=2146199) 'return_raw_input_ids': False, (TaskRunner pid=2146199) 'reward_fn_key': 'data_source', (TaskRunner pid=2146199) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2146199) 'seed': None, (TaskRunner pid=2146199) 'shuffle': True, (TaskRunner pid=2146199) 'tokenizer': None, (TaskRunner pid=2146199) 'tool_config_path': None, (TaskRunner pid=2146199) 'train_batch_size': 32, (TaskRunner pid=2146199) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/material/train.parquet'], (TaskRunner pid=2146199) 'train_max_samples': 3200, (TaskRunner pid=2146199) 'truncation': 'error', (TaskRunner pid=2146199) 'trust_remote_code': True, (TaskRunner pid=2146199) 'use_shm': False, (TaskRunner pid=2146199) 'val_batch_size': None, (TaskRunner pid=2146199) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/material/test.parquet'], (TaskRunner pid=2146199) 'val_max_samples': -1, (TaskRunner pid=2146199) 'validation_shuffle': False, (TaskRunner pid=2146199) 'video_key': 'videos'}, (TaskRunner pid=2146199) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2146199) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2146199) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2146199) 'cuda-memory-usage': 'true', (TaskRunner pid=2146199) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2146199) 'discrete': False, (TaskRunner pid=2146199) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2146199) 'capture-range-end': None, (TaskRunner pid=2146199) 'cuda-graph-trace': 'graph', (TaskRunner pid=2146199) 'cuda-memory-usage': 'true', (TaskRunner pid=2146199) 'kill': 'none', (TaskRunner pid=2146199) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2146199) 'torch_memory': {'context': 'all', (TaskRunner pid=2146199) 'kw_args': {}, (TaskRunner pid=2146199) 'stack_depth': 32, (TaskRunner pid=2146199) 'stacks': 'all', (TaskRunner pid=2146199) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2146199) 'profile_continuous_steps': False, (TaskRunner pid=2146199) 'save_path': 'outputs/profile', (TaskRunner pid=2146199) 'steps': None, (TaskRunner pid=2146199) 'tool': None}, (TaskRunner pid=2146199) 'max_model_len': 10240, (TaskRunner pid=2146199) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_material_grpo_Qwen3_4B', (TaskRunner pid=2146199) 'include_dashboard': False, (TaskRunner pid=2146199) 'num_cpus': None, (TaskRunner pid=2146199) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2146199) 'TASK': 'datasets/sciknoweval/material', (TaskRunner pid=2146199) 'USER': 'root'}}}, (TaskRunner pid=2146199) 'timeline_json_file': None}, (TaskRunner pid=2146199) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2146199) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2146199) 'name': 'custom_reward_manager', (TaskRunner pid=2146199) 'path': None}, (TaskRunner pid=2146199) 'name': 'naive', (TaskRunner pid=2146199) 'source': 'register'}, (TaskRunner pid=2146199) 'reward_model': {'enable': False, (TaskRunner pid=2146199) 'enable_resource_pool': False, (TaskRunner pid=2146199) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2146199) 'launch_reward_fn_async': False, (TaskRunner pid=2146199) 'max_length': None, (TaskRunner pid=2146199) 'micro_batch_size': None, (TaskRunner pid=2146199) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2146199) 'model': {'external_lib': None, (TaskRunner pid=2146199) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2146199) 'forward_prefetch': False, (TaskRunner pid=2146199) 'fsdp_size': -1, (TaskRunner pid=2146199) 'param_offload': False, (TaskRunner pid=2146199) 'reshard_after_forward': True, (TaskRunner pid=2146199) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2146199) 'input_tokenizer': 'Qwen/Qwen3-4B', (TaskRunner pid=2146199) 'override_config': {}, (TaskRunner pid=2146199) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2146199) 'trust_remote_code': False, (TaskRunner pid=2146199) 'use_fused_kernels': False, (TaskRunner pid=2146199) 'use_remove_padding': False, (TaskRunner pid=2146199) 'use_shm': False}, (TaskRunner pid=2146199) 'n_gpus_per_node': 8, (TaskRunner pid=2146199) 'nnodes': 0, (TaskRunner pid=2146199) 'num_workers': 1, (TaskRunner pid=2146199) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2146199) 'all_ranks': False, (TaskRunner pid=2146199) 'enable': False, (TaskRunner pid=2146199) 'ranks': [], (TaskRunner pid=2146199) 'save_path': 'outputs/profile', (TaskRunner pid=2146199) 'tool': None, (TaskRunner pid=2146199) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2146199) 'analysis': True, (TaskRunner pid=2146199) 'contents': [], (TaskRunner pid=2146199) 'discrete': False, (TaskRunner pid=2146199) 'level': 'level0'}, (TaskRunner pid=2146199) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2146199) 'discrete': False}, (TaskRunner pid=2146199) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2146199) 'step_end': None, (TaskRunner pid=2146199) 'step_start': 0}, (TaskRunner pid=2146199) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2146199) 'stack_depth': 32, (TaskRunner pid=2146199) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2146199) 'reward_loop_class_name': None, (TaskRunner pid=2146199) 'reward_loop_module_path': None, (TaskRunner pid=2146199) 'reward_loop_source': 'register', (TaskRunner pid=2146199) 'reward_manager': 'naive', (TaskRunner pid=2146199) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2146199) 'cudagraph_capture_sizes': None, (TaskRunner pid=2146199) 'data_parallel_size': 1, (TaskRunner pid=2146199) 'disable_log_stats': True, (TaskRunner pid=2146199) 'dtype': 'bfloat16', (TaskRunner pid=2146199) 'enable_chunked_prefill': True, (TaskRunner pid=2146199) 'enable_prefix_caching': True, (TaskRunner pid=2146199) 'enforce_eager': True, (TaskRunner pid=2146199) 'engine_kwargs': {}, (TaskRunner pid=2146199) 'expert_parallel_size': 1, (TaskRunner pid=2146199) 'free_cache_engine': True, (TaskRunner pid=2146199) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2146199) 'limit_images': None, (TaskRunner pid=2146199) 'load_format': 'auto', (TaskRunner pid=2146199) 'max_model_len': None, (TaskRunner pid=2146199) 'max_num_batched_tokens': 8192, (TaskRunner pid=2146199) 'max_num_seqs': 1024, (TaskRunner pid=2146199) 'name': '???', (TaskRunner pid=2146199) 'prompt_length': 2048, (TaskRunner pid=2146199) 'response_length': 2048, (TaskRunner pid=2146199) 'skip_tokenizer_init': False, (TaskRunner pid=2146199) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2146199) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2146199) 'memory_limit_mb': 1024, (TaskRunner pid=2146199) 'url': None}, (TaskRunner pid=2146199) 'strategy': 'fsdp', (TaskRunner pid=2146199) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2146199) 'use_dynamic_bsz': False, (TaskRunner pid=2146199) 'use_reward_loop': False}, (TaskRunner pid=2146199) 'trainer': {'balance_batch': True, (TaskRunner pid=2146199) 'critic_warmup': 0, (TaskRunner pid=2146199) 'default_hdfs_dir': None, (TaskRunner pid=2146199) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2146199) 'del_local_ckpt_after_load': False, (TaskRunner pid=2146199) 'device': 'cuda', (TaskRunner pid=2146199) 'esi_redundant_time': 0, (TaskRunner pid=2146199) 'experiment_name': 'qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2146199) 'group_name': 'QWEN3-GRPO-generalization', (TaskRunner pid=2146199) 'log_val_generations': 0, (TaskRunner pid=2146199) 'logger': ['console', 'wandb'], (TaskRunner pid=2146199) 'max_actor_ckpt_to_keep': 1, (TaskRunner pid=2146199) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2146199) 'n_gpus_per_node': 8, (TaskRunner pid=2146199) 'nnodes': 1, (TaskRunner pid=2146199) 'project_name': 'SDPO-root', (TaskRunner pid=2146199) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2146199) 'resume_from_path': None, (TaskRunner pid=2146199) 'resume_mode': 'auto', (TaskRunner pid=2146199) 'rollout_data_dir': None, (TaskRunner pid=2146199) 'save_freq': 10, (TaskRunner pid=2146199) 'test_freq': 10, (TaskRunner pid=2146199) 'total_epochs': 30, (TaskRunner pid=2146199) 'total_training_steps': 100, (TaskRunner pid=2146199) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2146199) 'val_before_train': False, (TaskRunner pid=2146199) 'val_only': False, (TaskRunner pid=2146199) 'validation_data_dir': None}, (TaskRunner pid=2146199) 'transfer_queue': {'enable': False}, (TaskRunner pid=2146199) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/material', (TaskRunner pid=2146199) 'dir': '/users/root/SDPO', (TaskRunner pid=2146199) 'log_dir': '/users/root/output', (TaskRunner pid=2146199) 'task': 'datasets/sciknoweval/material'}} (TaskRunner pid=2146199) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2146199) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2146199) use_critic=need_critic(config), (TaskRunner pid=2146199) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2146199) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2146199) Using dataset class: RLHFDataset (TaskRunner pid=2146199) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2146199) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2146199) dataset len: 841 (TaskRunner pid=2146199) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2146199) WARNING:2026-07-02 12:53:57,322:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2146199) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/841 [00:00 (TaskRunner pid=2146199) bind role actor_rollout method chat_completion to class .WorkerDict'> (TaskRunner pid=2146199) bind role actor_rollout method generate to class .WorkerDict'> (TaskRunner pid=2146199) bind role actor_rollout method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2146199) bind role actor_rollout method sleep to class .WorkerDict'> (TaskRunner pid=2146199) bind role actor_rollout method wake_up to class .WorkerDict'> (TaskRunner pid=2146199) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 94/94 [00:00<00:00, 149.22 examples/s] (TaskRunner pid=2146199) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2146199) self.use_critic = need_critic(self.config) (WorkerDict pid=2146579) [W702 12:54:06.660994833 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:50537 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2146579) [Gloo] Rank 2 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2146579) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2146579) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2146583) [W702 12:54:06.312883846 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:50537 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2146577) Model config after override: Qwen3Config { (WorkerDict pid=2146577) "architectures": [ (WorkerDict pid=2146577) "Qwen3ForCausalLM" (WorkerDict pid=2146577) ], (WorkerDict pid=2146577) "attention_bias": false, (WorkerDict pid=2146577) "attention_dropout": 0.0, (WorkerDict pid=2146577) "dtype": "bfloat16", (WorkerDict pid=2146577) "eos_token_id": 151645, (WorkerDict pid=2146577) "head_dim": 128, (WorkerDict pid=2146577) "hidden_act": "silu", (WorkerDict pid=2146577) "hidden_size": 2560, (WorkerDict pid=2146577) "initializer_range": 0.02, (WorkerDict pid=2146577) "intermediate_size": 9728, (WorkerDict pid=2146577) "layer_types": [ (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention" (WorkerDict pid=2146577) ], (WorkerDict pid=2146577) "max_position_embeddings": 40960, (WorkerDict pid=2146577) "max_window_layers": 36, (WorkerDict pid=2146577) "model_type": "qwen3", (WorkerDict pid=2146577) "num_attention_heads": 32, (WorkerDict pid=2146577) "num_hidden_layers": 36, (WorkerDict pid=2146577) "num_key_value_heads": 8, (WorkerDict pid=2146577) "pad_token_id": 151643, (WorkerDict pid=2146577) "rms_norm_eps": 1e-06, (WorkerDict pid=2146577) "rope_scaling": null, (WorkerDict pid=2146577) "rope_theta": 1000000, (WorkerDict pid=2146577) "sliding_window": null, (WorkerDict pid=2146577) "tie_word_embeddings": true, (WorkerDict pid=2146577) "transformers_version": "4.57.1", (WorkerDict pid=2146577) "use_cache": true, (WorkerDict pid=2146577) "use_sliding_window": false, (WorkerDict pid=2146577) "vocab_size": 151936 (WorkerDict pid=2146577) } (WorkerDict pid=2146577) (WorkerDict pid=2146579) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2146579) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2146579) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2146577) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2146577) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2146579) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2146579) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2146584) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.91s/it] [repeated 13x across cluster] (WorkerDict pid=2146581) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.94s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.98s/it] (WorkerDict pid=2146584) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.96s/it] [repeated 5x across cluster] (WorkerDict pid=2146577) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2146577) Actor use_remove_padding=True (WorkerDict pid=2146577) Actor use_fused_kernels=False (WorkerDict pid=2146577) Actor use_prefix_grouper=False (WorkerDict pid=2146581) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2146581) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2146579) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2146579) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2146578) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2146581) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2146581) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2146579) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2146579) warnings.warn( (WorkerDict pid=2146581) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster] (WorkerDict pid=2146581) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster] (TaskRunner pid=2146199) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2146199) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2146584) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster] (WorkerDict pid=2146584) warnings.warn( [repeated 7x across cluster] (vLLMHttpServer pid=2147446) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2147446) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2147445) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2147445) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2147446) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2147446) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2147445) ['serve', (vLLMHttpServer pid=2147445) 'Qwen/Qwen3-4B', (vLLMHttpServer pid=2147445) '--dtype', (vLLMHttpServer pid=2147445) 'bfloat16', (vLLMHttpServer pid=2147445) '--load_format', (vLLMHttpServer pid=2147445) 'dummy', (vLLMHttpServer pid=2147445) '--trust_remote_code', (vLLMHttpServer pid=2147445) '--max_model_len', (vLLMHttpServer pid=2147445) '40960', (vLLMHttpServer pid=2147445) '--max_num_seqs', (vLLMHttpServer pid=2147445) '1024', (vLLMHttpServer pid=2147445) '--enable_chunked_prefill', (vLLMHttpServer pid=2147445) '--max_num_batched_tokens', (vLLMHttpServer pid=2147445) '10240', (vLLMHttpServer pid=2147445) '--enable_prefix_caching', (vLLMHttpServer pid=2147445) '--enable_sleep_mode', (vLLMHttpServer pid=2147445) '--logprobs_mode', (vLLMHttpServer pid=2147445) 'processed_logprobs', (vLLMHttpServer pid=2147445) '--disable_custom_all_reduce', (vLLMHttpServer pid=2147445) '--gpu_memory_utilization', (vLLMHttpServer pid=2147445) '0.8', (vLLMHttpServer pid=2147445) '--disable_log_stats', (vLLMHttpServer pid=2147445) '--tensor_parallel_size', (vLLMHttpServer pid=2147445) '2', (vLLMHttpServer pid=2147445) '--seed', (vLLMHttpServer pid=2147445) '0', (vLLMHttpServer pid=2147445) '--override_generation_config', (vLLMHttpServer pid=2147445) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2147445) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2147445) '--hf_overrides', (vLLMHttpServer pid=2147445) '{}', (vLLMHttpServer pid=2147445) '--scheduling_policy', (vLLMHttpServer pid=2147445) 'fcfs'] (WorkerDict pid=2146583) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (vLLMHttpServer pid=2147446) WARNING 07-02 12:54:49 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2146579) WARNING 07-02 12:54:57 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2147448) WARNING 07-02 12:54:49 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2146579) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.770000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.770000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last): (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.770000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.770000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f: (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.770000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.770000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/aotautograd/ayamvxbmfo3ie54kyeiv7wcyjafynjwj3y4mmjujg4hvg4mrrnne/.2146580.130521564378816.tmp' (vLLMHttpServer pid=2147448) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 2x across cluster] (vLLMHttpServer pid=2147448) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 2x across cluster] (vLLMHttpServer pid=2147448) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2147448) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.925000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.925000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last): (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.925000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.925000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f: (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.925000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.925000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/fxgraph/vs/fvscv5dg7y3y3dcup4ddfpgbmpglg73u5zi54vpf2q4orgekbtq7/.2146582.128752478582464.tmp' (WorkerDict pid=2146579) 2026-07-02 12:55:11,940 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2146579) 2026-07-02 12:55:11,960 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2146579) [rank2]:W0702 12:55:07.327000 2146579 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph (WorkerDict pid=2146579) [rank2]:W0702 12:55:07.327000 2146579 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last): (WorkerDict pid=2146579) [rank2]:W0702 12:55:07.327000 2146579 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates (WorkerDict pid=2146579) [rank2]:W0702 12:55:07.327000 2146579 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f: (WorkerDict pid=2146579) [rank2]:W0702 12:55:07.327000 2146579 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (WorkerDict pid=2146579) [rank2]:W0702 12:55:07.327000 2146579 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/fxgraph/vs/fvscv5dg7y3y3dcup4ddfpgbmpglg73u5zi54vpf2q4orgekbtq7/.2146580.130521564378816.tmp' (WorkerDict pid=2146577) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2146199) ... (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) ... (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2146199) (TaskRunner pid=2146199) A (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) user (TaskRunner pid=2146199) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2146199) (TaskRunner pid=2146199) Lattice Parameters: (TaskRunner pid=2146199) (TaskRunner pid=2146199) a = 7.82 Å (TaskRunner pid=2146199) b = 7.82 Å (TaskRunner pid=2146199) c = 17.88 Å (TaskRunner pid=2146199) α = 90.00º (TaskRunner pid=2146199) β = 90.00º (TaskRunner pid=2146199) γ = 90.00º (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) A: 1025.67 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) B: 1094.31 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) C: 1200.45 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) D: 1150.78 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) Please reason step by step. (TaskRunner pid=2146199) assistant (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) [response] (TaskRunner pid=2146199) The volume of a unit cell in a crystal structure can be calculated using the formula: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ V = a \times b \times c $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Given the lattice parameters: (TaskRunner pid=2146199) - $ a = 7.82 $ Å (TaskRunner pid=2146199) - $ b = 7.82 $ Å (TaskRunner pid=2146199) - $ c = 17.88 $ Å (TaskRunner pid=2146199) (TaskRunner pid=2146199) Since all angles (α, β, γ) are 90.00º, the crystal structure is cubic, and the volume is simply the product of the three lattice parameters. (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ V = 7.82 \times 7.82 \times 17.88 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ 7.82 \times 7.82 = 61.1524 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Then multiply by 17.88: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ 61.1524 \times 17.88 \approx 1094.31 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Thus, the volume of the unit cell is approximately 1094.31 ų. (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) B (TaskRunner pid=2146199) (TaskRunner pid=2146199) [ground_truth] B (TaskRunner pid=2146199) [score] 1.0 (TaskRunner pid=2146199) [acc] 1.0 (TaskRunner pid=2146199) [pred] B (TaskRunner pid=2146199) [incorrect_format] 1 (TaskRunner pid=2146199) [feedback]