Files
ablation-pymethods2test-seq…/training_logs/ablation-pymethods2test-seqnorm_639476.out
ModelHub XC 67edc8d5a6 初始化项目,由ModelHub XC社区提供模型
Model: laion/ablation-pymethods2test-seqnorm-15-8B
Source: Original Platform
2026-08-08 06:25:19 +08:00

8686 lines
676 KiB
Plaintext
Raw Permalink Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

The following have been reloaded with a version change:
1) GCCcore/.14.3.0 => GCCcore/14.3.0
Lmod is automatically replacing "GCC/14.3.0" with
"nvidia-compilers/25.9-CUDA-13".
Deactivating conda environment: /e/scratch/jureap59/feuer1/miniforge3/envs/otagent
Activating RL environment: /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl
Python executable: /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/bin/python
Python path check: /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/bin/python
[ray] RAY_TMPDIR=/tmp/ray/ray_639476
[triton_cache] Triton cache: /tmp/triton_cache_feuer1_639476
[triton_cache] TorchInductor cache: /tmp/torchinductor_cache_feuer1_639476
[proxy] ✓ Found proxychains binary at /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4
[proxy] Setting up SSH tunnel to jpbl-s01-01
[proxy] SSH key: /e/home/jusers/feuer1/jupiter/.ssh/authorized_keys/id_ed25519_jsc
[proxy] Tunnel port: 7003
[proxy] Node IP: 10.128.32.49 (workers will connect here)
[proxy] ✓ SSH tunnel started successfully
[proxy] ✓ Generated proxychains config at /e/home/jusers/feuer1/jupiter/.proxychains/proxychains_639476.conf
[proxy] - Internal traffic (10.x.x.x, 172.x.x.x, 169.254.x.x) → DIRECT
[proxy] - External traffic (internet) → PROXY via tunnel
[proxy] ✓ Daytona timeout settings configured
[proxy] Testing proxy connectivity...
[proxychains] config file found: /e/home/jusers/feuer1/jupiter/.proxychains/proxychains_639476.conf
[proxychains] preloading /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/lib/libproxychains4.so
[proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e
[proxy] ✓ Proxy connectivity test passed (huggingface.co reachable via wrapped binary)
[proxy] ⚠ Tunnel not accessible at 10.128.32.49:7003 (workers may fail)
[proxy] ✓ Proxy setup complete (using wrapped binary for Ray workers)
[container_runtime] Using cloud backend: daytona (no local container setup)
=== Universal RL Training Runner ===
Config: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/configs/ablation-pymethods2test-seqnorm_rl_config.json
Working directory: /e/scratch/jureap59/feuer1/OpenThoughts-Agent
Python: /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/bin/python
Python version: Python 3.12.12
UV_USE_IO_URING: 0
Proxy: DISABLED (direct internet or not configured)
========================================
=== RLJobRunner: ablation-pymethods2test-seqnorm ===
[wandb_utils] Fixing permissions on: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/wandb
[wandb_utils] WandB directory ready: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/wandb
HF_TOKEN=****pDbg
HF_HUB_CACHE=/e/data1/datasets/playground/ot-baf/hf_hub
SUPABASE_URL=https://rpzmyuapoqilpghynmza.s... (direct Supabase config)
Environment configured:
TENSOR_PARALLEL_SIZE=1
NUM_INFERENCE_ENGINES=56
POLICY_NUM_NODES=14
WANDB_DIR=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/wandb
Starting Ray cluster with 14 nodes, 4 GPUs/node
Cleaning up existing Ray instances...
=== Starting Ray Cluster ===
Nodes: 14
GPUs per node: 4
CPUs per node: 288
Head node: jpbo-042-01 (10.128.32.49)
Ray port: 6379
============================
Starting Ray head on jpbo-042-01 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_head_jpbo-042-01.log)...
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.49 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-01 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --head --node-ip-address=10.128.32.49 --port=6379 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
Started Ray head on jpbo-042-01
Starting Ray worker on jpbo-042-02 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-02.log)...
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.50 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-02 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.50 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
Started Ray worker 1 on jpbo-042-02
Starting Ray worker on jpbo-042-03 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-03.log)...
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.51 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-03 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.51 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
Started Ray worker 2 on jpbo-042-03
Starting Ray worker on jpbo-042-04 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-04.log)...
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.52 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-04 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.52 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
Started Ray worker 3 on jpbo-042-04
Starting Ray worker on jpbo-042-05 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-05.log)...
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.53 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-05 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.53 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
Started Ray worker 4 on jpbo-042-05
Starting Ray worker on jpbo-042-06 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-06.log)...
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.54 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-06 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.54 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
Started Ray worker 5 on jpbo-042-06
Starting Ray worker on jpbo-042-07 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-07.log)...
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.55 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-07 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.55 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
Started Ray worker 6 on jpbo-042-07
Starting Ray worker on jpbo-042-08 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-08.log)...
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.56 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-08 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.56 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
Started Ray worker 7 on jpbo-042-08
Starting Ray worker on jpbo-042-09 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-09.log)...
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.57 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-09 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.57 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
Started Ray worker 8 on jpbo-042-09
Starting Ray worker on jpbo-042-10 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-10.log)...
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.58 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-10 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.58 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
Started Ray worker 9 on jpbo-042-10
Starting Ray worker on jpbo-042-11 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-11.log)...
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.59 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-11 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.59 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
Started Ray worker 10 on jpbo-042-11
Starting Ray worker on jpbo-042-12 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-12.log)...
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.60 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-12 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.60 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
Started Ray worker 11 on jpbo-042-12
Starting Ray worker on jpbo-042-13 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-13.log)...
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.61 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-13 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.61 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
Started Ray worker 12 on jpbo-042-13
Starting Ray worker on jpbo-042-14 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-14.log)...
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.62 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-14 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.62 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
Started Ray worker 13 on jpbo-042-14
Waiting for cluster (56 GPUs, 14 nodes)...
Connecting to Ray at 10.128.32.49:6379 (expecting 14 nodes, 56.0 GPUs)
Ray connection established, polling for resources...
[Ray wait] nodes=14/14 GPUs=56.0/56.0 resources={'object_store_memory': 601295421440.0, 'CPU': 4032.0, 'accelerator_type:GH200': 14.0, 'node:10.128.32.50': 1.0, 'memory': 10479941910528.0, 'GPU': 56.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.60': 1.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.55': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.52': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.49': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.54': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.53': 1.0, 'node:10.128.32.56': 1.0}
✓ Ray cluster ready
=== Ray Cluster Ready ===
Address: 10.128.32.49:6379
Total GPUs: 56
=========================
Ray cluster ready at 10.128.32.49:6379
Total GPUs available: 56
[RLJobRunner] Pinggy check: url=False, token=False, needs_tunnel=False (agent=terminus-2, env=daytona)
[RLJobRunner] No Pinggy tunnel needed, using local vLLM
Running SkyRL:
Python: /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/bin/python
Entrypoint: examples.terminal_bench.entrypoints.main_tbench
Args: 120 Hydra arguments
Working dir: /e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train
Using proxychains binary: /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4
Executing command with srun: /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f $PROXYCHAINS_CONF_FILE /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/bin/python -m examples.terminal_bench.entrypoints.main_tbench +terminal_bench_config=terminal_bench trainer.strategy=fsdp2 trainer.algorithm.advantage_estimator=rloo_n trainer.algorithm.use_kl_loss=false trainer.algorithm.kl_loss_coef=0.0 trainer.algorithm.eps_clip_low=0.2 trainer.algorithm.eps_clip_high=0.05 trainer.algorithm.loss_reduction=seq_mean_token_sum_norm_global trainer.epochs=2 trainer.max_steps=80 trainer.update_epochs_per_batch=1 trainer.train_batch_size=64 trainer.policy_mini_batch_size=64 trainer.eval_batch_size=64 trainer.micro_forward_batch_size_per_gpu=4 trainer.micro_train_batch_size_per_gpu=1 trainer.max_prompt_length=999999 trainer.eval_interval=999999 trainer.eval_before_train=false trainer.ckpt_interval=2 trainer.resume_mode=latest trainer.hf_save_interval=5 ++trainer.hf_hub_repo_id=laion/ablation-pymethods2test-seqnorm ++trainer.hf_hub_private=false ++trainer.hf_hub_revision=main ++trainer.enable_db_registration=false trainer.project_name=OpenThoughts-Agent trainer.log_level=INFO trainer.tracker_commit_each_step=true trainer.logger=console trainer.run_name=ablation-pymethods2test-seqnorm trainer.ckpt_path=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/ablation-pymethods2test-seqnorm/checkpoints trainer.export_path=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/ablation-pymethods2test-seqnorm/exports trainer.policy.optimizer_config.lr=8e-6 trainer.policy.optimizer_config.weight_decay=0.0 trainer.policy.optimizer_config.adam_betas=[0.9,0.999] trainer.policy.optimizer_config.max_grad_norm=0.9 trainer.policy.fsdp_config.cpu_offload=false trainer.policy.fsdp_config.reshard_after_forward=true trainer.policy.fsdp_config.fsdp_size=4 trainer.policy.model.path=/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6 trainer.ref.fsdp_config.cpu_offload=false trainer.ref.fsdp_config.reshard_after_forward=true trainer.ref.fsdp_config.fsdp_size=4 trainer.placement.colocate_all=false trainer.placement.policy_num_nodes=2 trainer.placement.ref_num_nodes=2 trainer.placement.policy_num_gpus_per_node=4 trainer.placement.ref_num_gpus_per_node=4 trainer.fully_async.max_staleness_steps=16 trainer.fully_async.num_parallel_generation_workers=338 generator.backend=vllm generator.timeout_multiplier=1.0 generator.model_dtype=bfloat16 generator.inference_engine_tensor_parallel_size=1 generator.num_inference_engines=48 generator.n_samples_per_prompt=8 generator.eval_n_samples_per_prompt=8 generator.gpu_memory_utilization=0.75 generator.max_num_seqs=24 generator.max_num_batched_tokens=65536 generator.enable_prefix_caching=true generator.enable_chunked_prefill=true generator.run_engines_locally=true generator.weight_sync_backend=nccl generator.async_engine=true generator.batched=false generator.enable_http_endpoint=true generator.enable_ray_prometheus_stats=false generator.vllm_stats_interval=1 generator.append_eos_token_after_stop_str_in_multi_turn=true generator.max_turns=999999 generator.sampling_params.max_generate_length=4096 generator.sampling_params.temperature=0.7 generator.sampling_params.top_p=0.95 generator.sampling_params.top_k=20 ++generator.engine_init_kwargs.max_model_len=32768 ++generator.engine_init_kwargs.custom_chat_template_chat_completion_path=chat_templates/qwen3_thinking_acc.jinja2 ++generator.engine_init_kwargs.served_model_name=0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6 data.train_data=["/e/scratch/jureap59/feuer1/tasks/exp_rpt_pymethods2test-large"] data.val_data=[] +terminal_bench_config.trials_dir=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/ablation-pymethods2test-seqnorm/trace_jobs +terminal_bench_config.harbor.name=terminus-2 +terminal_bench_config.harbor.max_episodes=999999 +terminal_bench_config.harbor.enable_summarize=false +terminal_bench_config.harbor.store_all_messages=true +terminal_bench_config.harbor.trajectory_config.raw_content=true +terminal_bench_config.harbor.enable_episode_logging=false +terminal_bench_config.harbor.record_terminal_session=false +terminal_bench_config.harbor.enable_pane_logging=false +terminal_bench_config.harbor.strict_json_parser=true +terminal_bench_config.harbor.interleaved_thinking=true +terminal_bench_config.harbor.extra_body.chat_template_kwargs.enable_thinking=true +terminal_bench_config.harbor.override_timeout_sec=900 +terminal_bench_config.harbor.override_cpus=1 +terminal_bench_config.harbor.override_memory_mb=2048 +terminal_bench_config.harbor.override_storage_mb=2048 +terminal_bench_config.harbor.auto_snapshot=true +terminal_bench_config.harbor.verifier_override_timeout_sec=120 +terminal_bench_config.harbor.max_retries=3 +terminal_bench_config.harbor.min_wait_sec=60.0 +terminal_bench_config.harbor.max_wait_sec=600.0 +terminal_bench_config.harbor.wait_multiplier=2.0 +terminal_bench_config.harbor.exclude_exceptions=["VerifierTimeoutError","VerifierRuntimeError","RewardFileNotFoundError","RewardFileEmptyError","VerifierOutputParseError"] +terminal_bench_config.harbor.n_concurrent_trials=675 +terminal_bench_config.harbor.log_level=INFO +terminal_bench_config.harbor.enable_reward_shaping=false +terminal_bench_config.harbor.enable_error_classification=true +terminal_bench_config.harbor.mask_exceptions=["DaytonaError","EnvironmentStartTimeoutError","NetworkError","ConnectionError","RewardFileNotFoundError","RewardFileEmptyError","AgentEnvironmentTimeoutError","ContextLengthExceededError"] +terminal_bench_config.harbor.default_error_treatment=zero +terminal_bench_config.harbor.passthrough_exceptions=["AgentTimeoutError"] +terminal_bench_config.harbor.zero_exceptions=[] +terminal_bench_config.model_info.max_input_tokens=32000 +terminal_bench_config.model_info.max_output_tokens=4096 +terminal_bench_config.archiving.enabled=false +terminal_bench_config.trace_upload.enabled=true +terminal_bench_config.trace_upload.repo_org=DCAgent +terminal_bench_config.trace_upload.episodes=last +terminal_bench_config.trace_upload.dataset_type=SFT +terminal_bench_config.trace_upload.cleanup=true
[proxychains] config file found: /e/home/jusers/feuer1/jupiter/.proxychains/proxychains_639476.conf
[proxychains] preloading /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/lib/libproxychains4.so
[proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e
2026-06-08 20:41:23.641 | INFO | skyrl_train.utils.utils:prepare_runtime_environment:765 - Exporting wandb api key to ray runtime env
2026-06-08 20:41:23.642 | INFO | skyrl_train.utils.utils:prepare_runtime_environment:784 - Exporting RAY_ADDRESS to ray runtime env
2026-06-08 20:41:23.642 | INFO | skyrl_train.utils.utils:prepare_runtime_environment:809 - Exporting `NCCL_SOCKET_IFNAME` to ray runtime env: ib0
2026-06-08 20:41:23.642 | INFO | skyrl_train.utils.utils:prepare_runtime_environment:809 - Exporting `NCCL_SOCKET_FAMILY` to ray runtime env: AF_INET
2026-06-08 20:41:23.642 | INFO | skyrl_train.utils.utils:prepare_runtime_environment:809 - Exporting `NCCL_DEBUG` to ray runtime env: WARN
2026-06-08 20:41:23,642 INFO worker.py:1680 -- Using address 10.128.32.49:6379 set in the environment variable RAY_ADDRESS
2026-06-08 20:41:23,675 INFO worker.py:1821 -- Connecting to existing Ray cluster at address: 10.128.32.49:6379...
2026-06-08 20:41:23,686 INFO worker.py:2007 -- Connected to Ray cluster.
/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
warnings.warn(
(raylet, ip=10.128.32.54) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e
2026-06-08 20:41:25.873 | INFO | skyrl_train.utils.ppo_utils:sync_registries:546 - Synced registries to ray actor
(pid=530083) [2026-06-08 20:41:27,275 E 530083 530758] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14
(raylet) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 17x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.)
(skyrl_entrypoint pid=546644) 2026-06-08 20:41:32.678 | INFO  | skyrl_train.entrypoints.main_base:_configure_log_level:214 - SkyRL log level set to: INFO
(pid=530096) [2026-06-08 20:41:29,941 E 530096 531589] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 [repeated 17x across cluster]
(skyrl_entrypoint pid=546644) The tokenizer you are loading from '/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
(skyrl_entrypoint pid=546644) 2026-06-08 20:41:32.962 | INFO  | examples.terminal_bench.dataset:_load_data_files:40 - Loading data from: /e/scratch/jureap59/feuer1/tasks/exp_rpt_pymethods2test-large
(skyrl_entrypoint pid=546644) 2026-06-08 20:41:34.762 | INFO  | examples.terminal_bench.dataset:_load_data_files:50 - Found 5000 valid task directories out of 5000 total directories
(skyrl_entrypoint pid=546644) 2026-06-08 20:41:34.763 | INFO  | examples.terminal_bench.dataset:__init__:27 - TerminalBenchTaskDataset initialized with 5000 task paths
(skyrl_entrypoint pid=546644) 2026-06-08 20:41:34.775 | INFO  | skyrl_train.entrypoints.main_base:_setup_trainer:407 - data:
(skyrl_entrypoint pid=546644) train_data:
(skyrl_entrypoint pid=546644) - /e/scratch/jureap59/feuer1/tasks/exp_rpt_pymethods2test-large
(skyrl_entrypoint pid=546644) val_data: []
(skyrl_entrypoint pid=546644) trainer:
(skyrl_entrypoint pid=546644) placement:
(skyrl_entrypoint pid=546644) colocate_all: false
(skyrl_entrypoint pid=546644) colocate_policy_ref: true
(skyrl_entrypoint pid=546644) policy_num_nodes: 2
(skyrl_entrypoint pid=546644) policy_num_gpus_per_node: 4
(skyrl_entrypoint pid=546644) critic_num_nodes: 1
(skyrl_entrypoint pid=546644) critic_num_gpus_per_node: 4
(skyrl_entrypoint pid=546644) ref_num_nodes: 2
(skyrl_entrypoint pid=546644) ref_num_gpus_per_node: 4
(skyrl_entrypoint pid=546644) policy_strict_spread_pg: false
(skyrl_entrypoint pid=546644) policy_per_gpu_bundles: false
(skyrl_entrypoint pid=546644) policy_force_cvd_mask: false
(skyrl_entrypoint pid=546644) sequence_parallel_backend: ulysses
(skyrl_entrypoint pid=546644) strategy: fsdp2
(skyrl_entrypoint pid=546644) policy:
(skyrl_entrypoint pid=546644) model:
(skyrl_entrypoint pid=546644) path: /e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6
(skyrl_entrypoint pid=546644) lora:
(skyrl_entrypoint pid=546644) rank: 0
(skyrl_entrypoint pid=546644) alpha: 16
(skyrl_entrypoint pid=546644) dropout: 0
(skyrl_entrypoint pid=546644) lora_sync_path: /tmp/skyrl_lora_sync
(skyrl_entrypoint pid=546644) target_modules: all-linear
(skyrl_entrypoint pid=546644) exclude_modules: null
(skyrl_entrypoint pid=546644) deepspeed_config: ${deepspeed_config.train}
(skyrl_entrypoint pid=546644) optimizer_config:
(skyrl_entrypoint pid=546644) optimizer: AdamW
(skyrl_entrypoint pid=546644) lr: 8.0e-06
(skyrl_entrypoint pid=546644) adam_betas:
(skyrl_entrypoint pid=546644) - 0.9
(skyrl_entrypoint pid=546644) - 0.999
(skyrl_entrypoint pid=546644) weight_decay: 0.0
(skyrl_entrypoint pid=546644) max_grad_norm: 0.9
(skyrl_entrypoint pid=546644) offload_after_step: true
(skyrl_entrypoint pid=546644) num_warmup_steps: 0
(skyrl_entrypoint pid=546644) scheduler: constant_with_warmup
(skyrl_entrypoint pid=546644) optimizer_kwargs: {}
(skyrl_entrypoint pid=546644) fsdp_config:
(skyrl_entrypoint pid=546644) cpu_offload: false
(skyrl_entrypoint pid=546644) reshard_after_forward: true
(skyrl_entrypoint pid=546644) fsdp_size: 4
(skyrl_entrypoint pid=546644) expert_model_parallel_size: 1
(skyrl_entrypoint pid=546644) expert_tensor_parallel_size: 1
(skyrl_entrypoint pid=546644) moe_token_dispatcher_type: alltoall
(skyrl_entrypoint pid=546644) moe_router_replay: false
(skyrl_entrypoint pid=546644) moe_grouped_gemm: false
(skyrl_entrypoint pid=546644) ep_comm_backend: torch
(skyrl_entrypoint pid=546644) deepep_num_sms: 20
(skyrl_entrypoint pid=546644) deepep_token_chunk_size: null
(skyrl_entrypoint pid=546644) sequence_parallel_size: 1
(skyrl_entrypoint pid=546644) use_torch_compile: false
(skyrl_entrypoint pid=546644) record_memory: false
(skyrl_entrypoint pid=546644) megatron_config:
(skyrl_entrypoint pid=546644) tensor_model_parallel_size: 1
(skyrl_entrypoint pid=546644) pipeline_model_parallel_size: 1
(skyrl_entrypoint pid=546644) context_parallel_size: 1
(skyrl_entrypoint pid=546644) expert_model_parallel_size: 1
(skyrl_entrypoint pid=546644) expert_tensor_parallel_size: null
(skyrl_entrypoint pid=546644) ddp_config:
(skyrl_entrypoint pid=546644) grad_reduce_in_fp32: true
(skyrl_entrypoint pid=546644) overlap_grad_reduce: false
(skyrl_entrypoint pid=546644) overlap_param_gather: false
(skyrl_entrypoint pid=546644) average_in_collective: true
(skyrl_entrypoint pid=546644) model_config_kwargs: {}
(skyrl_entrypoint pid=546644) torch_profiler_config:
(skyrl_entrypoint pid=546644) enable: false
(skyrl_entrypoint pid=546644) ranks: []
(skyrl_entrypoint pid=546644) save_path: null
(skyrl_entrypoint pid=546644) optimizer_config_kwargs:
(skyrl_entrypoint pid=546644) overlap_cpu_optimizer_d2h_h2d: false
(skyrl_entrypoint pid=546644) use_precision_aware_optimizer: false
(skyrl_entrypoint pid=546644) optimizer_cpu_offload: false
(skyrl_entrypoint pid=546644) optimizer_offload_fraction: 0.0
(skyrl_entrypoint pid=546644) transformer_config_kwargs:
(skyrl_entrypoint pid=546644) recompute_granularity: full
(skyrl_entrypoint pid=546644) recompute_modules:
(skyrl_entrypoint pid=546644) - core_attn
(skyrl_entrypoint pid=546644) recompute_method: uniform
(skyrl_entrypoint pid=546644) recompute_num_layers: 1
(skyrl_entrypoint pid=546644) empty_cuda_cache: true
(skyrl_entrypoint pid=546644) ref:
(skyrl_entrypoint pid=546644) model:
(skyrl_entrypoint pid=546644) path: ${trainer.policy.model.path}
(skyrl_entrypoint pid=546644) sequence_parallel_size: 1
(skyrl_entrypoint pid=546644) deepspeed_config: ${deepspeed_config.eval}
(skyrl_entrypoint pid=546644) fsdp_config:
(skyrl_entrypoint pid=546644) cpu_offload: false
(skyrl_entrypoint pid=546644) reshard_after_forward: true
(skyrl_entrypoint pid=546644) fsdp_size: 4
(skyrl_entrypoint pid=546644) expert_model_parallel_size: 1
(skyrl_entrypoint pid=546644) expert_tensor_parallel_size: 1
(skyrl_entrypoint pid=546644) moe_token_dispatcher_type: alltoall
(skyrl_entrypoint pid=546644) moe_router_replay: false
(skyrl_entrypoint pid=546644) moe_grouped_gemm: false
(skyrl_entrypoint pid=546644) ep_comm_backend: torch
(skyrl_entrypoint pid=546644) deepep_num_sms: 20
(skyrl_entrypoint pid=546644) deepep_token_chunk_size: null
(skyrl_entrypoint pid=546644) megatron_config:
(skyrl_entrypoint pid=546644) tensor_model_parallel_size: 1
(skyrl_entrypoint pid=546644) pipeline_model_parallel_size: 1
(skyrl_entrypoint pid=546644) context_parallel_size: 1
(skyrl_entrypoint pid=546644) expert_model_parallel_size: 1
(skyrl_entrypoint pid=546644) expert_tensor_parallel_size: 1
(skyrl_entrypoint pid=546644) model_config_kwargs: {}
(skyrl_entrypoint pid=546644) transformer_config_kwargs: {}
(skyrl_entrypoint pid=546644) critic:
(skyrl_entrypoint pid=546644) model:
(skyrl_entrypoint pid=546644) path: null
(skyrl_entrypoint pid=546644) lora:
(skyrl_entrypoint pid=546644) rank: 0
(skyrl_entrypoint pid=546644) alpha: 16
(skyrl_entrypoint pid=546644) dropout: 0
(skyrl_entrypoint pid=546644) target_modules: all-linear
(skyrl_entrypoint pid=546644) exclude_modules: null
(skyrl_entrypoint pid=546644) deepspeed_config: ${deepspeed_config.train}
(skyrl_entrypoint pid=546644) optimizer_config:
(skyrl_entrypoint pid=546644) optimizer: AdamW
(skyrl_entrypoint pid=546644) lr: 5.0e-06
(skyrl_entrypoint pid=546644) adam_betas:
(skyrl_entrypoint pid=546644) - 0.9
(skyrl_entrypoint pid=546644) - 0.999
(skyrl_entrypoint pid=546644) weight_decay: 0.01
(skyrl_entrypoint pid=546644) max_grad_norm: 1.0
(skyrl_entrypoint pid=546644) offload_after_step: true
(skyrl_entrypoint pid=546644) num_warmup_steps: 0
(skyrl_entrypoint pid=546644) scheduler: constant_with_warmup
(skyrl_entrypoint pid=546644) optimizer_kwargs: {}
(skyrl_entrypoint pid=546644) fsdp_config:
(skyrl_entrypoint pid=546644) cpu_offload: false
(skyrl_entrypoint pid=546644) reshard_after_forward: true
(skyrl_entrypoint pid=546644) fsdp_size: -1
(skyrl_entrypoint pid=546644) expert_model_parallel_size: 1
(skyrl_entrypoint pid=546644) expert_tensor_parallel_size: 1
(skyrl_entrypoint pid=546644) moe_token_dispatcher_type: alltoall
(skyrl_entrypoint pid=546644) moe_router_replay: false
(skyrl_entrypoint pid=546644) moe_grouped_gemm: false
(skyrl_entrypoint pid=546644) ep_comm_backend: torch
(skyrl_entrypoint pid=546644) deepep_num_sms: 20
(skyrl_entrypoint pid=546644) deepep_token_chunk_size: null
(skyrl_entrypoint pid=546644) sequence_parallel_size: 1
(skyrl_entrypoint pid=546644) algorithm:
(skyrl_entrypoint pid=546644) advantage_estimator: rloo_n
(skyrl_entrypoint pid=546644) kl_ctrl:
(skyrl_entrypoint pid=546644) type: fixed
(skyrl_entrypoint pid=546644) kl_target: 0.1
(skyrl_entrypoint pid=546644) horizon: 10000
(skyrl_entrypoint pid=546644) kl_estimator_type: k3
(skyrl_entrypoint pid=546644) use_kl_estimator_k3: false
(skyrl_entrypoint pid=546644) use_abs_kl: false
(skyrl_entrypoint pid=546644) use_kl_in_reward: false
(skyrl_entrypoint pid=546644) use_kl_loss: false
(skyrl_entrypoint pid=546644) kl_loss_coef: 0.0
(skyrl_entrypoint pid=546644) use_entropy_loss: false
(skyrl_entrypoint pid=546644) entropy_loss_coef: 0.01
(skyrl_entrypoint pid=546644) advantage_batch_normalize: false
(skyrl_entrypoint pid=546644) value_head_prefix: value_head
(skyrl_entrypoint pid=546644) policy_loss_type: regular
(skyrl_entrypoint pid=546644) loss_reduction: seq_mean_token_sum_norm_global
(skyrl_entrypoint pid=546644) global_loss_denom: null
(skyrl_entrypoint pid=546644) grpo_norm_by_std: true
(skyrl_entrypoint pid=546644) rloo_n_min_group_size: 4
(skyrl_entrypoint pid=546644) rloo_n_filter_zero_reward_groups: true
(skyrl_entrypoint pid=546644) lambd: 1.0
(skyrl_entrypoint pid=546644) gamma: 1.0
(skyrl_entrypoint pid=546644) eps_clip_low: 0.2
(skyrl_entrypoint pid=546644) eps_clip_high: 0.05
(skyrl_entrypoint pid=546644) clip_ratio_c: 3.0
(skyrl_entrypoint pid=546644) tis_imp_ratio_cap: -1.0
(skyrl_entrypoint pid=546644) use_tis: false
(skyrl_entrypoint pid=546644) sapo:
(skyrl_entrypoint pid=546644) tau_pos: 1.0
(skyrl_entrypoint pid=546644) tau_neg: 1.05
(skyrl_entrypoint pid=546644) value_clip: 0.2
(skyrl_entrypoint pid=546644) dynamic_sampling:
(skyrl_entrypoint pid=546644) type: null
(skyrl_entrypoint pid=546644) max_sample_batches: 30
(skyrl_entrypoint pid=546644) min_replace_ratio: 0.3
(skyrl_entrypoint pid=546644) clip_cov:
(skyrl_entrypoint pid=546644) clip_ratio: 0.0002
(skyrl_entrypoint pid=546644) clip_cov_lb: 1.0
(skyrl_entrypoint pid=546644) clip_cov_ub: 5.0
(skyrl_entrypoint pid=546644) kl_cov:
(skyrl_entrypoint pid=546644) kl_cov_frac: 0.2
(skyrl_entrypoint pid=546644) ppo_kl_coef: 1.0
(skyrl_entrypoint pid=546644) cispo:
(skyrl_entrypoint pid=546644) cispo_eps_clip_low: 0
(skyrl_entrypoint pid=546644) cispo_eps_clip_high: 5
(skyrl_entrypoint pid=546644) z_clip:
(skyrl_entrypoint pid=546644) enabled: false
(skyrl_entrypoint pid=546644) alpha: 0.97
(skyrl_entrypoint pid=546644) z_thresh: 2.5
(skyrl_entrypoint pid=546644) warmup_steps: 25
(skyrl_entrypoint pid=546644) mode: zscore
(skyrl_entrypoint pid=546644) clip_option: adaptive_scaling
(skyrl_entrypoint pid=546644) clip_factor: 1.0
(skyrl_entrypoint pid=546644) skip_update_on_spike: false
(skyrl_entrypoint pid=546644) stale_clip:
(skyrl_entrypoint pid=546644) enabled: false
(skyrl_entrypoint pid=546644) alpha: 0.3
(skyrl_entrypoint pid=546644) entropy_threshold: 0.15
(skyrl_entrypoint pid=546644) entropy_window: 10
(skyrl_entrypoint pid=546644) min_lr_scale: 0.1
(skyrl_entrypoint pid=546644) max_seq_len: 1004095
(skyrl_entrypoint pid=546644) fully_async:
(skyrl_entrypoint pid=546644) max_staleness_steps: 16
(skyrl_entrypoint pid=546644) num_parallel_generation_workers: 338
(skyrl_entrypoint pid=546644) gradient_checkpointing: true
(skyrl_entrypoint pid=546644) gradient_checkpointing_use_reentrant: false
(skyrl_entrypoint pid=546644) seed: 42
(skyrl_entrypoint pid=546644) resume_mode: latest
(skyrl_entrypoint pid=546644) resume_path: null
(skyrl_entrypoint pid=546644) ckpt_path: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/ablation-pymethods2test-seqnorm/checkpoints
(skyrl_entrypoint pid=546644) max_ckpts_to_keep: -1
(skyrl_entrypoint pid=546644) ckpt_interval: 2
(skyrl_entrypoint pid=546644) hf_save_interval: 5
(skyrl_entrypoint pid=546644) hf_upload_mode: latest
(skyrl_entrypoint pid=546644) export_path: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/ablation-pymethods2test-seqnorm/exports
(skyrl_entrypoint pid=546644) bf16: true
(skyrl_entrypoint pid=546644) epochs: 2
(skyrl_entrypoint pid=546644) max_steps: 80
(skyrl_entrypoint pid=546644) update_epochs_per_batch: 1
(skyrl_entrypoint pid=546644) train_batch_size: 64
(skyrl_entrypoint pid=546644) policy_mini_batch_size: 64
(skyrl_entrypoint pid=546644) critic_mini_batch_size: 256
(skyrl_entrypoint pid=546644) micro_train_batch_size_per_gpu: 1
(skyrl_entrypoint pid=546644) micro_forward_batch_size_per_gpu: 4
(skyrl_entrypoint pid=546644) update_ref_every_epoch: false
(skyrl_entrypoint pid=546644) use_sample_packing: true
(skyrl_entrypoint pid=546644) eval_batch_size: 64
(skyrl_entrypoint pid=546644) eval_before_train: false
(skyrl_entrypoint pid=546644) eval_interval: 999999
(skyrl_entrypoint pid=546644) max_prompt_length: 999999
(skyrl_entrypoint pid=546644) flash_attn: true
(skyrl_entrypoint pid=546644) disable_fast_tokenizer: false
(skyrl_entrypoint pid=546644) target_modules: null
(skyrl_entrypoint pid=546644) exclude_modules: null
(skyrl_entrypoint pid=546644) project_name: OpenThoughts-Agent
(skyrl_entrypoint pid=546644) run_name: ablation-pymethods2test-seqnorm
(skyrl_entrypoint pid=546644) logger: console
(skyrl_entrypoint pid=546644) tracker_commit_each_step: true
(skyrl_entrypoint pid=546644) dump_data_batch: false
(skyrl_entrypoint pid=546644) dump_eval_results: true
(skyrl_entrypoint pid=546644) log_level: INFO
(skyrl_entrypoint pid=546644) rope_scaling: null
(skyrl_entrypoint pid=546644) rope_theta: null
(skyrl_entrypoint pid=546644) step_wise_training: false
(skyrl_entrypoint pid=546644) hf_hub_repo_id: laion/ablation-pymethods2test-seqnorm
(skyrl_entrypoint pid=546644) hf_hub_private: false
(skyrl_entrypoint pid=546644) hf_hub_revision: main
(skyrl_entrypoint pid=546644) enable_db_registration: false
(skyrl_entrypoint pid=546644) generator:
(skyrl_entrypoint pid=546644) model_name: ${trainer.policy.model.path}
(skyrl_entrypoint pid=546644) model_dtype: bfloat16
(skyrl_entrypoint pid=546644) timeout_multiplier: 1.0
(skyrl_entrypoint pid=546644) run_engines_locally: true
(skyrl_entrypoint pid=546644) num_inference_engines: 48
(skyrl_entrypoint pid=546644) backend: vllm
(skyrl_entrypoint pid=546644) weight_sync_backend: nccl
(skyrl_entrypoint pid=546644) fuse_weights: false
(skyrl_entrypoint pid=546644) weight_transfer_threshold_cuda_ipc_GB: 1.0
(skyrl_entrypoint pid=546644) inference_engine_tensor_parallel_size: 1
(skyrl_entrypoint pid=546644) inference_engine_pipeline_parallel_size: 1
(skyrl_entrypoint pid=546644) inference_engine_expert_parallel_size: 1
(skyrl_entrypoint pid=546644) inference_engine_data_parallel_size: 1
(skyrl_entrypoint pid=546644) inference_engine_mp_backend: false
(skyrl_entrypoint pid=546644) n_samples_per_prompt: 8
(skyrl_entrypoint pid=546644) async_engine: true
(skyrl_entrypoint pid=546644) batched: false
(skyrl_entrypoint pid=546644) max_input_length: ${trainer.max_prompt_length}
(skyrl_entrypoint pid=546644) vllm_v1_disable_multiproc: true
(skyrl_entrypoint pid=546644) enable_prefix_caching: true
(skyrl_entrypoint pid=546644) enable_chunked_prefill: true
(skyrl_entrypoint pid=546644) max_num_batched_tokens: 65536
(skyrl_entrypoint pid=546644) enforce_eager: true
(skyrl_entrypoint pid=546644) fully_sharded_loras: false
(skyrl_entrypoint pid=546644) enable_ray_prometheus_stats: false
(skyrl_entrypoint pid=546644) vllm_stats_interval: 1
(skyrl_entrypoint pid=546644) gpu_memory_utilization: 0.75
(skyrl_entrypoint pid=546644) max_num_seqs: 24
(skyrl_entrypoint pid=546644) remote_inference_engine_urls:
(skyrl_entrypoint pid=546644) - 127.0.0.1:8001
(skyrl_entrypoint pid=546644) enable_http_endpoint: true
(skyrl_entrypoint pid=546644) http_endpoint_host: 127.0.0.1
(skyrl_entrypoint pid=546644) http_endpoint_port: 8000
(skyrl_entrypoint pid=546644) max_turns: 999999
(skyrl_entrypoint pid=546644) chat_template:
(skyrl_entrypoint pid=546644) source: name
(skyrl_entrypoint pid=546644) name_or_path: null
(skyrl_entrypoint pid=546644) chat_template_kwargs: {}
(skyrl_entrypoint pid=546644) engine_init_kwargs:
(skyrl_entrypoint pid=546644) max_model_len: 32768
(skyrl_entrypoint pid=546644) custom_chat_template_chat_completion_path: chat_templates/qwen3_thinking_acc.jinja2
(skyrl_entrypoint pid=546644) served_model_name: 0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6
(skyrl_entrypoint pid=546644) override_existing_update_group: disable
(skyrl_entrypoint pid=546644) sampling_params:
(skyrl_entrypoint pid=546644) max_generate_length: 4096
(skyrl_entrypoint pid=546644) repetition_penalty: 1.0
(skyrl_entrypoint pid=546644) temperature: 0.7
(skyrl_entrypoint pid=546644) top_p: 0.95
(skyrl_entrypoint pid=546644) min_p: 0.0
(skyrl_entrypoint pid=546644) top_k: 20
(skyrl_entrypoint pid=546644) logprobs: null
(skyrl_entrypoint pid=546644) stop: null
(skyrl_entrypoint pid=546644) use_conversation_multi_turn: true
(skyrl_entrypoint pid=546644) append_eos_token_after_stop_str_in_multi_turn: true
(skyrl_entrypoint pid=546644) eval_sampling_params:
(skyrl_entrypoint pid=546644) max_generate_length: ${generator.sampling_params.max_generate_length}
(skyrl_entrypoint pid=546644) repetition_penalty: 1.0
(skyrl_entrypoint pid=546644) temperature: 0.0
(skyrl_entrypoint pid=546644) top_p: 1.0
(skyrl_entrypoint pid=546644) min_p: 0.0
(skyrl_entrypoint pid=546644) top_k: -1
(skyrl_entrypoint pid=546644) logprobs: null
(skyrl_entrypoint pid=546644) stop: null
(skyrl_entrypoint pid=546644) eval_n_samples_per_prompt: 8
(skyrl_entrypoint pid=546644) zero_reward_on_non_stop: false
(skyrl_entrypoint pid=546644) apply_overlong_filtering: false
(skyrl_entrypoint pid=546644) rope_scaling: ${trainer.rope_scaling}
(skyrl_entrypoint pid=546644) rope_theta: ${trainer.rope_theta}
(skyrl_entrypoint pid=546644) teacher:
(skyrl_entrypoint pid=546644) model_path: null
(skyrl_entrypoint pid=546644) top_k_logprobs: 256
(skyrl_entrypoint pid=546644) num_inference_engines: 1
(skyrl_entrypoint pid=546644) inference_engine_tensor_parallel_size: 1
(skyrl_entrypoint pid=546644) inference_engine_pipeline_parallel_size: 1
(skyrl_entrypoint pid=546644) gpu_memory_utilization: 0.9
(skyrl_entrypoint pid=546644) enforce_eager: false
(skyrl_entrypoint pid=546644) backend: vllm
(skyrl_entrypoint pid=546644) engine_init_kwargs: {}
(skyrl_entrypoint pid=546644) environment:
(skyrl_entrypoint pid=546644) env_class: gsm8k
(skyrl_entrypoint pid=546644) skyrl_gym:
(skyrl_entrypoint pid=546644) max_env_workers: 32
(skyrl_entrypoint pid=546644) text2sql:
(skyrl_entrypoint pid=546644) db_path: /home/ray/default/sql_data
(skyrl_entrypoint pid=546644) llm_as_a_judge:
(skyrl_entrypoint pid=546644) model: gpt-4o-mini
(skyrl_entrypoint pid=546644) base_url: null
(skyrl_entrypoint pid=546644) search:
(skyrl_entrypoint pid=546644) log_requests: false
(skyrl_entrypoint pid=546644) search_url: http://127.0.0.1:8000/retrieve
(skyrl_entrypoint pid=546644) topk: 3
(skyrl_entrypoint pid=546644) timeout: 30
(skyrl_entrypoint pid=546644) rollout:
(skyrl_entrypoint pid=546644) fanout:
(skyrl_entrypoint pid=546644) enabled: true
(skyrl_entrypoint pid=546644) num_coordinators: 4
(skyrl_entrypoint pid=546644) cpus_per_coordinator: 8
(skyrl_entrypoint pid=546644) deepspeed_config:
(skyrl_entrypoint pid=546644) train:
(skyrl_entrypoint pid=546644) zero_optimization:
(skyrl_entrypoint pid=546644) stage: 3
(skyrl_entrypoint pid=546644) offload_param:
(skyrl_entrypoint pid=546644) device: none
(skyrl_entrypoint pid=546644) offload_optimizer:
(skyrl_entrypoint pid=546644) device: none
(skyrl_entrypoint pid=546644) pin_memory: true
(skyrl_entrypoint pid=546644) sub_group_size: auto
(skyrl_entrypoint pid=546644) reduce_bucket_size: auto
(skyrl_entrypoint pid=546644) stage3_param_persistence_threshold: auto
(skyrl_entrypoint pid=546644) stage3_prefetch_bucket_size: auto
(skyrl_entrypoint pid=546644) stage3_max_live_parameters: auto
(skyrl_entrypoint pid=546644) stage3_max_reuse_distance: auto
(skyrl_entrypoint pid=546644) round_robin_gradients: true
(skyrl_entrypoint pid=546644) zero_hpz_partition_size: 1
(skyrl_entrypoint pid=546644) zero_quantized_weights: false
(skyrl_entrypoint pid=546644) zero_quantized_gradients: false
(skyrl_entrypoint pid=546644) torch_autocast:
(skyrl_entrypoint pid=546644) enabled: true
(skyrl_entrypoint pid=546644) dtype: bfloat16
(skyrl_entrypoint pid=546644) disable_trace_cache: false
(skyrl_entrypoint pid=546644) data_types:
(skyrl_entrypoint pid=546644) grad_accum_dtype: fp32
(skyrl_entrypoint pid=546644) gradient_clipping: 1.0
(skyrl_entrypoint pid=546644) wall_clock_breakdown: false
(skyrl_entrypoint pid=546644) prescale_gradient: false
(skyrl_entrypoint pid=546644) eval:
(skyrl_entrypoint pid=546644) zero_optimization:
(skyrl_entrypoint pid=546644) stage: 3
(skyrl_entrypoint pid=546644) stage3_param_persistence_threshold: auto
(skyrl_entrypoint pid=546644) offload_param:
(skyrl_entrypoint pid=546644) device: cpu
(skyrl_entrypoint pid=546644) pin_memory: true
(skyrl_entrypoint pid=546644) torch_autocast:
(skyrl_entrypoint pid=546644) enabled: true
(skyrl_entrypoint pid=546644) dtype: bfloat16
(skyrl_entrypoint pid=546644) gradient_clipping: 1.0
(skyrl_entrypoint pid=546644) prescale_gradient: false
(skyrl_entrypoint pid=546644) wall_clock_breakdown: false
(skyrl_entrypoint pid=546644) terminal_bench_config:
(skyrl_entrypoint pid=546644) trials_dir: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/ablation-pymethods2test-seqnorm/trace_jobs
(skyrl_entrypoint pid=546644) harbor:
(skyrl_entrypoint pid=546644) name: terminus-2
(skyrl_entrypoint pid=546644) max_episodes: 999999
(skyrl_entrypoint pid=546644) enable_summarize: false
(skyrl_entrypoint pid=546644) store_all_messages: true
(skyrl_entrypoint pid=546644) trajectory_config:
(skyrl_entrypoint pid=546644) raw_content: true
(skyrl_entrypoint pid=546644) enable_episode_logging: false
(skyrl_entrypoint pid=546644) record_terminal_session: false
(skyrl_entrypoint pid=546644) enable_pane_logging: false
(skyrl_entrypoint pid=546644) strict_json_parser: true
(skyrl_entrypoint pid=546644) interleaved_thinking: true
(skyrl_entrypoint pid=546644) extra_body:
(skyrl_entrypoint pid=546644) chat_template_kwargs:
(skyrl_entrypoint pid=546644) enable_thinking: true
(skyrl_entrypoint pid=546644) override_timeout_sec: 900
(skyrl_entrypoint pid=546644) override_cpus: 1
(skyrl_entrypoint pid=546644) override_memory_mb: 2048
(skyrl_entrypoint pid=546644) override_storage_mb: 2048
(skyrl_entrypoint pid=546644) auto_snapshot: true
(skyrl_entrypoint pid=546644) verifier_override_timeout_sec: 120
(skyrl_entrypoint pid=546644) max_retries: 3
(skyrl_entrypoint pid=546644) min_wait_sec: 60.0
(skyrl_entrypoint pid=546644) max_wait_sec: 600.0
(skyrl_entrypoint pid=546644) wait_multiplier: 2.0
(skyrl_entrypoint pid=546644) exclude_exceptions:
(skyrl_entrypoint pid=546644) - VerifierTimeoutError
(skyrl_entrypoint pid=546644) - VerifierRuntimeError
(skyrl_entrypoint pid=546644) - RewardFileNotFoundError
(skyrl_entrypoint pid=546644) - RewardFileEmptyError
(skyrl_entrypoint pid=546644) - VerifierOutputParseError
(skyrl_entrypoint pid=546644) n_concurrent_trials: 675
(skyrl_entrypoint pid=546644) log_level: INFO
(skyrl_entrypoint pid=546644) enable_reward_shaping: false
(skyrl_entrypoint pid=546644) enable_error_classification: true
(skyrl_entrypoint pid=546644) mask_exceptions:
(skyrl_entrypoint pid=546644) - DaytonaError
(skyrl_entrypoint pid=546644) - EnvironmentStartTimeoutError
(skyrl_entrypoint pid=546644) - NetworkError
(skyrl_entrypoint pid=546644) - ConnectionError
(skyrl_entrypoint pid=546644) - RewardFileNotFoundError
(skyrl_entrypoint pid=546644) - RewardFileEmptyError
(skyrl_entrypoint pid=546644) - AgentEnvironmentTimeoutError
(skyrl_entrypoint pid=546644) - ContextLengthExceededError
(skyrl_entrypoint pid=546644) default_error_treatment: zero
(skyrl_entrypoint pid=546644) passthrough_exceptions:
(skyrl_entrypoint pid=546644) - AgentTimeoutError
(skyrl_entrypoint pid=546644) zero_exceptions: []
(skyrl_entrypoint pid=546644) model_info:
(skyrl_entrypoint pid=546644) max_input_tokens: 32000
(skyrl_entrypoint pid=546644) max_output_tokens: 4096
(skyrl_entrypoint pid=546644) archiving:
(skyrl_entrypoint pid=546644) enabled: false
(skyrl_entrypoint pid=546644) trace_upload:
(skyrl_entrypoint pid=546644) enabled: true
(skyrl_entrypoint pid=546644) repo_org: DCAgent
(skyrl_entrypoint pid=546644) episodes: last
(skyrl_entrypoint pid=546644) dataset_type: SFT
(skyrl_entrypoint pid=546644) cleanup: true
(skyrl_entrypoint pid=546644) 
(skyrl_entrypoint pid=546644) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash:
(skyrl_entrypoint pid=546644) No module named 'vllm._version'
(skyrl_entrypoint pid=546644) from .version import __version__, __version_tuple__ # isort:skip
(skyrl_entrypoint pid=546644) W0608 20:41:39.902000 546644 envs/rl/lib/python3.12/site-packages/torch/utils/cpp_extension.py:117] No CUDA runtime is found, using CUDA_HOME='/e/software/default/stages/2026/software/CUDA/13'
(pid=530352) [2026-06-08 20:41:37,547 E 530352 544574] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 [repeated 269x across cluster]
(raylet) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e
(raylet, ip=10.128.32.57) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 43x across cluster]
(pid=2689828, ip=10.128.32.54) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash:
(pid=2689828, ip=10.128.32.54) No module named 'vllm._version'
(pid=2689828, ip=10.128.32.54) from .version import __version__, __version_tuple__ # isort:skip
(raylet, ip=10.128.32.58) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 109x across cluster]
[2026-06-08 20:41:53,824 E 546292 546616] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14
(pid=321222, ip=10.128.32.56) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: [repeated 15x across cluster]
(pid=321222, ip=10.128.32.56) No module named 'vllm._version' [repeated 15x across cluster]
(pid=321222, ip=10.128.32.56) from .version import __version__, __version_tuple__ # isort:skip [repeated 15x across cluster]
(RegistryActor pid=2689760, ip=10.128.32.54) [2026-06-08 20:41:54,703 E 2689760 2689800] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14
(raylet, ip=10.128.32.55) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 71x across cluster]
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) 2026-06-08 20:41:59.392 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:152 - setup_envvars_for_vllm: distributed_executor_backend=uni, SKYRL_ENABLE_NUMA_AFFINITY=1, CUDA_VISIBLE_DEVICES=0, VLLM_ENABLE_V1_MULTIPROCESSING=<unset>
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) 2026-06-08 20:41:59.393 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:167 - setup_envvars_for_vllm: numa_enabled=True
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) 2026-06-08 20:41:59.393 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:170 - setup_envvars_for_vllm: set VLLM_ENABLE_V1_MULTIPROCESSING=0 for NUMA affinity
(pid=2794792, ip=10.128.32.52) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: [repeated 9x across cluster]
(pid=2794792, ip=10.128.32.52) No module named 'vllm._version' [repeated 9x across cluster]
(pid=2794792, ip=10.128.32.52) from .version import __version__, __version_tuple__ # isort:skip [repeated 9x across cluster]
(skyrl_entrypoint pid=546644) [2026-06-08 20:41:56,879 E 546644 546687] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 [repeated 2x across cluster]
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) 2026-06-08 20:42:00.144 | INFO | skyrl_train.utils.numa:set_numa_affinity_for_gpu:340 - NUMA affinity: bound process to CPUs 0-71 (NUMA node 0) for GPU 0
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) 2026-06-08 20:42:00.168 | DEBUG | skyrl_train.utils.numa:_set_membind_via_libnuma:375 - NUMA affinity: set memory preferred to NUMA node 0
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) 2026-06-08 20:42:00.168 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:618 - BaseVLLMInferenceEngine: vllm_v1_disable_multiproc=True, vllm.__version__=dev, VLLM_ENABLE_V1_MULTIPROCESSING=0
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) 2026-06-08 20:42:00.168 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:626 - BaseVLLMInferenceEngine: set VLLM_ENABLE_V1_MULTIPROCESSING=0
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) 2026-06-08 20:42:00.168 | WARNING | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1192 - OpenAI API sampling params overridden: temperature=0.7, top_p=0.95, top_k=20
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) 2026-06-08 20:42:00.190 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1244 - Engine startup stagger: sleeping 2.00s (attempt 1/5) to avoid port collisions
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) The tokenizer you are loading from '/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
(raylet, ip=10.128.32.61) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 128x across cluster]
(skyrl_entrypoint pid=546644) [2026-06-08 20:42:04] INFO inference_engine_client_http_endpoint.py:350: Starting server on 0.0.0.0:8000
(AsyncVLLMInferenceEngine pid=3869940, ip=10.128.32.57) 2026-06-08 20:42:03.798 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:152 - setup_envvars_for_vllm: distributed_executor_backend=uni, SKYRL_ENABLE_NUMA_AFFINITY=1, CUDA_VISIBLE_DEVICES=1, VLLM_ENABLE_V1_MULTIPROCESSING=<unset> [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=3869940, ip=10.128.32.57) 2026-06-08 20:42:03.799 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:167 - setup_envvars_for_vllm: numa_enabled=True [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=3869940, ip=10.128.32.57) 2026-06-08 20:42:03.799 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:170 - setup_envvars_for_vllm: set VLLM_ENABLE_V1_MULTIPROCESSING=0 for NUMA affinity [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=2689961, ip=10.128.32.54) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: [repeated 19x across cluster]
(AsyncVLLMInferenceEngine pid=2689961, ip=10.128.32.54) No module named 'vllm._version' [repeated 19x across cluster]
(AsyncVLLMInferenceEngine pid=2689961, ip=10.128.32.54) from .version import __version__, __version_tuple__ # isort:skip [repeated 19x across cluster]
(skyrl_entrypoint pid=546644) [2026-06-08 20:42:04] INFO inference_engine_client_http_endpoint.py:242: Starting inference HTTP endpoint...
(skyrl_entrypoint pid=546644) [2026-06-08 20:42:05] INFO inference_engine_client_http_endpoint.py:229: Server ready after 2 attempts (2 seconds)
(skyrl_entrypoint pid=546644) 2026-06-08 20:42:05.367 | INFO  | skyrl_train.inference_engines.inference_engine_client:_spin_up_http_endpoint:969 - InferenceEngineClient HTTP endpoint started on 127.0.0.1:8000
(skyrl_entrypoint pid=546644) 2026-06-08 20:42:05.367 | INFO  | skyrl_train.inference_engines.inference_engine_client:__init__:61 - InferenceEngineClient initialized with 48 engines.
(skyrl_entrypoint pid=546644) 2026-06-08 20:42:05.369 | INFO  | examples.terminal_bench.terminal_bench_generator:_configure_harbor_logging:242 - Harbor logging level set to INFO
(skyrl_entrypoint pid=546644) 2026-06-08 20:42:05.370 | INFO  | examples.terminal_bench.terminal_bench_generator:__init__:142 - TerminalBenchGenerator initialized with HarborConfigBuilder. Exposed fields: ['name', 'max_episodes', 'enable_summarize', 'store_all_messages', 'trajectory_config', 'enable_episode_logging', 'record_terminal_session', 'enable_pane_logging', 'strict_json_parser', 'interleaved_thinking', 'extra_body', 'override_timeout_sec', 'override_cpus', 'override_memory_mb', 'override_storage_mb', 'auto_snapshot', 'verifier_override_timeout_sec', 'max_retries', 'min_wait_sec', 'max_wait_sec', 'wait_multiplier', 'exclude_exceptions', 'n_concurrent_trials', 'log_level', 'enable_reward_shaping', 'enable_error_classification', 'mask_exceptions', 'default_error_treatment', 'passthrough_exceptions', 'zero_exceptions']. Retry config: max_retries=3, backoff=60.0-600.0s. Concurrent trials: 675. Reward shaping: enabled=False, shaper=pass_ratio. Error classification: enabled=True
(skyrl_entrypoint pid=546644) 2026-06-08 20:42:05.371 | INFO  | examples.terminal_bench.terminal_bench_generator:__init__:158 - TerminalBenchGenerator initialized with custom chat template read from: chat_templates/qwen3_thinking_acc.jinja2
(skyrl_entrypoint pid=546644) 2026-06-08 20:42:05.372 | INFO  | skyrl_train.utils.trainer_utils:build_dataloader:656 - Total steps: 156
(skyrl_entrypoint pid=546644) 2026-06-08 20:42:05.372 | INFO  | skyrl_train.fully_async_trainer:_build_train_dataloader_and_compute_training_steps:357 - Length of train_dataloader: 5000
(skyrl_entrypoint pid=546644) 2026-06-08 20:42:05.372 | INFO  | skyrl_train.fully_async_trainer:_build_train_dataloader_and_compute_training_steps:358 - Number of steps per epoch: 78
(skyrl_entrypoint pid=546644) 2026-06-08 20:42:05.372 | INFO  | skyrl_train.fully_async_trainer:_build_train_dataloader_and_compute_training_steps:359 - Total training steps: 80
(AsyncVLLMInferenceEngine pid=3869940, ip=10.128.32.57) 2026-06-08 20:42:05.012 | INFO | skyrl_train.utils.numa:set_numa_affinity_for_gpu:340 - NUMA affinity: bound process to CPUs 72-143 (NUMA node 1) for GPU 1 [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=3869940, ip=10.128.32.57) 2026-06-08 20:42:05.034 | DEBUG | skyrl_train.utils.numa:_set_membind_via_libnuma:375 - NUMA affinity: set memory preferred to NUMA node 1 [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=3869940, ip=10.128.32.57) 2026-06-08 20:42:05.034 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:618 - BaseVLLMInferenceEngine: vllm_v1_disable_multiproc=True, vllm.__version__=dev, VLLM_ENABLE_V1_MULTIPROCESSING=0 [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=3869940, ip=10.128.32.57) 2026-06-08 20:42:05.034 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:626 - BaseVLLMInferenceEngine: set VLLM_ENABLE_V1_MULTIPROCESSING=0 [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=3869940, ip=10.128.32.57) 2026-06-08 20:42:05.034 | WARNING | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1192 - OpenAI API sampling params overridden: temperature=0.7, top_p=0.95, top_k=20 [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=3869940, ip=10.128.32.57) 2026-06-08 20:42:05.061 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1244 - Engine startup stagger: sleeping 1.75s (attempt 1/5) to avoid port collisions [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 22x across cluster]
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) The tokenizer you are loading from '/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 45x across cluster]
(AsyncVLLMInferenceEngine pid=2999029, ip=10.128.32.55) 2026-06-08 20:42:09.392 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:152 - setup_envvars_for_vllm: distributed_executor_backend=uni, SKYRL_ENABLE_NUMA_AFFINITY=1, CUDA_VISIBLE_DEVICES=1, VLLM_ENABLE_V1_MULTIPROCESSING=<unset> [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=2999029, ip=10.128.32.55) 2026-06-08 20:42:09.393 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:167 - setup_envvars_for_vllm: numa_enabled=True [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=2999029, ip=10.128.32.55) 2026-06-08 20:42:09.393 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:170 - setup_envvars_for_vllm: set VLLM_ENABLE_V1_MULTIPROCESSING=0 for NUMA affinity [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=321222, ip=10.128.32.56) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: [repeated 18x across cluster]
(AsyncVLLMInferenceEngine pid=321222, ip=10.128.32.56) No module named 'vllm._version' [repeated 18x across cluster]
(AsyncVLLMInferenceEngine pid=321222, ip=10.128.32.56) from .version import __version__, __version_tuple__ # isort:skip [repeated 18x across cluster]
(pid=3886563, ip=10.128.32.62) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead.
(AsyncVLLMInferenceEngine pid=3947796, ip=10.128.32.58) 2026-06-08 20:42:07.994 | INFO | skyrl_train.utils.numa:set_numa_affinity_for_gpu:340 - NUMA affinity: bound process to CPUs 216-287 (NUMA node 3) for GPU 3 [repeated 8x across cluster]
(AsyncVLLMInferenceEngine pid=3947796, ip=10.128.32.58) 2026-06-08 20:42:08.016 | DEBUG | skyrl_train.utils.numa:_set_membind_via_libnuma:375 - NUMA affinity: set memory preferred to NUMA node 3 [repeated 8x across cluster]
(AsyncVLLMInferenceEngine pid=3947796, ip=10.128.32.58) 2026-06-08 20:42:08.016 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:618 - BaseVLLMInferenceEngine: vllm_v1_disable_multiproc=True, vllm.__version__=dev, VLLM_ENABLE_V1_MULTIPROCESSING=0 [repeated 8x across cluster]
(AsyncVLLMInferenceEngine pid=3947796, ip=10.128.32.58) 2026-06-08 20:42:08.017 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:626 - BaseVLLMInferenceEngine: set VLLM_ENABLE_V1_MULTIPROCESSING=0 [repeated 8x across cluster]
(AsyncVLLMInferenceEngine pid=3947796, ip=10.128.32.58) 2026-06-08 20:42:08.017 | WARNING | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1192 - OpenAI API sampling params overridden: temperature=0.7, top_p=0.95, top_k=20 [repeated 8x across cluster]
(AsyncVLLMInferenceEngine pid=3947796, ip=10.128.32.58) 2026-06-08 20:42:08.046 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1244 - Engine startup stagger: sleeping 2.17s (attempt 1/5) to avoid port collisions [repeated 8x across cluster]
(bundle_reservation_check_func pid=546719) [2026-06-08 20:42:12,634 E 546719 546759] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14
(AsyncVLLMInferenceEngine pid=3947798, ip=10.128.32.58) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3947798, ip=10.128.32.58) The tokenizer you are loading from '/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. [repeated 8x across cluster]
(AsyncVLLMInferenceEngine pid=2999029, ip=10.128.32.55) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 18x across cluster]
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) 2026-06-08 20:42:10.747 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:152 - setup_envvars_for_vllm: distributed_executor_backend=uni, SKYRL_ENABLE_NUMA_AFFINITY=1, CUDA_VISIBLE_DEVICES=2, VLLM_ENABLE_V1_MULTIPROCESSING=<unset> [repeated 9x across cluster]
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) 2026-06-08 20:42:10.748 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:167 - setup_envvars_for_vllm: numa_enabled=True [repeated 9x across cluster]
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) 2026-06-08 20:42:10.748 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:170 - setup_envvars_for_vllm: set VLLM_ENABLE_V1_MULTIPROCESSING=0 for NUMA affinity [repeated 9x across cluster]
(AsyncVLLMInferenceEngine pid=2999030, ip=10.128.32.55) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2999030, ip=10.128.32.55) No module named 'vllm._version' [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2999030, ip=10.128.32.55) from .version import __version__, __version_tuple__ # isort:skip [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) 2026-06-08 20:42:12.143 | INFO | skyrl_train.utils.numa:set_numa_affinity_for_gpu:340 - NUMA affinity: bound process to CPUs 144-215 (NUMA node 2) for GPU 2 [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) 2026-06-08 20:42:12.166 | DEBUG | skyrl_train.utils.numa:_set_membind_via_libnuma:375 - NUMA affinity: set memory preferred to NUMA node 2 [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) 2026-06-08 20:42:12.166 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:618 - BaseVLLMInferenceEngine: vllm_v1_disable_multiproc=True, vllm.__version__=dev, VLLM_ENABLE_V1_MULTIPROCESSING=0 [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) 2026-06-08 20:42:12.167 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:626 - BaseVLLMInferenceEngine: set VLLM_ENABLE_V1_MULTIPROCESSING=0 [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) 2026-06-08 20:42:12.167 | WARNING | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1192 - OpenAI API sampling params overridden: temperature=0.7, top_p=0.95, top_k=20 [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) 2026-06-08 20:42:12.200 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1244 - Engine startup stagger: sleeping 2.09s (attempt 1/5) to avoid port collisions [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2729962, ip=10.128.32.53) (EngineCore_DP0 pid=2730194) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/torch/__init__.py:1617: UserWarning: Please use the new API settings to control TF32 behavior, such as torch.backends.cudnn.conv.fp32_precision = 'tf32' or torch.backends.cuda.matmul.fp32_precision = 'ieee'. Old settings, e.g, torch.backends.cuda.matmul.allow_tf32 = True, torch.backends.cudnn.allow_tf32 = True, allowTF32CuDNN() and allowTF32CuBLAS() will be deprecated after Pytorch 2.9. Please see https://pytorch.org/docs/main/notes/cuda.html#tensorfloat-32-tf32-on-ampere-and-later-devices (Triggered internally at /pytorch/aten/src/ATen/Context.cpp:80.)
(AsyncVLLMInferenceEngine pid=2729962, ip=10.128.32.53) (EngineCore_DP0 pid=2730194) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=2729834, ip=10.128.32.53) [2026-06-08 20:42:16,474 E 2729834 2729931] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 [repeated 6x across cluster]
(AsyncVLLMInferenceEngine pid=2794925, ip=10.128.32.52) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 24x across cluster]
(AsyncVLLMInferenceEngine pid=2794925, ip=10.128.32.52) The tokenizer you are loading from '/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2729961, ip=10.128.32.53) (EngineCore_DP0 pid=2730198) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=2794925, ip=10.128.32.52) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 22x across cluster]
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) (EngineCore_DP0 pid=2690185) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=2689961, ip=10.128.32.54) (EngineCore_DP0 pid=2690206) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=3893192, ip=10.128.32.61) 2026-06-08 20:42:18.145 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:152 - setup_envvars_for_vllm: distributed_executor_backend=uni, SKYRL_ENABLE_NUMA_AFFINITY=1, CUDA_VISIBLE_DEVICES=3, VLLM_ENABLE_V1_MULTIPROCESSING=<unset> [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3893192, ip=10.128.32.61) 2026-06-08 20:42:18.165 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:167 - setup_envvars_for_vllm: numa_enabled=True [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3893192, ip=10.128.32.61) 2026-06-08 20:42:18.165 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:170 - setup_envvars_for_vllm: set VLLM_ENABLE_V1_MULTIPROCESSING=0 for NUMA affinity [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=4003708, ip=10.128.32.60) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: [repeated 8x across cluster]
(AsyncVLLMInferenceEngine pid=4003708, ip=10.128.32.60) No module named 'vllm._version' [repeated 8x across cluster]
(AsyncVLLMInferenceEngine pid=4003708, ip=10.128.32.60) from .version import __version__, __version_tuple__ # isort:skip [repeated 8x across cluster]
(AsyncVLLMInferenceEngine pid=2689962, ip=10.128.32.54) (EngineCore_DP0 pid=2690189) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=3869940, ip=10.128.32.57) (EngineCore_DP0 pid=3870167) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=2730089, ip=10.128.32.53) (EngineCore_DP0 pid=2730214) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=3869809, ip=10.128.32.57) (EngineCore_DP0 pid=3870171) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) [W608 20:42:20.803504224 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [jpbo-042-06.jupiter.internal]:38653 (errno: 97 - Address family not supported by protocol).
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) [W608 20:42:20.805060772 Utils.hpp:166] Warning: Environment variable NCCL_BLOCKING_WAIT is deprecated; use TORCH_NCCL_BLOCKING_WAIT instead (function operator())
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) [rank0]:[W608 20:42:20.810164201 Utils.hpp:137] Warning: Environment variable TORCH_NCCL_TRACE_BUFFER_SIZE is deprecated; use TORCH_FR_BUFFER_SIZE instead (function operator())
(AsyncVLLMInferenceEngine pid=2689963, ip=10.128.32.54) (EngineCore_DP0 pid=2690201) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=2729834, ip=10.128.32.53) (EngineCore_DP0 pid=2730190) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=321222, ip=10.128.32.56) (EngineCore_DP0 pid=321457) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=3893192, ip=10.128.32.61) 2026-06-08 20:42:19.386 | INFO | skyrl_train.utils.numa:set_numa_affinity_for_gpu:340 - NUMA affinity: bound process to CPUs 216-287 (NUMA node 3) for GPU 3 [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3893193, ip=10.128.32.61) 2026-06-08 20:42:19.409 | DEBUG | skyrl_train.utils.numa:_set_membind_via_libnuma:375 - NUMA affinity: set memory preferred to NUMA node 2 [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3893193, ip=10.128.32.61) 2026-06-08 20:42:19.409 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:618 - BaseVLLMInferenceEngine: vllm_v1_disable_multiproc=True, vllm.__version__=dev, VLLM_ENABLE_V1_MULTIPROCESSING=0 [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3893193, ip=10.128.32.61) 2026-06-08 20:42:19.409 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:626 - BaseVLLMInferenceEngine: set VLLM_ENABLE_V1_MULTIPROCESSING=0 [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3893193, ip=10.128.32.61) 2026-06-08 20:42:19.409 | WARNING | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1192 - OpenAI API sampling params overridden: temperature=0.7, top_p=0.95, top_k=20 [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3893192, ip=10.128.32.61) 2026-06-08 20:42:19.436 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1244 - Engine startup stagger: sleeping 2.66s (attempt 1/5) to avoid port collisions [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=321092, ip=10.128.32.56) (EngineCore_DP0 pid=321462) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) (EngineCore_DP0 pid=3870181) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=321221, ip=10.128.32.56) (EngineCore_DP0 pid=321449) _C._set_float32_matmul_precision(precision)
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62) 2026-06-08 20:42:21 INFO [ipv4-debug] hostname=jpbo-042-14.jupiter.internal
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62) 2026-06-08 20:42:21 INFO [ipv4-debug] _global_node.node_ip_address=10.128.32.62
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62) 2026-06-08 20:42:21 INFO [ipv4-debug] get_node_ip_address()=10.128.32.62
(AsyncVLLMInferenceEngine pid=3947796, ip=10.128.32.58) (EngineCore_DP0 pid=3948025) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=3869941, ip=10.128.32.57) (EngineCore_DP0 pid=3870176) _C._set_float32_matmul_precision(precision)
(skyrl_entrypoint pid=546644) 2026-06-08 20:42:22.012 | INFO  | skyrl_train.workers.worker:_initiate_actors:636 - Initializing process group for RayActorGroup
(AsyncVLLMInferenceEngine pid=2689962, ip=10.128.32.54) (EngineCore_DP0 pid=2690189)
Loading safetensors checkpoint shards: 0% Completed | 0/4 [00:00<?, ?it/s]
(AsyncVLLMInferenceEngine pid=321220, ip=10.128.32.56) (EngineCore_DP0 pid=321469) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) (EngineCore_DP0 pid=3948034) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) (EngineCore_DP0 pid=3948034) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/torch/__init__.py:1617: UserWarning: Please use the new API settings to control TF32 behavior, such as torch.backends.cudnn.conv.fp32_precision = 'tf32' or torch.backends.cuda.matmul.fp32_precision = 'ieee'. Old settings, e.g, torch.backends.cuda.matmul.allow_tf32 = True, torch.backends.cudnn.allow_tf32 = True, allowTF32CuDNN() and allowTF32CuBLAS() will be deprecated after Pytorch 2.9. Please see https://pytorch.org/docs/main/notes/cuda.html#tensorfloat-32-tf32-on-ampere-and-later-devices (Triggered internally at /pytorch/aten/src/ATen/Context.cpp:80.) [repeated 17x across cluster]
(AsyncVLLMInferenceEngine pid=321222, ip=10.128.32.56) [2026-06-08 20:42:21,583 E 321222 321386] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 [repeated 18x across cluster]
(AsyncVLLMInferenceEngine pid=3893191, ip=10.128.32.61) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 32x across cluster]
(AsyncVLLMInferenceEngine pid=3893191, ip=10.128.32.61) The tokenizer you are loading from '/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3893191, ip=10.128.32.61) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 74x across cluster]
(AsyncVLLMInferenceEngine pid=3947798, ip=10.128.32.58) (EngineCore_DP0 pid=3948043) _C._set_float32_matmul_precision(precision)
(pid=3886639, ip=10.128.32.62) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead.
(AsyncVLLMInferenceEngine pid=3947669, ip=10.128.32.58) (EngineCore_DP0 pid=3948029) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=2999155, ip=10.128.32.55) (EngineCore_DP0 pid=2999262) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=2999029, ip=10.128.32.55) (EngineCore_DP0 pid=2999257) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=3893191, ip=10.128.32.61) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: [repeated 14x across cluster]
(AsyncVLLMInferenceEngine pid=3893191, ip=10.128.32.61) No module named 'vllm._version' [repeated 14x across cluster]
(AsyncVLLMInferenceEngine pid=3893191, ip=10.128.32.61) from .version import __version__, __version_tuple__ # isort:skip [repeated 14x across cluster]
(AsyncVLLMInferenceEngine pid=3947669, ip=10.128.32.58) [W608 20:42:24.383004564 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [jpbo-042-10.jupiter.internal]:42977 (errno: 97 - Address family not supported by protocol). [repeated 20x across cluster]
(AsyncVLLMInferenceEngine pid=3947669, ip=10.128.32.58) [W608 20:42:24.383500619 Utils.hpp:166] Warning: Environment variable NCCL_BLOCKING_WAIT is deprecated; use TORCH_NCCL_BLOCKING_WAIT instead (function operator()) [repeated 19x across cluster]
(AsyncVLLMInferenceEngine pid=3947669, ip=10.128.32.58) [rank0]:[W608 20:42:24.385697605 Utils.hpp:137] Warning: Environment variable TORCH_NCCL_TRACE_BUFFER_SIZE is deprecated; use TORCH_FR_BUFFER_SIZE instead (function operator()) [repeated 19x across cluster]
(AsyncVLLMInferenceEngine pid=2794924, ip=10.128.32.52) (EngineCore_DP0 pid=2795150) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) (EngineCore_DP0 pid=2999273) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) (EngineCore_DP0 pid=2795159) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=2999030, ip=10.128.32.55) (EngineCore_DP0 pid=2999277) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=2729834, ip=10.128.32.53) (EngineCore_DP0 pid=2730190)
Loading safetensors checkpoint shards: 25% Completed | 1/4 [00:04<00:13, 4.40s/it]
(AsyncVLLMInferenceEngine pid=2794925, ip=10.128.32.52) (EngineCore_DP0 pid=2795174) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=3947796, ip=10.128.32.58) (EngineCore_DP0 pid=3948025)
Loading safetensors checkpoint shards: 0% Completed | 0/4 [00:00<?, ?it/s] [repeated 19x across cluster]
(AsyncVLLMInferenceEngine pid=3120157, ip=10.128.32.50) (EngineCore_DP0 pid=3120386) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=2794792, ip=10.128.32.52) (EngineCore_DP0 pid=2795155) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=2794792, ip=10.128.32.52) (EngineCore_DP0 pid=2795155) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/torch/__init__.py:1617: UserWarning: Please use the new API settings to control TF32 behavior, such as torch.backends.cudnn.conv.fp32_precision = 'tf32' or torch.backends.cuda.matmul.fp32_precision = 'ieee'. Old settings, e.g, torch.backends.cuda.matmul.allow_tf32 = True, torch.backends.cudnn.allow_tf32 = True, allowTF32CuDNN() and allowTF32CuBLAS() will be deprecated after Pytorch 2.9. Please see https://pytorch.org/docs/main/notes/cuda.html#tensorfloat-32-tf32-on-ampere-and-later-devices (Triggered internally at /pytorch/aten/src/ATen/Context.cpp:80.) [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) [2026-06-08 20:42:27,451 E 2998900 2999003] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=3120155, ip=10.128.32.50) (EngineCore_DP0 pid=3120390) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=3120027, ip=10.128.32.50) (EngineCore_DP0 pid=3120402) _C._set_float32_matmul_precision(precision)
(pid=546824) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 6x across cluster]
(AsyncVLLMInferenceEngine pid=3120156, ip=10.128.32.50) (EngineCore_DP0 pid=3120407) _C._set_float32_matmul_precision(precision)
(FSDPPolicyWorkerBase pid=3886641, ip=10.128.32.62) [W608 20:42:30.447721940 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [jpbo-042-14.jupiter.internal]:42967 (errno: 97 - Address family not supported by protocol). [repeated 14x across cluster]
(FSDPPolicyWorkerBase pid=3886641, ip=10.128.32.62) [W608 20:42:30.448104973 Utils.hpp:166] Warning: Environment variable NCCL_BLOCKING_WAIT is deprecated; use TORCH_NCCL_BLOCKING_WAIT instead (function operator()) [repeated 14x across cluster]
(AsyncVLLMInferenceEngine pid=3120156, ip=10.128.32.50) [rank0]:[W608 20:42:29.576075878 Utils.hpp:137] Warning: Environment variable TORCH_NCCL_TRACE_BUFFER_SIZE is deprecated; use TORCH_FR_BUFFER_SIZE instead (function operator()) [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=768606, ip=10.128.32.59) (EngineCore_DP0 pid=768833) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=4003708, ip=10.128.32.60) (EngineCore_DP0 pid=4003938) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=321222, ip=10.128.32.56) (EngineCore_DP0 pid=321457)
Loading safetensors checkpoint shards: 50% Completed | 2/4 [00:07<00:07, 3.89s/it] [repeated 35x across cluster]
(AsyncVLLMInferenceEngine pid=2916172, ip=10.128.32.51) (EngineCore_DP0 pid=2916405) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=768604, ip=10.128.32.59) (EngineCore_DP0 pid=768838) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=3893193, ip=10.128.32.61) (EngineCore_DP0 pid=3893421) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=2916173, ip=10.128.32.51) (EngineCore_DP0 pid=2916417) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=3120156, ip=10.128.32.50) (EngineCore_DP0 pid=3120407)
Loading safetensors checkpoint shards: 0% Completed | 0/4 [00:00<?, ?it/s] [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=4003710, ip=10.128.32.60) (EngineCore_DP0 pid=4003961) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=4003710, ip=10.128.32.60) (EngineCore_DP0 pid=4003961) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/torch/__init__.py:1617: UserWarning: Please use the new API settings to control TF32 behavior, such as torch.backends.cudnn.conv.fp32_precision = 'tf32' or torch.backends.cuda.matmul.fp32_precision = 'ieee'. Old settings, e.g, torch.backends.cuda.matmul.allow_tf32 = True, torch.backends.cudnn.allow_tf32 = True, allowTF32CuDNN() and allowTF32CuBLAS() will be deprecated after Pytorch 2.9. Please see https://pytorch.org/docs/main/notes/cuda.html#tensorfloat-32-tf32-on-ampere-and-later-devices (Triggered internally at /pytorch/aten/src/ATen/Context.cpp:80.) [repeated 10x across cluster]
(AsyncVLLMInferenceEngine pid=768606, ip=10.128.32.59) [2026-06-08 20:42:33,267 E 768606 768714] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 [repeated 17x across cluster]
(skyrl_entrypoint pid=546644) 2026-06-08 20:42:33.310 | INFO  | skyrl_train.workers.worker:_initiate_actors:638 - Initialized process group for RayActorGroup
(skyrl_entrypoint pid=546644) 2026-06-08 20:42:33.320 | INFO  | skyrl_train.workers.worker:_initiate_actors:640 - Mesh Ranks: [MeshRank(dp=0, sp=0, tp=0, pp=0, world_size=8, dp_size=8, pp_size=1), MeshRank(dp=1, sp=0, tp=0, pp=0, world_size=8, dp_size=8, pp_size=1), MeshRank(dp=2, sp=0, tp=0, pp=0, world_size=8, dp_size=8, pp_size=1), MeshRank(dp=3, sp=0, tp=0, pp=0, world_size=8, dp_size=8, pp_size=1), MeshRank(dp=4, sp=0, tp=0, pp=0, world_size=8, dp_size=8, pp_size=1), MeshRank(dp=5, sp=0, tp=0, pp=0, world_size=8, dp_size=8, pp_size=1), MeshRank(dp=6, sp=0, tp=0, pp=0, world_size=8, dp_size=8, pp_size=1), MeshRank(dp=7, sp=0, tp=0, pp=0, world_size=8, dp_size=8, pp_size=1)]
(FSDPPolicyWorkerBase pid=546822) `torch_dtype` is deprecated! Use `dtype` instead!
(AsyncVLLMInferenceEngine pid=3893064, ip=10.128.32.61) (EngineCore_DP0 pid=3893429) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=4003709, ip=10.128.32.60) (EngineCore_DP0 pid=4003956) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=768605, ip=10.128.32.59) (EngineCore_DP0 pid=768849) _C._set_float32_matmul_precision(precision)
(FSDPPolicyWorkerBase pid=546822)
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
(FSDPPolicyWorkerBase pid=546822)
Loading checkpoint shards: 25%|██▌ | 1/4 [00:00<00:00, 9.73it/s]
(AsyncVLLMInferenceEngine pid=4003580, ip=10.128.32.60) (EngineCore_DP0 pid=4003942) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=2916039, ip=10.128.32.51) (EngineCore_DP0 pid=2916423) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=768476, ip=10.128.32.59) (EngineCore_DP0 pid=768842) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=3893191, ip=10.128.32.61) (EngineCore_DP0 pid=3893442) _C._set_float32_matmul_precision(precision)
(FSDPPolicyWorkerBase pid=546822)
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:00<00:00, 2.95it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00, 4.51it/s]
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62)
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00, 4.95it/s]
(AsyncVLLMInferenceEngine pid=2916174, ip=10.128.32.51) (EngineCore_DP0 pid=2916413) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=3893192, ip=10.128.32.61) (EngineCore_DP0 pid=3893434) _C._set_float32_matmul_precision(precision)
(AsyncVLLMInferenceEngine pid=3893192, ip=10.128.32.61) [W608 20:42:35.989727680 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [jpbo-042-13.jupiter.internal]:59347 (errno: 97 - Address family not supported by protocol). [repeated 21x across cluster]
(AsyncVLLMInferenceEngine pid=3893192, ip=10.128.32.61) [W608 20:42:35.990281111 Utils.hpp:166] Warning: Environment variable NCCL_BLOCKING_WAIT is deprecated; use TORCH_NCCL_BLOCKING_WAIT instead (function operator()) [repeated 22x across cluster]
(AsyncVLLMInferenceEngine pid=3893192, ip=10.128.32.61) [rank0]:[W608 20:42:35.992209591 Utils.hpp:137] Warning: Environment variable TORCH_NCCL_TRACE_BUFFER_SIZE is deprecated; use TORCH_FR_BUFFER_SIZE instead (function operator()) [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) (EngineCore_DP0 pid=2690185)
(AsyncVLLMInferenceEngine pid=2689963, ip=10.128.32.54) (EngineCore_DP0 pid=2690201)
(AsyncVLLMInferenceEngine pid=2689962, ip=10.128.32.54) (EngineCore_DP0 pid=2690189)
(AsyncVLLMInferenceEngine pid=2689961, ip=10.128.32.54) (EngineCore_DP0 pid=2690206)
(AsyncVLLMInferenceEngine pid=2729834, ip=10.128.32.53) (EngineCore_DP0 pid=2730190)
(AsyncVLLMInferenceEngine pid=2729961, ip=10.128.32.53) (EngineCore_DP0 pid=2730198)
(AsyncVLLMInferenceEngine pid=2729962, ip=10.128.32.53) (EngineCore_DP0 pid=2730194)
(AsyncVLLMInferenceEngine pid=2730089, ip=10.128.32.53) (EngineCore_DP0 pid=2730214)
(AsyncVLLMInferenceEngine pid=3869809, ip=10.128.32.57) (EngineCore_DP0 pid=3870171)
(AsyncVLLMInferenceEngine pid=3869941, ip=10.128.32.57) (EngineCore_DP0 pid=3870176)
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) (EngineCore_DP0 pid=3870181)
(AsyncVLLMInferenceEngine pid=3869940, ip=10.128.32.57) (EngineCore_DP0 pid=3870167)
(AsyncVLLMInferenceEngine pid=321092, ip=10.128.32.56) (EngineCore_DP0 pid=321462)
(AsyncVLLMInferenceEngine pid=321092, ip=10.128.32.56) (EngineCore_DP0 pid=321462)
Loading safetensors checkpoint shards: 100% Completed | 4/4 [00:12<00:00, 3.22s/it] [repeated 66x across cluster]
(AsyncVLLMInferenceEngine pid=321221, ip=10.128.32.56) (EngineCore_DP0 pid=321449)
(AsyncVLLMInferenceEngine pid=321220, ip=10.128.32.56) (EngineCore_DP0 pid=321469)
(AsyncVLLMInferenceEngine pid=321222, ip=10.128.32.56) (EngineCore_DP0 pid=321457)
(AsyncVLLMInferenceEngine pid=3893192, ip=10.128.32.61) (EngineCore_DP0 pid=3893434)
Loading safetensors checkpoint shards: 0% Completed | 0/4 [00:00<?, ?it/s] [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3947669, ip=10.128.32.58) (EngineCore_DP0 pid=3948029)
(AsyncVLLMInferenceEngine pid=3893192, ip=10.128.32.61) (EngineCore_DP0 pid=3893434) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/torch/__init__.py:1617: UserWarning: Please use the new API settings to control TF32 behavior, such as torch.backends.cudnn.conv.fp32_precision = 'tf32' or torch.backends.cuda.matmul.fp32_precision = 'ieee'. Old settings, e.g, torch.backends.cuda.matmul.allow_tf32 = True, torch.backends.cudnn.allow_tf32 = True, allowTF32CuDNN() and allowTF32CuBLAS() will be deprecated after Pytorch 2.9. Please see https://pytorch.org/docs/main/notes/cuda.html#tensorfloat-32-tf32-on-ampere-and-later-devices (Triggered internally at /pytorch/aten/src/ATen/Context.cpp:80.) [repeated 9x across cluster]
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62) [2026-06-08 20:42:36,503 E 3886563 3886603] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 [repeated 8x across cluster]
(FSDPPolicyWorkerBase pid=3886639, ip=10.128.32.62) `torch_dtype` is deprecated! Use `dtype` instead! [repeated 7x across cluster]
(FSDPPolicyWorkerBase pid=3886639, ip=10.128.32.62)
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] [repeated 7x across cluster]
(FSDPPolicyWorkerBase pid=3886639, ip=10.128.32.62)
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:00<00:00, 3.54it/s] [repeated 15x across cluster]
(AsyncVLLMInferenceEngine pid=3947798, ip=10.128.32.58) (EngineCore_DP0 pid=3948043)
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) (EngineCore_DP0 pid=3948034)
(AsyncVLLMInferenceEngine pid=3947796, ip=10.128.32.58) (EngineCore_DP0 pid=3948025)
(FSDPPolicyWorkerBase pid=546824)
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:00<00:00, 2.95it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00, 4.49it/s] [repeated 3x across cluster]
(FSDPPolicyWorkerBase pid=3886639, ip=10.128.32.62)
Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00, 4.94it/s] [repeated 3x across cluster]
(AsyncVLLMInferenceEngine pid=2689961, ip=10.128.32.54) (EngineCore_DP0 pid=2690206) The tokenizer you are loading from '/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
(AsyncVLLMInferenceEngine pid=2689961, ip=10.128.32.54) 2026-06-08 20:42:40.983 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1301 - Initializing OpenAIServingChat with custom_chat_template read from: chat_templates/qwen3_thinking_acc.jinja2
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) (EngineCore_DP0 pid=2999273)
(AsyncVLLMInferenceEngine pid=2999030, ip=10.128.32.55) (EngineCore_DP0 pid=2999277)
(AsyncVLLMInferenceEngine pid=2999029, ip=10.128.32.55) (EngineCore_DP0 pid=2999257)
(AsyncVLLMInferenceEngine pid=2999155, ip=10.128.32.55) (EngineCore_DP0 pid=2999262)
(AsyncVLLMInferenceEngine pid=2794792, ip=10.128.32.52) (EngineCore_DP0 pid=2795155)
Loading safetensors checkpoint shards: 75% Completed | 3/4 [00:12<00:04, 4.23s/it] [repeated 59x across cluster]
(AsyncVLLMInferenceEngine pid=768476, ip=10.128.32.59) (EngineCore_DP0 pid=768842)
(AsyncVLLMInferenceEngine pid=768606, ip=10.128.32.59) (EngineCore_DP0 pid=768833)
(AsyncVLLMInferenceEngine pid=768604, ip=10.128.32.59) (EngineCore_DP0 pid=768838)
(AsyncVLLMInferenceEngine pid=768605, ip=10.128.32.59) (EngineCore_DP0 pid=768849)
(AsyncVLLMInferenceEngine pid=2794792, ip=10.128.32.52) (EngineCore_DP0 pid=2795155)
(AsyncVLLMInferenceEngine pid=2794925, ip=10.128.32.52) (EngineCore_DP0 pid=2795174)
(AsyncVLLMInferenceEngine pid=2794924, ip=10.128.32.52) (EngineCore_DP0 pid=2795150)
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) (EngineCore_DP0 pid=2795159)
(AsyncVLLMInferenceEngine pid=3120027, ip=10.128.32.50) (EngineCore_DP0 pid=3120402)
(AsyncVLLMInferenceEngine pid=2999155, ip=10.128.32.55) (EngineCore_DP0 pid=2999262) The tokenizer you are loading from '/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. [repeated 23x across cluster]
(AsyncVLLMInferenceEngine pid=3120157, ip=10.128.32.50) (EngineCore_DP0 pid=3120386)
(AsyncVLLMInferenceEngine pid=3120155, ip=10.128.32.50) (EngineCore_DP0 pid=3120390)
(AsyncVLLMInferenceEngine pid=3120156, ip=10.128.32.50) (EngineCore_DP0 pid=3120407)
(FSDPPolicyWorkerBase pid=546822) [rank6]:[W608 20:42:46.478528610 Utils.hpp:137] Warning: Environment variable TORCH_NCCL_TRACE_BUFFER_SIZE is deprecated; use TORCH_FR_BUFFER_SIZE instead (function operator())
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) 2026-06-08 20:42:45.205 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1301 - Initializing OpenAIServingChat with custom_chat_template read from: chat_templates/qwen3_thinking_acc.jinja2 [repeated 23x across cluster]
(AsyncVLLMInferenceEngine pid=3120156, ip=10.128.32.50) (EngineCore_DP0 pid=3120407)
Loading safetensors checkpoint shards: 100% Completed | 4/4 [00:14<00:00, 3.56s/it] [repeated 43x across cluster]
(raylet) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e
(skyrl_entrypoint pid=546644) 2026-06-08 20:42:48.797 | INFO  | skyrl_train.trainer:build_models:851 - init policy/ref/critic models done
(skyrl_entrypoint pid=546644) 2026-06-08 20:42:48.802 | INFO  | skyrl_train.fully_async_trainer:_maybe_enable_rollout_fanout:456 - Rollout fan-out ENABLED: replacing single-process generator with RolloutDispatcher (K=4, cpus_per_coordinator=8).
(skyrl_entrypoint pid=546644) 2026-06-08 20:42:48.824 | INFO  | examples.terminal_bench.rollout_coordinator:__init__:387 - [RolloutDispatcher] fan-out path: overriding inference host 127.0.0.1 -> 10.128.32.49 (routable head IP) for coordinator litellm base_url connectivity
(skyrl_entrypoint pid=546644) 2026-06-08 20:42:48.824 | INFO  | examples.terminal_bench.rollout_coordinator:__init__:405 - [RolloutDispatcher] configured num_coordinators=4, cpus_per_coordinator=8
(skyrl_entrypoint pid=546644) 2026-06-08 20:42:48.829 | INFO  | examples.terminal_bench.rollout_coordinator:startup:434 - [RolloutDispatcher] PlacementGroup ready: 4 bundles x 8 CPU (SPREAD)
(AsyncVLLMInferenceEngine pid=3893064, ip=10.128.32.61) (EngineCore_DP0 pid=3893429)
(AsyncVLLMInferenceEngine pid=3893193, ip=10.128.32.61) (EngineCore_DP0 pid=3893421)
(AsyncVLLMInferenceEngine pid=3893191, ip=10.128.32.61) (EngineCore_DP0 pid=3893442)
(AsyncVLLMInferenceEngine pid=3893192, ip=10.128.32.61) (EngineCore_DP0 pid=3893434)
(AsyncVLLMInferenceEngine pid=4003580, ip=10.128.32.60) (EngineCore_DP0 pid=4003942)
(AsyncVLLMInferenceEngine pid=4003710, ip=10.128.32.60) (EngineCore_DP0 pid=4003961)
(AsyncVLLMInferenceEngine pid=4003709, ip=10.128.32.60) (EngineCore_DP0 pid=4003956)
(AsyncVLLMInferenceEngine pid=4003708, ip=10.128.32.60) (EngineCore_DP0 pid=4003938)
(AsyncVLLMInferenceEngine pid=2916039, ip=10.128.32.51) (EngineCore_DP0 pid=2916423)
(AsyncVLLMInferenceEngine pid=2916173, ip=10.128.32.51) (EngineCore_DP0 pid=2916417)
(AsyncVLLMInferenceEngine pid=2916174, ip=10.128.32.51) (EngineCore_DP0 pid=2916413)
(AsyncVLLMInferenceEngine pid=2916172, ip=10.128.32.51) (EngineCore_DP0 pid=2916405)
(AsyncVLLMInferenceEngine pid=3893064, ip=10.128.32.61) (EngineCore_DP0 pid=3893429) The tokenizer you are loading from '/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. [repeated 13x across cluster]
(FSDPPolicyWorkerBase pid=3886639, ip=10.128.32.62) [rank1]:[W608 20:42:46.956187698 Utils.hpp:137] Warning: Environment variable TORCH_NCCL_TRACE_BUFFER_SIZE is deprecated; use TORCH_FR_BUFFER_SIZE instead (function operator()) [repeated 7x across cluster]
(AsyncVLLMInferenceEngine pid=3120157, ip=10.128.32.50) 2026-06-08 20:42:49.702 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1301 - Initializing OpenAIServingChat with custom_chat_template read from: chat_templates/qwen3_thinking_acc.jinja2 [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2916172, ip=10.128.32.51) (EngineCore_DP0 pid=2916405)
Loading safetensors checkpoint shards: 100% Completed | 4/4 [00:13<00:00, 3.45s/it] [repeated 36x across cluster]
(FSDPPolicyWorkerBase pid=546825) [2026-06-08 20:42:52,793 E 546825 546973] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14
(RolloutCoordinator pid=547199) 2026-06-08 20:42:54.371 | INFO | examples.terminal_bench.rollout_coordinator:_scale_terminal_bench_cfg:121 - [RolloutCoordinator] scaled n_concurrent_trials 675 -> 168 (// 4)
(raylet) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 5x across cluster]
(skyrl_entrypoint pid=546644) 2026-06-08 20:42:54.609 | INFO  | examples.terminal_bench.rollout_coordinator:startup:469 - [RolloutDispatcher] coordinator 1/4 started
(RolloutCoordinator pid=547199) 2026-06-08 20:42:54.604 | INFO | examples.terminal_bench.terminal_bench_generator:_configure_harbor_logging:242 - Harbor logging level set to INFO
(RolloutCoordinator pid=547199) 2026-06-08 20:42:54.604 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:142 - TerminalBenchGenerator initialized with HarborConfigBuilder. Exposed fields: ['name', 'max_episodes', 'enable_summarize', 'store_all_messages', 'trajectory_config', 'enable_episode_logging', 'record_terminal_session', 'enable_pane_logging', 'strict_json_parser', 'interleaved_thinking', 'extra_body', 'override_timeout_sec', 'override_cpus', 'override_memory_mb', 'override_storage_mb', 'auto_snapshot', 'verifier_override_timeout_sec', 'max_retries', 'min_wait_sec', 'max_wait_sec', 'wait_multiplier', 'exclude_exceptions', 'n_concurrent_trials', 'log_level', 'enable_reward_shaping', 'enable_error_classification', 'mask_exceptions', 'default_error_treatment', 'passthrough_exceptions', 'zero_exceptions']. Retry config: max_retries=3, backoff=60.0-600.0s. Concurrent trials: 168. Reward shaping: enabled=False, shaper=pass_ratio. Error classification: enabled=True
(RolloutCoordinator pid=547199) 2026-06-08 20:42:54.604 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:158 - TerminalBenchGenerator initialized with custom chat template read from: chat_templates/qwen3_thinking_acc.jinja2
(RolloutCoordinator pid=547199) 2026-06-08 20:42:54.604 | INFO | examples.terminal_bench.rollout_coordinator:__init__:246 - [RolloutCoordinator 0/4] constructed (http=10.128.32.49:8000)
(RolloutCoordinator pid=547199) 2026-06-08 20:42:54.608 | INFO | examples.terminal_bench.terminal_bench_generator:_create_orchestrator:298 - QueueOrchestrator created and started with n_concurrent_trials=168, rollback_hook registered for ContextLengthExceededError/AgentTimeoutError
(RolloutCoordinator pid=547199) 2026-06-08 20:42:54.608 | INFO | examples.terminal_bench.terminal_bench_generator:startup:255 - TerminalBenchGenerator startup complete. Shared orchestrator ready with n_concurrent_trials=168
(RolloutCoordinator pid=547199) 2026-06-08 20:42:54.608 | INFO | examples.terminal_bench.rollout_coordinator:startup:254 - [RolloutCoordinator 0] startup complete
(RolloutCoordinator pid=547199) The tokenizer you are loading from '/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2916174, ip=10.128.32.51) 2026-06-08 20:42:53.595 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1301 - Initializing OpenAIServingChat with custom_chat_template read from: chat_templates/qwen3_thinking_acc.jinja2 [repeated 12x across cluster]
(FSDPPolicyWorkerBase pid=546824) [2026-06-08 20:42:52,896 E 546824 547051] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 [repeated 6x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56) 2026-06-08 20:43:01.985 | INFO | examples.terminal_bench.rollout_coordinator:_scale_terminal_bench_cfg:121 - [RolloutCoordinator] scaled n_concurrent_trials 675 -> 168 (// 4)
(raylet, ip=10.128.32.56) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 6x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56) 2026-06-08 20:43:02.202 | INFO | examples.terminal_bench.terminal_bench_generator:_configure_harbor_logging:242 - Harbor logging level set to INFO
(RolloutCoordinator pid=321702, ip=10.128.32.56) 2026-06-08 20:43:02.202 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:142 - TerminalBenchGenerator initialized with HarborConfigBuilder. Exposed fields: ['name', 'max_episodes', 'enable_summarize', 'store_all_messages', 'trajectory_config', 'enable_episode_logging', 'record_terminal_session', 'enable_pane_logging', 'strict_json_parser', 'interleaved_thinking', 'extra_body', 'override_timeout_sec', 'override_cpus', 'override_memory_mb', 'override_storage_mb', 'auto_snapshot', 'verifier_override_timeout_sec', 'max_retries', 'min_wait_sec', 'max_wait_sec', 'wait_multiplier', 'exclude_exceptions', 'n_concurrent_trials', 'log_level', 'enable_reward_shaping', 'enable_error_classification', 'mask_exceptions', 'default_error_treatment', 'passthrough_exceptions', 'zero_exceptions']. Retry config: max_retries=3, backoff=60.0-600.0s. Concurrent trials: 168. Reward shaping: enabled=False, shaper=pass_ratio. Error classification: enabled=True
(RolloutCoordinator pid=321702, ip=10.128.32.56) 2026-06-08 20:43:02.202 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:158 - TerminalBenchGenerator initialized with custom chat template read from: chat_templates/qwen3_thinking_acc.jinja2
(RolloutCoordinator pid=321702, ip=10.128.32.56) 2026-06-08 20:43:02.202 | INFO | examples.terminal_bench.rollout_coordinator:__init__:246 - [RolloutCoordinator 1/4] constructed (http=10.128.32.49:8000)
(RolloutCoordinator pid=321702, ip=10.128.32.56) 2026-06-08 20:43:02.205 | INFO | examples.terminal_bench.terminal_bench_generator:_create_orchestrator:298 - QueueOrchestrator created and started with n_concurrent_trials=168, rollback_hook registered for ContextLengthExceededError/AgentTimeoutError
(RolloutCoordinator pid=321702, ip=10.128.32.56) 2026-06-08 20:43:02.205 | INFO | examples.terminal_bench.terminal_bench_generator:startup:255 - TerminalBenchGenerator startup complete. Shared orchestrator ready with n_concurrent_trials=168
(RolloutCoordinator pid=321702, ip=10.128.32.56) 2026-06-08 20:43:02.205 | INFO | examples.terminal_bench.rollout_coordinator:startup:254 - [RolloutCoordinator 1] startup complete
(skyrl_entrypoint pid=546644) 2026-06-08 20:43:02.205 | INFO  | examples.terminal_bench.rollout_coordinator:startup:469 - [RolloutDispatcher] coordinator 2/4 started
(RolloutCoordinator pid=2690439, ip=10.128.32.54) 2026-06-08 20:43:10.597 | INFO | examples.terminal_bench.rollout_coordinator:_scale_terminal_bench_cfg:121 - [RolloutCoordinator] scaled n_concurrent_trials 675 -> 168 (// 4)
(RolloutCoordinator pid=321702, ip=10.128.32.56) The tokenizer you are loading from '/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
(raylet, ip=10.128.32.54) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 6x across cluster]
(skyrl_entrypoint pid=546644) 2026-06-08 20:43:10.821 | INFO  | examples.terminal_bench.rollout_coordinator:startup:469 - [RolloutDispatcher] coordinator 3/4 started
(RolloutCoordinator pid=2690439, ip=10.128.32.54) The tokenizer you are loading from '/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
(RolloutCoordinator pid=2690439, ip=10.128.32.54) 2026-06-08 20:43:10.817 | INFO | examples.terminal_bench.terminal_bench_generator:_configure_harbor_logging:242 - Harbor logging level set to INFO
(RolloutCoordinator pid=2690439, ip=10.128.32.54) 2026-06-08 20:43:10.817 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:142 - TerminalBenchGenerator initialized with HarborConfigBuilder. Exposed fields: ['name', 'max_episodes', 'enable_summarize', 'store_all_messages', 'trajectory_config', 'enable_episode_logging', 'record_terminal_session', 'enable_pane_logging', 'strict_json_parser', 'interleaved_thinking', 'extra_body', 'override_timeout_sec', 'override_cpus', 'override_memory_mb', 'override_storage_mb', 'auto_snapshot', 'verifier_override_timeout_sec', 'max_retries', 'min_wait_sec', 'max_wait_sec', 'wait_multiplier', 'exclude_exceptions', 'n_concurrent_trials', 'log_level', 'enable_reward_shaping', 'enable_error_classification', 'mask_exceptions', 'default_error_treatment', 'passthrough_exceptions', 'zero_exceptions']. Retry config: max_retries=3, backoff=60.0-600.0s. Concurrent trials: 168. Reward shaping: enabled=False, shaper=pass_ratio. Error classification: enabled=True
(RolloutCoordinator pid=2690439, ip=10.128.32.54) 2026-06-08 20:43:10.817 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:158 - TerminalBenchGenerator initialized with custom chat template read from: chat_templates/qwen3_thinking_acc.jinja2
(RolloutCoordinator pid=2690439, ip=10.128.32.54) 2026-06-08 20:43:10.818 | INFO | examples.terminal_bench.rollout_coordinator:__init__:246 - [RolloutCoordinator 2/4] constructed (http=10.128.32.49:8000)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) 2026-06-08 20:43:10.820 | INFO | examples.terminal_bench.terminal_bench_generator:_create_orchestrator:298 - QueueOrchestrator created and started with n_concurrent_trials=168, rollback_hook registered for ContextLengthExceededError/AgentTimeoutError
(RolloutCoordinator pid=2690439, ip=10.128.32.54) 2026-06-08 20:43:10.820 | INFO | examples.terminal_bench.terminal_bench_generator:startup:255 - TerminalBenchGenerator startup complete. Shared orchestrator ready with n_concurrent_trials=168
(RolloutCoordinator pid=2690439, ip=10.128.32.54) 2026-06-08 20:43:10.820 | INFO | examples.terminal_bench.rollout_coordinator:startup:254 - [RolloutCoordinator 2] startup complete
(RolloutCoordinator pid=3120643, ip=10.128.32.50) 2026-06-08 20:43:18.562 | INFO | examples.terminal_bench.rollout_coordinator:_scale_terminal_bench_cfg:121 - [RolloutCoordinator] scaled n_concurrent_trials 675 -> 168 (// 4)
(raylet, ip=10.128.32.50) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 6x across cluster]
(skyrl_entrypoint pid=546644) 2026-06-08 20:43:18.782 | INFO  | examples.terminal_bench.rollout_coordinator:startup:469 - [RolloutDispatcher] coordinator 4/4 started
(skyrl_entrypoint pid=546644) 2026-06-08 20:43:18.782 | INFO  | examples.terminal_bench.rollout_coordinator:startup:477 - [RolloutDispatcher] 4 coordinators started
(skyrl_entrypoint pid=546644) 2026-06-08 20:43:18.782 | INFO  | skyrl_train.fully_async_trainer:train:485 - Generator startup complete
(skyrl_entrypoint pid=546644) 2026-06-08 20:43:18.782 | INFO  | skyrl_train.fully_async_trainer:_train_loop:510 - Started: 'load_checkpoints'
(skyrl_entrypoint pid=546644) 2026-06-08 20:43:18.788 | INFO  | skyrl_train.trainer:load_checkpoints:1735 - Loading checkpoint from: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/ablation-pymethods2test-seqnorm/checkpoints/global_step_83
(skyrl_entrypoint pid=546644) 2026-06-08 20:43:18.788 | INFO  | skyrl_train.trainer:load_checkpoints:1741 - Resuming from global_step: 83
(skyrl_entrypoint pid=546644) 2026-06-08 20:43:18.794 | INFO  | skyrl_train.trainer:load_checkpoints:1757 - Successfully loaded trainer state
(skyrl_entrypoint pid=546644) 2026-06-08 20:43:18.796 | INFO  | skyrl_train.trainer:load_checkpoints:1767 - Successfully loaded dataloader state
(skyrl_entrypoint pid=546644) 2026-06-08 20:43:18.796 | INFO  | skyrl_train.trainer:load_checkpoints:1776 - Loading policy checkpoint from /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/ablation-pymethods2test-seqnorm/checkpoints/global_step_83/policy
(RolloutCoordinator pid=3120643, ip=10.128.32.50) The tokenizer you are loading from '/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
(RolloutCoordinator pid=3120643, ip=10.128.32.50) 2026-06-08 20:43:18.778 | INFO | examples.terminal_bench.terminal_bench_generator:_configure_harbor_logging:242 - Harbor logging level set to INFO
(RolloutCoordinator pid=3120643, ip=10.128.32.50) 2026-06-08 20:43:18.778 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:142 - TerminalBenchGenerator initialized with HarborConfigBuilder. Exposed fields: ['name', 'max_episodes', 'enable_summarize', 'store_all_messages', 'trajectory_config', 'enable_episode_logging', 'record_terminal_session', 'enable_pane_logging', 'strict_json_parser', 'interleaved_thinking', 'extra_body', 'override_timeout_sec', 'override_cpus', 'override_memory_mb', 'override_storage_mb', 'auto_snapshot', 'verifier_override_timeout_sec', 'max_retries', 'min_wait_sec', 'max_wait_sec', 'wait_multiplier', 'exclude_exceptions', 'n_concurrent_trials', 'log_level', 'enable_reward_shaping', 'enable_error_classification', 'mask_exceptions', 'default_error_treatment', 'passthrough_exceptions', 'zero_exceptions']. Retry config: max_retries=3, backoff=60.0-600.0s. Concurrent trials: 168. Reward shaping: enabled=False, shaper=pass_ratio. Error classification: enabled=True
(RolloutCoordinator pid=3120643, ip=10.128.32.50) 2026-06-08 20:43:18.778 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:158 - TerminalBenchGenerator initialized with custom chat template read from: chat_templates/qwen3_thinking_acc.jinja2
(RolloutCoordinator pid=3120643, ip=10.128.32.50) 2026-06-08 20:43:18.778 | INFO | examples.terminal_bench.rollout_coordinator:__init__:246 - [RolloutCoordinator 3/4] constructed (http=10.128.32.49:8000)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) 2026-06-08 20:43:18.781 | INFO | examples.terminal_bench.terminal_bench_generator:_create_orchestrator:298 - QueueOrchestrator created and started with n_concurrent_trials=168, rollback_hook registered for ContextLengthExceededError/AgentTimeoutError
(RolloutCoordinator pid=3120643, ip=10.128.32.50) 2026-06-08 20:43:18.781 | INFO | examples.terminal_bench.terminal_bench_generator:startup:255 - TerminalBenchGenerator startup complete. Shared orchestrator ready with n_concurrent_trials=168
(RolloutCoordinator pid=3120643, ip=10.128.32.50) 2026-06-08 20:43:18.781 | INFO | examples.terminal_bench.rollout_coordinator:startup:254 - [RolloutCoordinator 3] startup complete
(RolloutCoordinator pid=547199) [2026-06-08 20:43:19,778 E 547199 547263] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14
(RolloutCoordinator pid=321702, ip=10.128.32.56) [2026-06-08 20:43:27,452 E 321702 321742] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14
(RolloutCoordinator pid=2690439, ip=10.128.32.54) [2026-06-08 20:43:35,103 E 2690439 2690479] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/torch/distributed/distributed_c10d.py:4876: UserWarning: barrier(): using the device under current context. You can specify `device_id` in `init_process_group` to mute this warning.
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62) warnings.warn( # warn only once
(skyrl_entrypoint pid=546644) 2026-06-08 20:43:37.246 | INFO  | skyrl_train.trainer:load_checkpoints:1786 - Successfully loaded policy checkpoint
(skyrl_entrypoint pid=546644) 2026-06-08 20:43:37.246 | INFO  | skyrl_train.trainer:load_checkpoints:1802 - Successfully loaded complete checkpoint state from global_step_83
(skyrl_entrypoint pid=546644) 2026-06-08 20:43:37.247 | INFO  | skyrl_train.fully_async_trainer:_train_loop:512 - Resumed training from global_step 83
(skyrl_entrypoint pid=546644) 2026-06-08 20:43:37.247 | WARNING  | skyrl_train.fully_async_trainer:_train_loop:517 - No data consumption state found in checkpoint — resume may re-train on already-consumed data
(skyrl_entrypoint pid=546644) 2026-06-08 20:43:37.248 | WARNING  | skyrl_train.fully_async_trainer:_train_loop:537 - Data consumption count mismatch on resume: expected 320, got 0. This can happen after epoch boundary transitions or error recovery.
(skyrl_entrypoint pid=546644) 2026-06-08 20:43:37.248 | INFO  | skyrl_train.fully_async_trainer:_train_loop:510 - Finished: 'load_checkpoints', time cost: 18.47s
(skyrl_entrypoint pid=546644) 2026-06-08 20:43:37.248 | INFO  | skyrl_train.fully_async_trainer:_train_loop:544 - Started: 'init_weight_sync_state'
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash:
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62) No module named 'vllm._version'
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62) from .version import __version__, __version_tuple__ # isort:skip
(RolloutCoordinator pid=3120643, ip=10.128.32.50) [2026-06-08 20:43:43,749 E 3120643 3120685] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14
(FSDPPolicyWorkerBase pid=546824) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: [repeated 7x across cluster]
(FSDPPolicyWorkerBase pid=546824) No module named 'vllm._version' [repeated 7x across cluster]
(FSDPPolicyWorkerBase pid=546824) from .version import __version__, __version_tuple__ # isort:skip [repeated 7x across cluster]
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62) 2026-06-08 20:43:43.980 | INFO  | logging:info:2216 - [weight-sync] hostname=jpbo-042-14.jupiter.internal
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62) 2026-06-08 20:43:43.980 | INFO  | logging:info:2216 - [weight-sync] _global_node.node_ip_address=10.128.32.62
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62) 2026-06-08 20:43:43.981 | INFO  | logging:info:2216 - [weight-sync] get_node_ip_address()=10.128.32.62
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62) 2026-06-08 20:43:43.981 | INFO  | logging:info:2216 - [weight-sync] Using master_addr=10.128.32.62, master_port=56205
(AsyncVLLMInferenceEngine pid=2916039, ip=10.128.32.51) (EngineCore_DP0 pid=2916423) 2026-06-08 20:43:44.137 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:init_weight_update_communicator:223 - torch.distributed.get_rank(): 0, rank_offset: 33, rank: 33, world_size: 49, group_name: skyrl
(AsyncVLLMInferenceEngine pid=2916173, ip=10.128.32.51) [rank0]:[W608 20:43:44.487817648 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [jpbo-042-14.jupiter.internal]:56205 (errno: 97 - Address family not supported by protocol).
(AsyncVLLMInferenceEngine pid=2916174, ip=10.128.32.51) (EngineCore_DP0 pid=2916413) 2026-06-08 20:43:44.140 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:init_weight_update_communicator:234 - init_weight_update_communicator: master_address=10.128.32.62, master_port=56205,
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/torch/distributed/distributed_c10d.py:4876: UserWarning: barrier(): using the device under current context. You can specify `device_id` in `init_process_group` to mute this warning.
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62) warnings.warn( # warn only once
(skyrl_entrypoint pid=546644) 2026-06-08 20:43:44.211 | INFO  | skyrl_train.trainer:init_weight_sync_state:874 - Initialized weight sync state for policy model and inference engines.
(skyrl_entrypoint pid=546644) 2026-06-08 20:43:44.212 | INFO  | skyrl_train.fully_async_trainer:_train_loop:544 - Finished: 'init_weight_sync_state', time cost: 6.96s
(AsyncVLLMInferenceEngine pid=321092, ip=10.128.32.56) [rank0]:[W608 20:43:47.408195056 Utils.hpp:137] Warning: Environment variable TORCH_NCCL_TRACE_BUFFER_SIZE is deprecated; use TORCH_FR_BUFFER_SIZE instead (function operator())
(skyrl_entrypoint pid=546644) 2026-06-08 20:43:52.780 | INFO  | skyrl_train.fully_async_trainer:_train_loop:548 - Finished: 'sync_weights_to_inference_engines', time cost: 8.57s
(skyrl_entrypoint pid=546644) 2026-06-08 20:43:52.781 | INFO  | skyrl_train.callbacks.builtin:on_train_begin:270 - HFHubUploadCallback initialized: repo=laion/ablation-pymethods2test-seqnorm, upload_steps=5, export_path=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/ablation-pymethods2test-seqnorm/exports
(skyrl_entrypoint pid=546644)
Training Step Progress: 83it [00:00, ?it/s]
(skyrl_entrypoint pid=546644)
Generation Buffer Progress: 0%| | 0/64 [00:00<?, ?it/s]2026-06-08 20:43:52.783 | INFO  | skyrl_train.fully_async_trainer:_train_loop:607 - Started: 'wait_for_generation_buffer'
(skyrl_entrypoint pid=546644) 2026-06-08 20:43:52.783 | INFO  | skyrl_train.fully_async_trainer:_train_loop:604 - Started: 'step' [repeated 2x across cluster]
(AsyncVLLMInferenceEngine pid=321222, ip=10.128.32.56) (EngineCore_DP0 pid=321457) 2026-06-08 20:43:44.134 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:init_weight_update_communicator:223 - torch.distributed.get_rank(): 0, rank_offset: 12, rank: 12, world_size: 49, group_name: skyrl [repeated 47x across cluster]
(AsyncVLLMInferenceEngine pid=321222, ip=10.128.32.56) [rank0]:[W608 20:43:44.602362089 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [jpbo-042-14.jupiter.internal]:56205 (errno: 97 - Address family not supported by protocol). [repeated 48x across cluster]
(AsyncVLLMInferenceEngine pid=321222, ip=10.128.32.56) (EngineCore_DP0 pid=321457) 2026-06-08 20:43:44.138 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:init_weight_update_communicator:234 - init_weight_update_communicator: master_address=10.128.32.62, master_port=56205,  [repeated 47x across cluster]
(RolloutCoordinator pid=547199) 2026-06-08 20:43:52.786 | INFO | examples.terminal_bench.terminal_bench_generator:generate:560 - Starting batch generation for 8 trials (mode=training)
(AsyncVLLMInferenceEngine pid=4003708, ip=10.128.32.60) [rank0]:[W608 20:43:47.875441937 Utils.hpp:137] Warning: Environment variable TORCH_NCCL_TRACE_BUFFER_SIZE is deprecated; use TORCH_FR_BUFFER_SIZE instead (function operator()) [repeated 47x across cluster]
(RolloutCoordinator pid=547199) 2026-06-08 20:43:58.155 | INFO | examples.terminal_bench.terminal_bench_generator:generate:560 - Starting batch generation for 8 trials (mode=training) [repeated 171x across cluster]
(skyrl_entrypoint pid=546644)
(skyrl_entrypoint pid=546644)
Generation Buffer Progress: 2%|▏ | 1/64 [08:07<8:31:29, 487.13s/it]
(skyrl_entrypoint pid=546644)
Generation Buffer Progress: 2%|▏ | 1/64 [08:07<8:31:29, 487.13s/it, buffer qsize=0]
(RolloutCoordinator pid=2690439, ip=10.128.32.54) 2026-06-08 20:44:02.931 | INFO | examples.terminal_bench.terminal_bench_generator:generate:560 - Starting batch generation for 8 trials (mode=training) [repeated 166x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54) 2026-06-08 20:51:59.908 | INFO | examples.terminal_bench.terminal_bench_generator:generate:801 - Batch generation complete: 8/8 successful, 0 failed instances, 0 masked (excluded from baseline)
(skyrl_entrypoint pid=546644)
(skyrl_entrypoint pid=546644)
Generation Buffer Progress: 3%|▎ | 2/64 [09:35<4:21:02, 252.62s/it, buffer qsize=0]
(skyrl_entrypoint pid=546644)
Generation Buffer Progress: 3%|▎ | 2/64 [09:35<4:21:02, 252.62s/it, buffer qsize=0]
(RolloutCoordinator pid=2690439, ip=10.128.32.54) 2026-06-08 20:51:59.912 | INFO | examples.terminal_bench.terminal_bench_generator:generate:560 - Starting batch generation for 8 trials (mode=training)
(RolloutCoordinator pid=321702, ip=10.128.32.56) 2026-06-08 20:53:28.374 | INFO | examples.terminal_bench.terminal_bench_generator:generate:801 - Batch generation complete: 8/8 successful, 0 failed instances, 0 masked (excluded from baseline)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) 2026-06-08 20:53:28.458 | INFO | examples.terminal_bench.terminal_bench_generator:generate:560 - Starting batch generation for 8 trials (mode=training)
(skyrl_entrypoint pid=546644)
(skyrl_entrypoint pid=546644)
Generation Buffer Progress: 5%|▍ | 3/64 [09:43<2:23:04, 140.73s/it, buffer qsize=0]
(skyrl_entrypoint pid=546644)
Generation Buffer Progress: 5%|▍ | 3/64 [09:43<2:23:04, 140.73s/it, buffer qsize=0]
(RolloutCoordinator pid=547199) 2026-06-08 20:53:35.958 | INFO | examples.terminal_bench.terminal_bench_generator:generate:560 - Starting batch generation for 8 trials (mode=training)
(RolloutCoordinator pid=321702, ip=10.128.32.56) 2026-06-08 20:53:35.918 | INFO | examples.terminal_bench.terminal_bench_generator:generate:801 - Batch generation complete: 8/8 successful, 0 failed instances, 0 masked (excluded from baseline)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) 2026-06-08 20:53:51.637 | INFO | examples.terminal_bench.terminal_bench_generator:generate:801 - Batch generation complete: 8/8 successful, 0 failed instances, 0 masked (excluded from baseline)
(skyrl_entrypoint pid=546644)
(skyrl_entrypoint pid=546644)
Generation Buffer Progress: 6%|▋ | 4/64 [09:58<1:31:22, 91.37s/it, buffer qsize=0] 
(skyrl_entrypoint pid=546644)
Generation Buffer Progress: 6%|▋ | 4/64 [09:58<1:31:22, 91.37s/it, buffer qsize=0]
(RolloutCoordinator pid=321702, ip=10.128.32.56) 2026-06-08 20:53:51.651 | INFO | examples.terminal_bench.terminal_bench_generator:generate:560 - Starting batch generation for 8 trials (mode=training)
(RolloutCoordinator pid=321702, ip=10.128.32.56) 2026-06-08 20:53:51.828 | INFO | examples.terminal_bench.terminal_bench_generator:generate:801 - Batch generation complete: 8/8 successful, 0 failed instances, 0 masked (excluded from baseline)
(skyrl_entrypoint pid=546644)
(skyrl_entrypoint pid=546644)
Generation Buffer Progress: 8%|▊ | 5/64 [09:59<57:30, 58.48s/it, buffer qsize=0] 
(skyrl_entrypoint pid=546644)
Generation Buffer Progress: 8%|▊ | 5/64 [09:59<57:30, 58.48s/it, buffer qsize=0]
(skyrl_entrypoint pid=546644)
(skyrl_entrypoint pid=546644)
Generation Buffer Progress: 9%|▉ | 6/64 [10:10<41:00, 42.42s/it, buffer qsize=0]
(skyrl_entrypoint pid=546644)
Generation Buffer Progress: 9%|▉ | 6/64 [10:10<41:00, 42.42s/it, buffer qsize=0]
(RolloutCoordinator pid=2690439, ip=10.128.32.54) 2026-06-08 20:53:51.832 | INFO | examples.terminal_bench.terminal_bench_generator:generate:560 - Starting batch generation for 8 trials (mode=training)
(RolloutCoordinator pid=547199) 2026-06-08 20:54:03.066 | INFO | examples.terminal_bench.terminal_bench_generator:generate:801 - Batch generation complete: 8/8 successful, 0 failed instances, 0 masked (excluded from baseline)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) 2026-06-08 20:54:03.436 | INFO | examples.terminal_bench.terminal_bench_generator:generate:560 - Starting batch generation for 8 trials (mode=training)
(skyrl_entrypoint pid=546644)
(skyrl_entrypoint pid=546644)
Generation Buffer Progress: 11%|█ | 7/64 [11:20<48:46, 51.35s/it, buffer qsize=0]
(skyrl_entrypoint pid=546644)
Generation Buffer Progress: 11%|█ | 7/64 [11:20<48:46, 51.35s/it, buffer qsize=0]
(RolloutCoordinator pid=547199) 2026-06-08 20:55:12.801 | INFO | examples.terminal_bench.terminal_bench_generator:generate:560 - Starting batch generation for 8 trials (mode=training)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) 2026-06-08 20:55:12.793 | INFO | examples.terminal_bench.terminal_bench_generator:generate:801 - Batch generation complete: 8/8 successful, 0 failed instances, 0 masked (excluded from baseline)
(skyrl_entrypoint pid=546644)
(skyrl_entrypoint pid=546644)
Generation Buffer Progress: 12%|█▎ | 8/64 [11:21<33:03, 35.41s/it, buffer qsize=0]
(skyrl_entrypoint pid=546644)
Generation Buffer Progress: 12%|█▎ | 8/64 [11:21<33:03, 35.41s/it, buffer qsize=0]
(skyrl_entrypoint pid=546644)
(skyrl_entrypoint pid=546644)
Generation Buffer Progress: 14%|█▍ | 9/64 [11:48<30:01, 32.76s/it, buffer qsize=0]
(skyrl_entrypoint pid=546644)
Generation Buffer Progress: 14%|█▍ | 9/64 [11:48<30:01, 32.76s/it, buffer qsize=0]
(RolloutCoordinator pid=321702, ip=10.128.32.56) 2026-06-08 20:55:14.094 | INFO | examples.terminal_bench.terminal_bench_generator:generate:560 - Starting batch generation for 8 trials (mode=training)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) 2026-06-08 20:55:14.080 | INFO | examples.terminal_bench.terminal_bench_generator:generate:801 - Batch generation complete: 8/8 successful, 0 failed instances, 0 masked (excluded from baseline)
(RolloutCoordinator pid=321702, ip=10.128.32.56) 2026-06-08 20:55:41.000 | INFO | examples.terminal_bench.terminal_bench_generator:generate:801 - Batch generation complete: 8/8 successful, 0 failed instances, 0 masked (excluded from baseline)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) 2026-06-08 20:55:41.005 | INFO | examples.terminal_bench.terminal_bench_generator:generate:560 - Starting batch generation for 8 trials (mode=training)
slurmstepd: error: *** JOB 639476 ON jpbo-042-01 CANCELLED AT 2026-06-08T20:55:54 ***
Preserving Ray logs to /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/ray_logs/
(skyrl_entrypoint pid=546644) 2026-06-08 20:55:54.883 | WARNING  | skyrl_train.inference_engines.inference_engine_client:_mark_engine_dead:68 - Inference engine 9 died (ActorDiedError). 47/48 engines remaining.
(skyrl_entrypoint pid=546644) 2026-06-08 20:55:54.894 | ERROR  | skyrl_train.fully_async_trainer:_run_generate_for_a_group_loop:961 - Generator worker errored out with exception: The actor died unexpectedly before finishing this task.
(skyrl_entrypoint pid=546644) class_name: RolloutCoordinator
(skyrl_entrypoint pid=546644) actor_id: ab4de8c56e2033b1453f2d2002000000
(skyrl_entrypoint pid=546644) pid: 321702
(skyrl_entrypoint pid=546644) namespace: 9a22cdec-1723-4b53-8bb1-11eb041ab632
(skyrl_entrypoint pid=546644) ip: 10.128.32.56
(skyrl_entrypoint pid=546644) The actor died because its node has died. Node Id: 3e123e53cc2ba79145f3a0ef6259451f129cfaa5304440e6f5817516
(skyrl_entrypoint pid=546644) the actor's node was terminated expectedly: received SIGTERM
(skyrl_entrypoint pid=546644) Traceback (most recent call last):
(skyrl_entrypoint pid=546644) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/skyrl_train/fully_async_trainer.py", line 930, in _run_generate_for_a_group_loop
(skyrl_entrypoint pid=546644) cur_generator_output: GeneratorOutput = await self.generator.generate(generator_input)
(skyrl_entrypoint pid=546644) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/examples/terminal_bench/rollout_coordinator.py", line 492, in generate
(skyrl_entrypoint pid=546644) return await actor.run_shard.remote(input_batch, global_step)
(skyrl_entrypoint pid=546644) ray.exceptions.ActorDiedError: The actor died unexpectedly before finishing this task.
(skyrl_entrypoint pid=546644) class_name: RolloutCoordinator
(skyrl_entrypoint pid=546644) actor_id: ab4de8c56e2033b1453f2d2002000000
(skyrl_entrypoint pid=546644) pid: 321702
(skyrl_entrypoint pid=546644) namespace: 9a22cdec-1723-4b53-8bb1-11eb041ab632
(skyrl_entrypoint pid=546644) ip: 10.128.32.56
(skyrl_entrypoint pid=546644) The actor died because its node has died. Node Id: 3e123e53cc2ba79145f3a0ef6259451f129cfaa5304440e6f5817516
(skyrl_entrypoint pid=546644) the actor's node was terminated expectedly: received SIGTERM
(skyrl_entrypoint pid=546644) 2026-06-08 20:55:54.951 | WARNING  | skyrl_train.inference_engines.inference_engine_client:_mark_engine_dead:68 - Inference engine 8 died (ActorDiedError). 46/48 engines remaining.
2026-06-08 20:55:55.045 | ERROR | __main__:main:134 - Training failed: Task failed because the node it was running on is dead or unavailable. Node IP: 10.128.32.49, node ID: 4cba01b979f7f003d04abf087b90b4e01890df9fe091283ac00c4962. This can happen if the node was preempted, had a hardware failure, or its raylet crashed unexpectedly. To see node death information, use `ray list nodes --filter node_id=4cba01b979f7f003d04abf087b90b4e01890df9fe091283ac00c4962`, check the Ray dashboard cluster page, search the node ID in the GCS logs, or use `ray logs raylet.out -ip 10.128.32.49`.
2026-06-08 20:55:55.045 | INFO | __main__:main:137 - Shutting down Ray on head node...
(skyrl_entrypoint pid=546644) [fd-monitor] Started monitoring (every 120s)
(skyrl_entrypoint pid=546644) [fd-monitor] [20:41:32] OK: 50 / 131,072 FDs open (0.0% of soft limit, hard limit: 131,072)
(skyrl_entrypoint pid=546644) [fd-monitor] [20:41:32] OK: RSS 1.38 GiB | node mem 148.6/858.0 GiB used (17.3%), avail 709.4 GiB
(skyrl_entrypoint pid=546644) ⚙️ Running in WANDB offline mode
(skyrl_entrypoint pid=546644) INFO 06-08 20:41:39 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s
(pid=2689828, ip=10.128.32.54) INFO 06-08 20:41:57 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s
(pid=2689963, ip=10.128.32.54) INFO 06-08 20:41:58 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) WARNING 06-08 20:42:02 [arg_utils.py:1256] The global random seed is set to 42. Since VLLM_ENABLE_V1_MULTIPROCESSING is set to False, this may affect the random state of the Python process that launched vLLM.
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) INFO 06-08 20:42:02 [model.py:529] Resolved architecture: Qwen3ForCausalLM
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) INFO 06-08 20:42:02 [model.py:1549] Using max model len 32768
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) INFO 06-08 20:42:02 [arg_utils.py:1469] Using ray runtime env (env vars redacted): {'env_vars': {'NCCL_CUMEM_ENABLE': '***', 'NCCL_DEBUG': '***', 'NCCL_SOCKET_FAMILY': '***', 'NCCL_SOCKET_IFNAME': '***', 'RAY_ADDRESS': '***', 'VLLM_ALLOW_INSECURE_SERIALIZATION': '***', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': '***', 'VLLM_DISABLE_COMPILE_CACHE': '***', 'WANDB_API_KEY': '***'}}
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) INFO 06-08 20:42:02 [scheduler.py:224] Chunked prefill is enabled with max_num_batched_tokens=65536.
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) INFO 06-08 20:42:02 [vllm.py:690] Asynchronous scheduling is enabled.
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) WARNING 06-08 20:42:02 [vllm.py:728] Enforce eager set, overriding optimization level to -O0
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) INFO 06-08 20:42:02 [vllm.py:846] Cudagraph is disabled under eager mode
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) WARNING 06-08 20:42:02 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) WARNING 06-08 20:42:02 [system_utils.py:140] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned
(pid=321222, ip=10.128.32.56) INFO 06-08 20:42:03 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s [repeated 14x across cluster]
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) WARNING 06-08 20:42:07 [arg_utils.py:1256] The global random seed is set to 49. Since VLLM_ENABLE_V1_MULTIPROCESSING is set to False, this may affect the random state of the Python process that launched vLLM. [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) INFO 06-08 20:42:07 [model.py:529] Resolved architecture: Qwen3ForCausalLM [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) INFO 06-08 20:42:07 [model.py:1549] Using max model len 32768 [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) INFO 06-08 20:42:07 [arg_utils.py:1469] Using ray runtime env (env vars redacted): {'env_vars': {'NCCL_CUMEM_ENABLE': '***', 'NCCL_DEBUG': '***', 'NCCL_SOCKET_FAMILY': '***', 'NCCL_SOCKET_IFNAME': '***', 'RAY_ADDRESS': '***', 'VLLM_ALLOW_INSECURE_SERIALIZATION': '***', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': '***', 'VLLM_DISABLE_COMPILE_CACHE': '***', 'WANDB_API_KEY': '***'}} [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) INFO 06-08 20:42:07 [scheduler.py:224] Chunked prefill is enabled with max_num_batched_tokens=65536. [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) INFO 06-08 20:42:07 [vllm.py:690] Asynchronous scheduling is enabled. [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) WARNING 06-08 20:42:07 [vllm.py:728] Enforce eager set, overriding optimization level to -O0 [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) INFO 06-08 20:42:07 [vllm.py:846] Cudagraph is disabled under eager mode [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) WARNING 06-08 20:42:07 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) WARNING 06-08 20:42:07 [system_utils.py:140] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 11x across cluster]
(pid=2794925, ip=10.128.32.52) INFO 06-08 20:42:08 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s [repeated 14x across cluster]
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) (EngineCore_DP0 pid=2690185) INFO 06-08 20:42:12 [core.py:97] Initializing a V1 LLM engine (vdev) with config: model='/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', speculative_config=None, tokenizer='/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=42, served_model_name=0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.NONE: 0>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'splitting_ops': [], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [65536], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.NONE: 0>, 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': 0, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': True, 'static_all_moe_layers': []}
(AsyncVLLMInferenceEngine pid=3947798, ip=10.128.32.58) WARNING 06-08 20:42:10 [arg_utils.py:1256] The global random seed is set to 60. Since VLLM_ENABLE_V1_MULTIPROCESSING is set to False, this may affect the random state of the Python process that launched vLLM. [repeated 8x across cluster]
(AsyncVLLMInferenceEngine pid=3947798, ip=10.128.32.58) INFO 06-08 20:42:10 [model.py:529] Resolved architecture: Qwen3ForCausalLM [repeated 8x across cluster]
(AsyncVLLMInferenceEngine pid=3947798, ip=10.128.32.58) INFO 06-08 20:42:10 [model.py:1549] Using max model len 32768 [repeated 8x across cluster]
(AsyncVLLMInferenceEngine pid=3947798, ip=10.128.32.58) INFO 06-08 20:42:10 [arg_utils.py:1469] Using ray runtime env (env vars redacted): {'env_vars': {'NCCL_CUMEM_ENABLE': '***', 'NCCL_DEBUG': '***', 'NCCL_SOCKET_FAMILY': '***', 'NCCL_SOCKET_IFNAME': '***', 'RAY_ADDRESS': '***', 'VLLM_ALLOW_INSECURE_SERIALIZATION': '***', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': '***', 'VLLM_DISABLE_COMPILE_CACHE': '***', 'WANDB_API_KEY': '***'}} [repeated 8x across cluster]
(AsyncVLLMInferenceEngine pid=3947798, ip=10.128.32.58) INFO 06-08 20:42:10 [scheduler.py:224] Chunked prefill is enabled with max_num_batched_tokens=65536. [repeated 8x across cluster]
(AsyncVLLMInferenceEngine pid=3947798, ip=10.128.32.58) INFO 06-08 20:42:10 [vllm.py:690] Asynchronous scheduling is enabled. [repeated 8x across cluster]
(AsyncVLLMInferenceEngine pid=3947798, ip=10.128.32.58) WARNING 06-08 20:42:10 [vllm.py:728] Enforce eager set, overriding optimization level to -O0 [repeated 8x across cluster]
(AsyncVLLMInferenceEngine pid=3947798, ip=10.128.32.58) INFO 06-08 20:42:10 [vllm.py:846] Cudagraph is disabled under eager mode [repeated 8x across cluster]
(AsyncVLLMInferenceEngine pid=2999029, ip=10.128.32.55) WARNING 06-08 20:42:13 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 [repeated 9x across cluster]
(AsyncVLLMInferenceEngine pid=2999029, ip=10.128.32.55) WARNING 06-08 20:42:13 [system_utils.py:140] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 9x across cluster]
(pid=4003708, ip=10.128.32.60) INFO 06-08 20:42:13 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s [repeated 10x across cluster]
(AsyncVLLMInferenceEngine pid=3869941, ip=10.128.32.57) (EngineCore_DP0 pid=3870176) INFO 06-08 20:42:17 [core.py:97] Initializing a V1 LLM engine (vdev) with config: model='/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', speculative_config=None, tokenizer='/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=48, served_model_name=0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.NONE: 0>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'splitting_ops': [], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [65536], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.NONE: 0>, 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': 0, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': True, 'static_all_moe_layers': []} [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2794925, ip=10.128.32.52) WARNING 06-08 20:42:15 [arg_utils.py:1256] The global random seed is set to 79. Since VLLM_ENABLE_V1_MULTIPROCESSING is set to False, this may affect the random state of the Python process that launched vLLM. [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2794925, ip=10.128.32.52) INFO 06-08 20:42:15 [model.py:529] Resolved architecture: Qwen3ForCausalLM [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2794925, ip=10.128.32.52) INFO 06-08 20:42:15 [model.py:1549] Using max model len 32768 [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2794925, ip=10.128.32.52) INFO 06-08 20:42:15 [arg_utils.py:1469] Using ray runtime env (env vars redacted): {'env_vars': {'NCCL_CUMEM_ENABLE': '***', 'NCCL_DEBUG': '***', 'NCCL_SOCKET_FAMILY': '***', 'NCCL_SOCKET_IFNAME': '***', 'RAY_ADDRESS': '***', 'VLLM_ALLOW_INSECURE_SERIALIZATION': '***', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': '***', 'VLLM_DISABLE_COMPILE_CACHE': '***', 'WANDB_API_KEY': '***'}} [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2794925, ip=10.128.32.52) INFO 06-08 20:42:15 [scheduler.py:224] Chunked prefill is enabled with max_num_batched_tokens=65536. [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2794925, ip=10.128.32.52) INFO 06-08 20:42:15 [vllm.py:690] Asynchronous scheduling is enabled. [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2794925, ip=10.128.32.52) WARNING 06-08 20:42:15 [vllm.py:728] Enforce eager set, overriding optimization level to -O0 [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2794925, ip=10.128.32.52) INFO 06-08 20:42:15 [vllm.py:846] Cudagraph is disabled under eager mode [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2729962, ip=10.128.32.53) (EngineCore_DP0 pid=2730194) INFO 06-08 20:42:17 [worker_base.py:289] Injected <class 'skyrl_train.inference_engines.vllm.vllm_engine.WorkerWrap'> into <class 'vllm.v1.worker.gpu_worker.Worker'> for extended collective_rpc calls ['_apply_fp8_weight_loader_patches', '_is_fp8_model', '_quantize_weights_for_fp8', '_restore_param_subclasses', '_undo_param_subclasses', 'begin_weight_update', 'destroy_weights_update_group', 'end_weight_update', 'init_weight_update_communicator', 'load_weights', 'read_named_weights', 'set_numa_affinity', 'test_rpc']
(AsyncVLLMInferenceEngine pid=2794925, ip=10.128.32.52) WARNING 06-08 20:42:15 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=2794925, ip=10.128.32.52) WARNING 06-08 20:42:15 [system_utils.py:140] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 11x across cluster]
(AsyncVLLMInferenceEngine pid=3947796, ip=10.128.32.58) INFO 06-08 20:42:18 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s [repeated 25x across cluster]
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) (EngineCore_DP0 pid=2690185) INFO 06-08 20:42:20 [parallel_state.py:1234] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.128.32.54:38653 backend=nccl
(AsyncVLLMInferenceEngine pid=2729961, ip=10.128.32.53) (EngineCore_DP0 pid=2730198) INFO 06-08 20:42:20 [parallel_state.py:1445] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) (EngineCore_DP0 pid=2690185) INFO 06-08 20:42:20 [gpu_model_runner.py:4125] Starting to load model /e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6...
(pid=3886563, ip=10.128.32.62) ⚙️ Running in WANDB offline mode
(AsyncVLLMInferenceEngine pid=2689962, ip=10.128.32.54) (EngineCore_DP0 pid=2690189) INFO 06-08 20:42:22 [cuda.py:367] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION'].
(AsyncVLLMInferenceEngine pid=2999029, ip=10.128.32.55) (EngineCore_DP0 pid=2999257) INFO 06-08 20:42:21 [core.py:97] Initializing a V1 LLM engine (vdev) with config: model='/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', speculative_config=None, tokenizer='/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=68, served_model_name=0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.NONE: 0>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'splitting_ops': [], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [65536], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.NONE: 0>, 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': 0, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': True, 'static_all_moe_layers': []} [repeated 9x across cluster]
(AsyncVLLMInferenceEngine pid=3893191, ip=10.128.32.61) WARNING 06-08 20:42:22 [arg_utils.py:1256] The global random seed is set to 87. Since VLLM_ENABLE_V1_MULTIPROCESSING is set to False, this may affect the random state of the Python process that launched vLLM. [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3893191, ip=10.128.32.61) INFO 06-08 20:42:22 [model.py:529] Resolved architecture: Qwen3ForCausalLM [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3893191, ip=10.128.32.61) INFO 06-08 20:42:22 [model.py:1549] Using max model len 32768 [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3893191, ip=10.128.32.61) INFO 06-08 20:42:22 [arg_utils.py:1469] Using ray runtime env (env vars redacted): {'env_vars': {'NCCL_CUMEM_ENABLE': '***', 'NCCL_DEBUG': '***', 'NCCL_SOCKET_FAMILY': '***', 'NCCL_SOCKET_IFNAME': '***', 'RAY_ADDRESS': '***', 'VLLM_ALLOW_INSECURE_SERIALIZATION': '***', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': '***', 'VLLM_DISABLE_COMPILE_CACHE': '***', 'WANDB_API_KEY': '***'}} [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3893191, ip=10.128.32.61) INFO 06-08 20:42:22 [scheduler.py:224] Chunked prefill is enabled with max_num_batched_tokens=65536. [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3893191, ip=10.128.32.61) INFO 06-08 20:42:22 [vllm.py:690] Asynchronous scheduling is enabled. [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3893191, ip=10.128.32.61) WARNING 06-08 20:42:22 [vllm.py:728] Enforce eager set, overriding optimization level to -O0 [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3893191, ip=10.128.32.61) INFO 06-08 20:42:22 [vllm.py:846] Cudagraph is disabled under eager mode [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) (EngineCore_DP0 pid=3948034) INFO 06-08 20:42:22 [worker_base.py:289] Injected <class 'skyrl_train.inference_engines.vllm.vllm_engine.WorkerWrap'> into <class 'vllm.v1.worker.gpu_worker.Worker'> for extended collective_rpc calls ['_apply_fp8_weight_loader_patches', '_is_fp8_model', '_quantize_weights_for_fp8', '_restore_param_subclasses', '_undo_param_subclasses', 'begin_weight_update', 'destroy_weights_update_group', 'end_weight_update', 'init_weight_update_communicator', 'load_weights', 'read_named_weights', 'set_numa_affinity', 'test_rpc'] [repeated 17x across cluster]
(AsyncVLLMInferenceEngine pid=3893191, ip=10.128.32.61) WARNING 06-08 20:42:22 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3893191, ip=10.128.32.61) WARNING 06-08 20:42:22 [system_utils.py:140] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3120027, ip=10.128.32.50) INFO 06-08 20:42:23 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s [repeated 15x across cluster]
(AsyncVLLMInferenceEngine pid=3947669, ip=10.128.32.58) (EngineCore_DP0 pid=3948029) INFO 06-08 20:42:24 [parallel_state.py:1234] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.128.32.58:42977 backend=nccl [repeated 19x across cluster]
(AsyncVLLMInferenceEngine pid=3947669, ip=10.128.32.58) (EngineCore_DP0 pid=3948029) INFO 06-08 20:42:24 [parallel_state.py:1445] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A [repeated 19x across cluster]
(AsyncVLLMInferenceEngine pid=3947669, ip=10.128.32.58) (EngineCore_DP0 pid=3948029) INFO 06-08 20:42:24 [gpu_model_runner.py:4125] Starting to load model /e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6... [repeated 19x across cluster]
(AsyncVLLMInferenceEngine pid=3947669, ip=10.128.32.58) (EngineCore_DP0 pid=3948029) INFO 06-08 20:42:25 [cuda.py:367] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION']. [repeated 19x across cluster]
(AsyncVLLMInferenceEngine pid=3120027, ip=10.128.32.50) (EngineCore_DP0 pid=3120402) INFO 06-08 20:42:24 [core.py:97] Initializing a V1 LLM engine (vdev) with config: model='/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', speculative_config=None, tokenizer='/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=62, served_model_name=0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.NONE: 0>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'splitting_ops': [], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [65536], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.NONE: 0>, 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': 0, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': True, 'static_all_moe_layers': []} [repeated 10x across cluster]
(AsyncVLLMInferenceEngine pid=3120157, ip=10.128.32.50) (EngineCore_DP0 pid=3120386) INFO 06-08 20:42:27 [worker_base.py:289] Injected <class 'skyrl_train.inference_engines.vllm.vllm_engine.WorkerWrap'> into <class 'vllm.v1.worker.gpu_worker.Worker'> for extended collective_rpc calls ['_apply_fp8_weight_loader_patches', '_is_fp8_model', '_quantize_weights_for_fp8', '_restore_param_subclasses', '_undo_param_subclasses', 'begin_weight_update', 'destroy_weights_update_group', 'end_weight_update', 'init_weight_update_communicator', 'load_weights', 'read_named_weights', 'set_numa_affinity', 'test_rpc'] [repeated 10x across cluster]
(pid=3886639, ip=10.128.32.62) ⚙️ Running in WANDB offline mode
(AsyncVLLMInferenceEngine pid=768604, ip=10.128.32.59) INFO 06-08 20:42:29 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s [repeated 8x across cluster]
(AsyncVLLMInferenceEngine pid=3120156, ip=10.128.32.50) (EngineCore_DP0 pid=3120407) INFO 06-08 20:42:29 [parallel_state.py:1234] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.128.32.50:36953 backend=nccl [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=3120156, ip=10.128.32.50) (EngineCore_DP0 pid=3120407) INFO 06-08 20:42:29 [parallel_state.py:1445] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=3120156, ip=10.128.32.50) (EngineCore_DP0 pid=3120407) INFO 06-08 20:42:30 [gpu_model_runner.py:4125] Starting to load model /e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6... [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=3120156, ip=10.128.32.50) (EngineCore_DP0 pid=3120407) INFO 06-08 20:42:31 [cuda.py:367] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION']. [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=3893192, ip=10.128.32.61) (EngineCore_DP0 pid=3893434) INFO 06-08 20:42:31 [core.py:97] Initializing a V1 LLM engine (vdev) with config: model='/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', speculative_config=None, tokenizer='/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=89, served_model_name=0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.NONE: 0>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'splitting_ops': [], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [65536], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.NONE: 0>, 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': 0, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': True, 'static_all_moe_layers': []} [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3893064, ip=10.128.32.61) (EngineCore_DP0 pid=3893429) INFO 06-08 20:42:33 [worker_base.py:289] Injected <class 'skyrl_train.inference_engines.vllm.vllm_engine.WorkerWrap'> into <class 'vllm.v1.worker.gpu_worker.Worker'> for extended collective_rpc calls ['_apply_fp8_weight_loader_patches', '_is_fp8_model', '_quantize_weights_for_fp8', '_restore_param_subclasses', '_undo_param_subclasses', 'begin_weight_update', 'destroy_weights_update_group', 'end_weight_update', 'init_weight_update_communicator', 'load_weights', 'read_named_weights', 'set_numa_affinity', 'test_rpc'] [repeated 12x across cluster]
(pid=546824) ⚙️ Running in WANDB offline mode [repeated 6x across cluster]
(AsyncVLLMInferenceEngine pid=3893192, ip=10.128.32.61) INFO 06-08 20:42:30 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s [repeated 8x across cluster]
(AsyncVLLMInferenceEngine pid=3893192, ip=10.128.32.61) (EngineCore_DP0 pid=3893434) INFO 06-08 20:42:35 [parallel_state.py:1234] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.128.32.61:59347 backend=nccl [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3893192, ip=10.128.32.61) (EngineCore_DP0 pid=3893434) INFO 06-08 20:42:35 [parallel_state.py:1445] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3893192, ip=10.128.32.61) (EngineCore_DP0 pid=3893434) INFO 06-08 20:42:35 [gpu_model_runner.py:4125] Starting to load model /e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6... [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) (EngineCore_DP0 pid=2690185) INFO 06-08 20:42:36 [default_loader.py:293] Loading weights took 14.13 seconds
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) (EngineCore_DP0 pid=2690185) INFO 06-08 20:42:36 [gpu_model_runner.py:4222] Model loading took 15.27 GiB memory and 15.646981 seconds
(AsyncVLLMInferenceEngine pid=3893192, ip=10.128.32.61) (EngineCore_DP0 pid=3893434) INFO 06-08 20:42:36 [cuda.py:367] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION']. [repeated 16x across cluster]
(AsyncVLLMInferenceEngine pid=3893192, ip=10.128.32.61) (EngineCore_DP0 pid=3893434) INFO 06-08 20:42:35 [worker_base.py:289] Injected <class 'skyrl_train.inference_engines.vllm.vllm_engine.WorkerWrap'> into <class 'vllm.v1.worker.gpu_worker.Worker'> for extended collective_rpc calls ['_apply_fp8_weight_loader_patches', '_is_fp8_model', '_quantize_weights_for_fp8', '_restore_param_subclasses', '_undo_param_subclasses', 'begin_weight_update', 'destroy_weights_update_group', 'end_weight_update', 'init_weight_update_communicator', 'load_weights', 'read_named_weights', 'set_numa_affinity', 'test_rpc'] [repeated 8x across cluster]
(AsyncVLLMInferenceEngine pid=321222, ip=10.128.32.56) (EngineCore_DP0 pid=321457) INFO 06-08 20:42:39 [gpu_worker.py:373] Available KV cache memory: 49.32 GiB
(AsyncVLLMInferenceEngine pid=321222, ip=10.128.32.56) (EngineCore_DP0 pid=321457) INFO 06-08 20:42:39 [kv_cache_utils.py:1307] GPU KV cache size: 359,120 tokens
(AsyncVLLMInferenceEngine pid=321222, ip=10.128.32.56) (EngineCore_DP0 pid=321457) INFO 06-08 20:42:39 [kv_cache_utils.py:1312] Maximum concurrency for 32,768 tokens per request: 10.96x
(AsyncVLLMInferenceEngine pid=2730089, ip=10.128.32.53) (EngineCore_DP0 pid=2730214) INFO 06-08 20:42:40 [kernel_warmup.py:44] Skipping FlashInfer autotune because it is disabled.
(AsyncVLLMInferenceEngine pid=3869809, ip=10.128.32.57) (EngineCore_DP0 pid=3870171) INFO 06-08 20:42:40 [core.py:278] init engine (profile, create kv cache, warmup model) took 3.33 seconds
(AsyncVLLMInferenceEngine pid=3869941, ip=10.128.32.57) (EngineCore_DP0 pid=3870176) WARNING 06-08 20:42:40 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1
(AsyncVLLMInferenceEngine pid=3869941, ip=10.128.32.57) (EngineCore_DP0 pid=3870176) INFO 06-08 20:42:40 [vllm.py:690] Asynchronous scheduling is enabled.
(AsyncVLLMInferenceEngine pid=3869941, ip=10.128.32.57) (EngineCore_DP0 pid=3870176) WARNING 06-08 20:42:40 [vllm.py:735] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored.
(AsyncVLLMInferenceEngine pid=3869941, ip=10.128.32.57) (EngineCore_DP0 pid=3870176) INFO 06-08 20:42:40 [vllm.py:846] Cudagraph is disabled under eager mode
(AsyncVLLMInferenceEngine pid=2689961, ip=10.128.32.54) WARNING 06-08 20:42:40 [model.py:1350] Default vLLM sampling parameters have been overridden by the model's `generation_config.json`: `{'temperature': 0.6, 'top_k': 20, 'top_p': 0.95}`. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`.
(AsyncVLLMInferenceEngine pid=2999155, ip=10.128.32.55) (EngineCore_DP0 pid=2999262) INFO 06-08 20:42:41 [default_loader.py:293] Loading weights took 13.38 seconds [repeated 23x across cluster]
(AsyncVLLMInferenceEngine pid=2999155, ip=10.128.32.55) (EngineCore_DP0 pid=2999262) INFO 06-08 20:42:41 [gpu_model_runner.py:4222] Model loading took 15.27 GiB memory and 14.428531 seconds [repeated 23x across cluster]
(AsyncVLLMInferenceEngine pid=2999029, ip=10.128.32.55) (EngineCore_DP0 pid=2999257) INFO 06-08 20:42:44 [gpu_worker.py:373] Available KV cache memory: 49.32 GiB [repeated 23x across cluster]
(AsyncVLLMInferenceEngine pid=2999029, ip=10.128.32.55) (EngineCore_DP0 pid=2999257) INFO 06-08 20:42:44 [kv_cache_utils.py:1307] GPU KV cache size: 359,104 tokens [repeated 23x across cluster]
(AsyncVLLMInferenceEngine pid=2999029, ip=10.128.32.55) (EngineCore_DP0 pid=2999257) INFO 06-08 20:42:44 [kv_cache_utils.py:1312] Maximum concurrency for 32,768 tokens per request: 10.96x [repeated 23x across cluster]
(AsyncVLLMInferenceEngine pid=2999029, ip=10.128.32.55) (EngineCore_DP0 pid=2999257) INFO 06-08 20:42:44 [kernel_warmup.py:44] Skipping FlashInfer autotune because it is disabled. [repeated 23x across cluster]
(AsyncVLLMInferenceEngine pid=2999155, ip=10.128.32.55) (EngineCore_DP0 pid=2999262) INFO 06-08 20:42:44 [core.py:278] init engine (profile, create kv cache, warmup model) took 2.76 seconds [repeated 23x across cluster]
(AsyncVLLMInferenceEngine pid=2999029, ip=10.128.32.55) (EngineCore_DP0 pid=2999257) WARNING 06-08 20:42:45 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 [repeated 23x across cluster]
(AsyncVLLMInferenceEngine pid=2999029, ip=10.128.32.55) (EngineCore_DP0 pid=2999257) INFO 06-08 20:42:45 [vllm.py:690] Asynchronous scheduling is enabled. [repeated 23x across cluster]
(AsyncVLLMInferenceEngine pid=2999029, ip=10.128.32.55) (EngineCore_DP0 pid=2999257) WARNING 06-08 20:42:45 [vllm.py:735] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored. [repeated 23x across cluster]
(AsyncVLLMInferenceEngine pid=2999029, ip=10.128.32.55) (EngineCore_DP0 pid=2999257) INFO 06-08 20:42:45 [vllm.py:846] Cudagraph is disabled under eager mode [repeated 23x across cluster]
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) WARNING 06-08 20:42:45 [model.py:1350] Default vLLM sampling parameters have been overridden by the model's `generation_config.json`: `{'temperature': 0.6, 'top_k': 20, 'top_p': 0.95}`. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`. [repeated 23x across cluster]
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62) NCCL version 2.27.7+cuda13.0
(AsyncVLLMInferenceEngine pid=3120156, ip=10.128.32.50) (EngineCore_DP0 pid=3120407) INFO 06-08 20:42:45 [default_loader.py:293] Loading weights took 14.23 seconds [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=3120156, ip=10.128.32.50) (EngineCore_DP0 pid=3120407) INFO 06-08 20:42:46 [gpu_model_runner.py:4222] Model loading took 15.27 GiB memory and 15.346670 seconds [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=3120157, ip=10.128.32.50) (EngineCore_DP0 pid=3120386) INFO 06-08 20:42:48 [gpu_worker.py:373] Available KV cache memory: 49.32 GiB [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=3120157, ip=10.128.32.50) (EngineCore_DP0 pid=3120386) INFO 06-08 20:42:48 [kv_cache_utils.py:1307] GPU KV cache size: 359,104 tokens [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=3120157, ip=10.128.32.50) (EngineCore_DP0 pid=3120386) INFO 06-08 20:42:48 [kv_cache_utils.py:1312] Maximum concurrency for 32,768 tokens per request: 10.96x [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=3120156, ip=10.128.32.50) (EngineCore_DP0 pid=3120407) INFO 06-08 20:42:48 [kernel_warmup.py:44] Skipping FlashInfer autotune because it is disabled. [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=3120157, ip=10.128.32.50) (EngineCore_DP0 pid=3120386) INFO 06-08 20:42:49 [core.py:278] init engine (profile, create kv cache, warmup model) took 2.84 seconds [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=3120157, ip=10.128.32.50) (EngineCore_DP0 pid=3120386) WARNING 06-08 20:42:49 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=3120157, ip=10.128.32.50) (EngineCore_DP0 pid=3120386) INFO 06-08 20:42:49 [vllm.py:690] Asynchronous scheduling is enabled. [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=3120157, ip=10.128.32.50) (EngineCore_DP0 pid=3120386) WARNING 06-08 20:42:49 [vllm.py:735] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored. [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=3120157, ip=10.128.32.50) (EngineCore_DP0 pid=3120386) INFO 06-08 20:42:49 [vllm.py:846] Cudagraph is disabled under eager mode [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=3120157, ip=10.128.32.50) WARNING 06-08 20:42:49 [model.py:1350] Default vLLM sampling parameters have been overridden by the model's `generation_config.json`: `{'temperature': 0.6, 'top_k': 20, 'top_p': 0.95}`. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`. [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2916172, ip=10.128.32.51) (EngineCore_DP0 pid=2916405) INFO 06-08 20:42:49 [default_loader.py:293] Loading weights took 13.80 seconds [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2916172, ip=10.128.32.51) (EngineCore_DP0 pid=2916405) INFO 06-08 20:42:50 [gpu_model_runner.py:4222] Model loading took 15.27 GiB memory and 14.929588 seconds [repeated 12x across cluster]
(RolloutCoordinator pid=547199) [fd-monitor] Started monitoring (every 120s)
(RolloutCoordinator pid=547199) [fd-monitor] [20:42:54] OK: 46 / 131,072 FDs open (0.0% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=547199) [fd-monitor] [20:42:54] OK: RSS 0.78 GiB | node mem 235.5/858.0 GiB used (27.5%), avail 622.4 GiB
(AsyncVLLMInferenceEngine pid=2916173, ip=10.128.32.51) (EngineCore_DP0 pid=2916417) INFO 06-08 20:42:52 [gpu_worker.py:373] Available KV cache memory: 49.32 GiB [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2916173, ip=10.128.32.51) (EngineCore_DP0 pid=2916417) INFO 06-08 20:42:52 [kv_cache_utils.py:1307] GPU KV cache size: 359,104 tokens [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2916173, ip=10.128.32.51) (EngineCore_DP0 pid=2916417) INFO 06-08 20:42:52 [kv_cache_utils.py:1312] Maximum concurrency for 32,768 tokens per request: 10.96x [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2916174, ip=10.128.32.51) (EngineCore_DP0 pid=2916413) INFO 06-08 20:42:52 [kernel_warmup.py:44] Skipping FlashInfer autotune because it is disabled. [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2916172, ip=10.128.32.51) (EngineCore_DP0 pid=2916405) INFO 06-08 20:42:52 [core.py:278] init engine (profile, create kv cache, warmup model) took 2.80 seconds [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2916172, ip=10.128.32.51) (EngineCore_DP0 pid=2916405) WARNING 06-08 20:42:53 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2916172, ip=10.128.32.51) (EngineCore_DP0 pid=2916405) INFO 06-08 20:42:53 [vllm.py:690] Asynchronous scheduling is enabled. [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2916172, ip=10.128.32.51) (EngineCore_DP0 pid=2916405) WARNING 06-08 20:42:53 [vllm.py:735] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored. [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2916172, ip=10.128.32.51) (EngineCore_DP0 pid=2916405) INFO 06-08 20:42:53 [vllm.py:846] Cudagraph is disabled under eager mode [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2916174, ip=10.128.32.51) WARNING 06-08 20:42:53 [model.py:1350] Default vLLM sampling parameters have been overridden by the model's `generation_config.json`: `{'temperature': 0.6, 'top_k': 20, 'top_p': 0.95}`. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`. [repeated 12x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56) [fd-monitor] Started monitoring (every 120s)
(RolloutCoordinator pid=321702, ip=10.128.32.56) [fd-monitor] [20:43:01] OK: 46 / 131,072 FDs open (0.0% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=321702, ip=10.128.32.56) [fd-monitor] [20:43:01] OK: RSS 0.78 GiB | node mem 399.2/858.0 GiB used (46.5%), avail 458.7 GiB
(RolloutCoordinator pid=2690439, ip=10.128.32.54) [fd-monitor] Started monitoring (every 120s)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) [fd-monitor] [20:43:10] OK: 46 / 131,072 FDs open (0.0% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) [fd-monitor] [20:43:10] OK: RSS 0.78 GiB | node mem 377.6/858.0 GiB used (44.0%), avail 480.4 GiB
(RolloutCoordinator pid=3120643, ip=10.128.32.50) [fd-monitor] Started monitoring (every 120s)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) [fd-monitor] [20:43:18] OK: 46 / 131,072 FDs open (0.0% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) [fd-monitor] [20:43:18] OK: RSS 0.78 GiB | node mem 415.1/858.0 GiB used (48.4%), avail 442.9 GiB
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62) [rank-0]: Loading model from /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/ablation-pymethods2test-seqnorm/checkpoints/global_step_83/policy/model_world_size_8_rank_0.pt
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62) [rank-0]: Loading extra_state from /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/ablation-pymethods2test-seqnorm/checkpoints/global_step_83/policy/extra_state_world_size_8_rank_0.pt
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62) [rank-0]: Loading optim from /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/ablation-pymethods2test-seqnorm/checkpoints/global_step_83/policy/optim_world_size_8_rank_0.pt
(skyrl_entrypoint pid=546644) [fd-monitor] [20:43:32] OK: 102 / 131,072 FDs open (0.1% of soft limit, hard limit: 131,072)
(skyrl_entrypoint pid=546644) [fd-monitor] [20:43:32] OK: RSS 1.64 GiB | node mem 271.4/858.0 GiB used (31.6%), avail 586.6 GiB
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62) [rank-0]: Successfully loaded model state dict
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62) [rank-0]: Successfully loaded optimizer state
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62) [rank-0]: Successfully loaded scheduler state
(FSDPPolicyWorkerBase pid=3886563, ip=10.128.32.62) [rank-0]: Checkpoint loaded successfully from /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/ablation-pymethods2test-seqnorm/checkpoints/global_step_83/policy
(FSDPPolicyWorkerBase pid=546822) INFO 06-08 20:43:40 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s
(AsyncVLLMInferenceEngine pid=2729834, ip=10.128.32.53) (EngineCore_DP0 pid=2730190) INFO 06-08 20:43:52 [block_pool.py:452] Successfully reset prefix cache
(FSDPPolicyWorkerBase pid=3886639, ip=10.128.32.62) INFO 06-08 20:43:42 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s [repeated 7x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(AsyncVLLMInferenceEngine pid=3120156, ip=10.128.32.50) (EngineCore_DP0 pid=3120407) INFO 06-08 20:43:52 [block_pool.py:452] Successfully reset prefix cache [repeated 47x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 1512x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 503x across cluster]
(AsyncVLLMInferenceEngine pid=3869940, ip=10.128.32.57) INFO 06-08 20:44:42 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
(AsyncVLLMInferenceEngine pid=3869940, ip=10.128.32.57) INFO 06-08 20:44:42 [loggers.py:259] Engine 000: Avg prompt throughput: 8.4 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.3%, Prefix cache hit rate: 0.0%
(AsyncVLLMInferenceEngine pid=3869940, ip=10.128.32.57) INFO 06-08 20:44:48 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 45.2 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.4%, Prefix cache hit rate: 0.0%
(AsyncVLLMInferenceEngine pid=2916173, ip=10.128.32.51) INFO 06-08 20:44:48 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
(AsyncVLLMInferenceEngine pid=2916173, ip=10.128.32.51) INFO 06-08 20:44:48 [loggers.py:259] Engine 000: Avg prompt throughput: 8.8 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.3%, Prefix cache hit rate: 0.0%
(AsyncVLLMInferenceEngine pid=4003580, ip=10.128.32.60) INFO 06-08 20:44:52 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. [repeated 2x across cluster]
(AsyncVLLMInferenceEngine pid=2916173, ip=10.128.32.51) INFO 06-08 20:44:53 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 47.0 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.3%, Prefix cache hit rate: 0.0% [repeated 4x across cluster]
(RolloutCoordinator pid=547199) [fd-monitor] [20:44:54] OK: 215 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=547199) [fd-monitor] [20:44:54] OK: RSS 1.07 GiB | node mem 256.5/858.0 GiB used (29.9%), avail 601.5 GiB
(AsyncVLLMInferenceEngine pid=2689828, ip=10.128.32.54) INFO 06-08 20:44:57 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
(AsyncVLLMInferenceEngine pid=2916173, ip=10.128.32.51) INFO 06-08 20:44:58 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 47.3 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.4%, Prefix cache hit rate: 0.0% [repeated 5x across cluster]
(AsyncVLLMInferenceEngine pid=2794792, ip=10.128.32.52) INFO 06-08 20:44:59 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
(RolloutCoordinator pid=321702, ip=10.128.32.56) [fd-monitor] [20:45:01] OK: 330 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=321702, ip=10.128.32.56) [fd-monitor] [20:45:01] OK: RSS 1.07 GiB | node mem 407.9/858.0 GiB used (47.5%), avail 450.1 GiB
(AsyncVLLMInferenceEngine pid=2916173, ip=10.128.32.51) INFO 06-08 20:45:03 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 47.7 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.5%, Prefix cache hit rate: 0.0% [repeated 7x across cluster]
(AsyncVLLMInferenceEngine pid=3893064, ip=10.128.32.61) INFO 06-08 20:45:04 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. [repeated 4x across cluster]
(AsyncVLLMInferenceEngine pid=3120027, ip=10.128.32.50) INFO 06-08 20:45:08 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 47.0 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.3%, Prefix cache hit rate: 0.0% [repeated 9x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54) [fd-monitor] [20:45:10] OK: 393 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) [fd-monitor] [20:45:10] OK: RSS 1.11 GiB | node mem 386.3/858.0 GiB used (45.0%), avail 471.7 GiB
(AsyncVLLMInferenceEngine pid=3869941, ip=10.128.32.57) INFO 06-08 20:45:10 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(AsyncVLLMInferenceEngine pid=2916174, ip=10.128.32.51) INFO 06-08 20:45:13 [loggers.py:259] Engine 000: Avg prompt throughput: 8.8 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.3%, Prefix cache hit rate: 0.0% [repeated 9x across cluster]
(AsyncVLLMInferenceEngine pid=768605, ip=10.128.32.59) INFO 06-08 20:45:15 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. [repeated 2x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50) [fd-monitor] [20:45:18] OK: 403 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) [fd-monitor] [20:45:18] OK: RSS 1.12 GiB | node mem 423.5/858.0 GiB used (49.4%), avail 434.4 GiB
(AsyncVLLMInferenceEngine pid=2916039, ip=10.128.32.51) INFO 06-08 20:45:18 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 43.7 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.5%, Prefix cache hit rate: 0.0% [repeated 12x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=2689963, ip=10.128.32.54) INFO 06-08 20:45:19 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) INFO 06-08 20:45:22 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(AsyncVLLMInferenceEngine pid=2916039, ip=10.128.32.51) INFO 06-08 20:45:23 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 43.9 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.5%, Prefix cache hit rate: 0.0% [repeated 12x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster]
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:45:26 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
(AsyncVLLMInferenceEngine pid=2916039, ip=10.128.32.51) INFO 06-08 20:45:28 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 44.0 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.6%, Prefix cache hit rate: 0.0% [repeated 12x across cluster]
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199) Provider List: https://docs.litellm.ai/docs/providers [repeated 2x across cluster]
(skyrl_entrypoint pid=546644) [fd-monitor] [20:45:32] OK: 135 / 131,072 FDs open (0.1% of soft limit, hard limit: 131,072)
(skyrl_entrypoint pid=546644) [fd-monitor] [20:45:32] OK: RSS 1.75 GiB | node mem 256.6/858.0 GiB used (29.9%), avail 601.4 GiB
(AsyncVLLMInferenceEngine pid=2916039, ip=10.128.32.51) INFO 06-08 20:45:33 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 44.1 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.6%, Prefix cache hit rate: 0.0% [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=2689963, ip=10.128.32.54) INFO 06-08 20:45:39 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 44.5 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.5%, Prefix cache hit rate: 0.0% [repeated 14x across cluster]
(AsyncVLLMInferenceEngine pid=3120027, ip=10.128.32.50) INFO 06-08 20:45:45 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 47.7 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.6%, Prefix cache hit rate: 53.2% [repeated 14x across cluster]
(AsyncVLLMInferenceEngine pid=768604, ip=10.128.32.59) INFO 06-08 20:45:49 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
(AsyncVLLMInferenceEngine pid=3120027, ip=10.128.32.50) INFO 06-08 20:45:51 [loggers.py:259] Engine 000: Avg prompt throughput: 153.9 tokens/s, Avg generation throughput: 74.9 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.1%, Prefix cache hit rate: 56.2% [repeated 17x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(AsyncVLLMInferenceEngine pid=2794924, ip=10.128.32.52) INFO 06-08 20:45:53 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. [repeated 5x across cluster]
(AsyncVLLMInferenceEngine pid=3120027, ip=10.128.32.50) INFO 06-08 20:45:56 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 86.4 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.3%, Prefix cache hit rate: 56.2% [repeated 20x across cluster]
(AsyncVLLMInferenceEngine pid=3947669, ip=10.128.32.58) INFO 06-08 20:45:59 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
(AsyncVLLMInferenceEngine pid=3120027, ip=10.128.32.50) INFO 06-08 20:46:01 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 86.5 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.4%, Prefix cache hit rate: 56.2% [repeated 22x across cluster]
(AsyncVLLMInferenceEngine pid=4003710, ip=10.128.32.60) INFO 06-08 20:46:04 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. [repeated 2x across cluster]
(AsyncVLLMInferenceEngine pid=3120027, ip=10.128.32.50) INFO 06-08 20:46:06 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 52.4 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.8%, Prefix cache hit rate: 56.2% [repeated 22x across cluster]
(AsyncVLLMInferenceEngine pid=2794925, ip=10.128.32.52) INFO 06-08 20:46:06 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. [repeated 2x across cluster]
(AsyncVLLMInferenceEngine pid=3120027, ip=10.128.32.50) INFO 06-08 20:46:11 [loggers.py:259] Engine 000: Avg prompt throughput: 76.6 tokens/s, Avg generation throughput: 55.2 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.0%, Prefix cache hit rate: 57.2% [repeated 27x across cluster]
(AsyncVLLMInferenceEngine pid=2999029, ip=10.128.32.55) INFO 06-08 20:46:11 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. [repeated 4x across cluster]
(AsyncVLLMInferenceEngine pid=2689962, ip=10.128.32.54) INFO 06-08 20:46:15 [loggers.py:259] Engine 000: Avg prompt throughput: 24.7 tokens/s, Avg generation throughput: 1.0 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.3%, Prefix cache hit rate: 32.8% [repeated 27x across cluster]
(AsyncVLLMInferenceEngine pid=2730089, ip=10.128.32.53) INFO 06-08 20:46:16 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. [repeated 3x across cluster]
(AsyncVLLMInferenceEngine pid=3120027, ip=10.128.32.50) INFO 06-08 20:46:21 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 225.2 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.3%, Prefix cache hit rate: 66.7% [repeated 31x across cluster]
(AsyncVLLMInferenceEngine pid=768606, ip=10.128.32.59) INFO 06-08 20:46:21 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. [repeated 2x across cluster]
(AsyncVLLMInferenceEngine pid=3120027, ip=10.128.32.50) INFO 06-08 20:46:26 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 191.1 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.6%, Prefix cache hit rate: 66.7% [repeated 31x across cluster]
(AsyncVLLMInferenceEngine pid=2729962, ip=10.128.32.53) INFO 06-08 20:46:29 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. [repeated 2x across cluster]
(AsyncVLLMInferenceEngine pid=4003709, ip=10.128.32.60) INFO 06-08 20:46:31 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 49.0 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.5%, Prefix cache hit rate: 0.0% [repeated 36x across cluster]
(AsyncVLLMInferenceEngine pid=3893192, ip=10.128.32.61) INFO 06-08 20:46:32 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
(AsyncVLLMInferenceEngine pid=4003709, ip=10.128.32.60) INFO 06-08 20:46:36 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 48.9 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.6%, Prefix cache hit rate: 0.0% [repeated 36x across cluster]
(AsyncVLLMInferenceEngine pid=4003709, ip=10.128.32.60) INFO 06-08 20:46:41 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 48.9 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.6%, Prefix cache hit rate: 0.0% [repeated 33x across cluster]
(AsyncVLLMInferenceEngine pid=4003709, ip=10.128.32.60) INFO 06-08 20:46:46 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 48.0 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.7%, Prefix cache hit rate: 0.0% [repeated 35x across cluster]
(AsyncVLLMInferenceEngine pid=4003709, ip=10.128.32.60) INFO 06-08 20:46:51 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 47.4 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.8%, Prefix cache hit rate: 0.0% [repeated 37x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(AsyncVLLMInferenceEngine pid=2999155, ip=10.128.32.55) INFO 06-08 20:46:53 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=547199) [fd-monitor] [20:46:54] OK: 256 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=547199) [fd-monitor] [20:46:54] OK: RSS 1.11 GiB | node mem 256.6/858.0 GiB used (29.9%), avail 601.4 GiB
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(AsyncVLLMInferenceEngine pid=4003709, ip=10.128.32.60) INFO 06-08 20:46:56 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 48.6 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.8%, Prefix cache hit rate: 0.0% [repeated 38x across cluster]
(RolloutCoordinator pid=547199) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster]
(AsyncVLLMInferenceEngine pid=4003708, ip=10.128.32.60) INFO 06-08 20:46:55 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. [repeated 2x across cluster]
(AsyncVLLMInferenceEngine pid=4003709, ip=10.128.32.60) INFO 06-08 20:47:01 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 48.2 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.9%, Prefix cache hit rate: 0.0% [repeated 36x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56) [fd-monitor] [20:47:01] OK: 260 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=321702, ip=10.128.32.56) [fd-monitor] [20:47:01] OK: RSS 1.12 GiB | node mem 408.0/858.0 GiB used (47.5%), avail 450.0 GiB
(AsyncVLLMInferenceEngine pid=3947796, ip=10.128.32.58) INFO 06-08 20:46:59 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(AsyncVLLMInferenceEngine pid=4003709, ip=10.128.32.60) INFO 06-08 20:47:06 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 48.5 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.0%, Prefix cache hit rate: 0.0% [repeated 38x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) [fd-monitor] [20:47:10] OK: 388 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) [fd-monitor] [20:47:10] OK: RSS 1.13 GiB | node mem 386.3/858.0 GiB used (45.0%), avail 471.7 GiB
(AsyncVLLMInferenceEngine pid=4003709, ip=10.128.32.60) INFO 06-08 20:47:11 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 48.4 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.0%, Prefix cache hit rate: 0.0% [repeated 38x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 2x across cluster]
(AsyncVLLMInferenceEngine pid=4003709, ip=10.128.32.60) INFO 06-08 20:47:16 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 48.3 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.1%, Prefix cache hit rate: 0.0% [repeated 36x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) [fd-monitor] [20:47:18] OK: 404 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) [fd-monitor] [20:47:18] OK: RSS 1.13 GiB | node mem 423.8/858.0 GiB used (49.4%), avail 434.2 GiB
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(AsyncVLLMInferenceEngine pid=2999030, ip=10.128.32.55) INFO 06-08 20:47:21 [loggers.py:259] Engine 000: Avg prompt throughput: 88.4 tokens/s, Avg generation throughput: 30.7 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.2%, Prefix cache hit rate: 76.0% [repeated 39x across cluster]
(RolloutCoordinator pid=547199) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster]
(AsyncVLLMInferenceEngine pid=768606, ip=10.128.32.59) INFO 06-08 20:47:26 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 188.0 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.9%, Prefix cache hit rate: 69.0% [repeated 43x across cluster]
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster]
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:47:31 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 46.5 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.4%, Prefix cache hit rate: 63.1% [repeated 38x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(skyrl_entrypoint pid=546644) [fd-monitor] [20:47:32] OK: 228 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072)
(skyrl_entrypoint pid=546644) [fd-monitor] [20:47:32] OK: RSS 1.76 GiB | node mem 256.6/858.0 GiB used (29.9%), avail 601.4 GiB
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster]
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:47:36 [loggers.py:259] Engine 000: Avg prompt throughput: 153.0 tokens/s, Avg generation throughput: 82.7 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.5%, Prefix cache hit rate: 67.6% [repeated 40x across cluster]
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:47:41 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 92.5 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.6%, Prefix cache hit rate: 67.6% [repeated 38x across cluster]
(AsyncVLLMInferenceEngine pid=3947798, ip=10.128.32.58) INFO 06-08 20:47:43 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:47:46 [loggers.py:259] Engine 000: Avg prompt throughput: 75.9 tokens/s, Avg generation throughput: 70.1 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.3%, Prefix cache hit rate: 70.5% [repeated 37x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:47:51 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 91.3 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.4%, Prefix cache hit rate: 70.5% [repeated 39x across cluster]
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster]
(AsyncVLLMInferenceEngine pid=3120157, ip=10.128.32.50) INFO 06-08 20:47:54 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:47:56 [loggers.py:259] Engine 000: Avg prompt throughput: 140.2 tokens/s, Avg generation throughput: 108.4 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.9%, Prefix cache hit rate: 73.9% [repeated 42x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster]
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:48:01 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 134.0 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.2%, Prefix cache hit rate: 73.9% [repeated 42x across cluster]
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster]
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:48:06 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 90.1 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.3%, Prefix cache hit rate: 73.9% [repeated 41x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster]
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:48:11 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 90.3 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.4%, Prefix cache hit rate: 73.9% [repeated 45x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) INFO 06-08 20:48:16 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
(AsyncVLLMInferenceEngine pid=3120156, ip=10.128.32.50) INFO 06-08 20:48:16 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 50.8 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.2%, Prefix cache hit rate: 84.6% [repeated 44x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:48:21 [loggers.py:259] Engine 000: Avg prompt throughput: 119.5 tokens/s, Avg generation throughput: 166.8 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.8%, Prefix cache hit rate: 75.5% [repeated 44x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:48:26 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 136.6 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 5.0%, Prefix cache hit rate: 75.5% [repeated 42x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 6x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:48:31 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 129.7 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.0%, Prefix cache hit rate: 75.5% [repeated 40x across cluster]
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 27x across cluster]
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:48:37 [loggers.py:259] Engine 000: Avg prompt throughput: 76.6 tokens/s, Avg generation throughput: 170.5 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 5.5%, Prefix cache hit rate: 78.4% [repeated 41x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster]
(AsyncVLLMInferenceEngine pid=321092, ip=10.128.32.56) INFO 06-08 20:48:40 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:48:42 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 183.5 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 5.7%, Prefix cache hit rate: 78.4% [repeated 44x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster]
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:48:47 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 185.4 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 6.0%, Prefix cache hit rate: 78.4% [repeated 45x across cluster]
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:48:52 [loggers.py:259] Engine 000: Avg prompt throughput: 72.9 tokens/s, Avg generation throughput: 199.7 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 6.6%, Prefix cache hit rate: 78.5% [repeated 46x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=547199) [fd-monitor] [20:48:54] OK: 284 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=547199) [fd-monitor] [20:48:54] OK: RSS 1.14 GiB | node mem 256.7/858.0 GiB used (29.9%), avail 601.3 GiB
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 16x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:48:57 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 185.2 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 5.0%, Prefix cache hit rate: 78.5% [repeated 45x across cluster]
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199) Provider List: https://docs.litellm.ai/docs/providers [repeated 19x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56) [fd-monitor] [20:49:01] OK: 383 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=321702, ip=10.128.32.56) [fd-monitor] [20:49:01] OK: RSS 1.16 GiB | node mem 408.0/858.0 GiB used (47.6%), avail 450.0 GiB
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:49:02 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 138.2 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 5.2%, Prefix cache hit rate: 78.5% [repeated 44x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster]
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:49:07 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 116.8 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.9%, Prefix cache hit rate: 78.5% [repeated 45x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) [fd-monitor] [20:49:10] OK: 345 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) [fd-monitor] [20:49:10] OK: RSS 1.14 GiB | node mem 386.3/858.0 GiB used (45.0%), avail 471.7 GiB
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(AsyncVLLMInferenceEngine pid=2689963, ip=10.128.32.54) INFO 06-08 20:49:11 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 138.8 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.5%, Prefix cache hit rate: 84.1% [repeated 44x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 46x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:49:17 [loggers.py:259] Engine 000: Avg prompt throughput: 67.2 tokens/s, Avg generation throughput: 122.2 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 5.6%, Prefix cache hit rate: 78.8% [repeated 45x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50) [fd-monitor] [20:49:18] OK: 264 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) [fd-monitor] [20:49:18] OK: RSS 1.14 GiB | node mem 423.8/858.0 GiB used (49.4%), avail 434.1 GiB
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:49:22 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 96.0 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.8%, Prefix cache hit rate: 78.8% [repeated 45x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:49:27 [loggers.py:259] Engine 000: Avg prompt throughput: 113.4 tokens/s, Avg generation throughput: 63.8 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.6%, Prefix cache hit rate: 78.9% [repeated 44x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:49:32 [loggers.py:259] Engine 000: Avg prompt throughput: 33.9 tokens/s, Avg generation throughput: 97.4 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.6%, Prefix cache hit rate: 79.8% [repeated 44x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(skyrl_entrypoint pid=546644) [fd-monitor] [20:49:32] OK: 287 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072)
(skyrl_entrypoint pid=546644) [fd-monitor] [20:49:32] OK: RSS 1.77 GiB | node mem 256.7/858.0 GiB used (29.9%), avail 601.3 GiB
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 21x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:49:37 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 72.7 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.0%, Prefix cache hit rate: 79.8% [repeated 45x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:49:42 [loggers.py:259] Engine 000: Avg prompt throughput: 22.7 tokens/s, Avg generation throughput: 50.2 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.8%, Prefix cache hit rate: 80.8% [repeated 45x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=3947797, ip=10.128.32.58) INFO 06-08 20:49:47 [loggers.py:259] Engine 000: Avg prompt throughput: 210.5 tokens/s, Avg generation throughput: 117.7 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 6.6%, Prefix cache hit rate: 82.3% [repeated 45x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster]
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) INFO 06-08 20:49:52 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 51.7 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.4%, Prefix cache hit rate: 75.6% [repeated 46x across cluster]
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 6x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) INFO 06-08 20:49:57 [loggers.py:259] Engine 000: Avg prompt throughput: 122.0 tokens/s, Avg generation throughput: 50.0 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.7%, Prefix cache hit rate: 80.2% [repeated 46x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 14x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) INFO 06-08 20:50:02 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 93.9 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.8%, Prefix cache hit rate: 80.2% [repeated 45x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) INFO 06-08 20:50:07 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 93.8 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.0%, Prefix cache hit rate: 80.2% [repeated 45x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 22x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) INFO 06-08 20:50:12 [loggers.py:259] Engine 000: Avg prompt throughput: 23.2 tokens/s, Avg generation throughput: 77.7 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.1%, Prefix cache hit rate: 85.4% [repeated 44x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=321222, ip=10.128.32.56) INFO 06-08 20:50:13 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 14x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) INFO 06-08 20:50:17 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 94.2 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.2%, Prefix cache hit rate: 85.4% [repeated 47x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster]
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) INFO 06-08 20:50:22 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 94.1 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.4%, Prefix cache hit rate: 85.4% [repeated 46x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 19x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) INFO 06-08 20:50:27 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 93.8 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.5%, Prefix cache hit rate: 85.4% [repeated 44x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) INFO 06-08 20:50:32 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 75.1 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.9%, Prefix cache hit rate: 85.4% [repeated 45x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) INFO 06-08 20:50:37 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 47.2 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.0%, Prefix cache hit rate: 85.4% [repeated 46x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) INFO 06-08 20:50:42 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 47.1 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.0%, Prefix cache hit rate: 85.4% [repeated 46x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) INFO 06-08 20:50:47 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 47.1 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.1%, Prefix cache hit rate: 85.4% [repeated 46x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(AsyncVLLMInferenceEngine pid=3869939, ip=10.128.32.57) INFO 06-08 20:50:52 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 47.1 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.2%, Prefix cache hit rate: 85.4% [repeated 47x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199) [fd-monitor] [20:50:54] OK: 273 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=547199) [fd-monitor] [20:50:54] OK: RSS 1.15 GiB | node mem 256.7/858.0 GiB used (29.9%), avail 601.3 GiB
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster]
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) INFO 06-08 20:50:57 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 123.5 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 8.3%, Prefix cache hit rate: 85.8% [repeated 46x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) Provider List: https://docs.litellm.ai/docs/providers [repeated 11x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56) [fd-monitor] [20:51:01] OK: 399 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=321702, ip=10.128.32.56) [fd-monitor] [20:51:01] OK: RSS 1.16 GiB | node mem 408.0/858.0 GiB used (47.6%), avail 450.0 GiB
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) INFO 06-08 20:51:02 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 92.4 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 8.5%, Prefix cache hit rate: 85.8% [repeated 45x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster]
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) INFO 06-08 20:51:07 [loggers.py:259] Engine 000: Avg prompt throughput: 136.6 tokens/s, Avg generation throughput: 135.6 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 9.6%, Prefix cache hit rate: 85.7% [repeated 45x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) [fd-monitor] [20:51:10] OK: 361 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) [fd-monitor] [20:51:10] OK: RSS 1.15 GiB | node mem 386.3/858.0 GiB used (45.0%), avail 471.6 GiB
(RolloutCoordinator pid=3120643, ip=10.128.32.50) Provider List: https://docs.litellm.ai/docs/providers [repeated 4x across cluster]
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) INFO 06-08 20:51:12 [loggers.py:259] Engine 000: Avg prompt throughput: 42.8 tokens/s, Avg generation throughput: 164.5 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 10.4%, Prefix cache hit rate: 85.9% [repeated 47x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster]
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) INFO 06-08 20:51:17 [loggers.py:259] Engine 000: Avg prompt throughput: 99.3 tokens/s, Avg generation throughput: 216.3 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 10.9%, Prefix cache hit rate: 85.5% [repeated 47x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50) [fd-monitor] [20:51:18] OK: 276 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) [fd-monitor] [20:51:18] OK: RSS 1.15 GiB | node mem 423.9/858.0 GiB used (49.4%), avail 434.1 GiB
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) INFO 06-08 20:51:22 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 206.8 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 10.4%, Prefix cache hit rate: 85.5% [repeated 49x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) INFO 06-08 20:51:27 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 184.9 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 10.7%, Prefix cache hit rate: 85.5% [repeated 47x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(skyrl_entrypoint pid=546644) [fd-monitor] [20:51:32] OK: 332 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072)
(skyrl_entrypoint pid=546644) [fd-monitor] [20:51:32] OK: RSS 1.78 GiB | node mem 256.7/858.0 GiB used (29.9%), avail 601.2 GiB
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) INFO 06-08 20:51:32 [loggers.py:259] Engine 000: Avg prompt throughput: 92.1 tokens/s, Avg generation throughput: 219.6 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 11.1%, Prefix cache hit rate: 85.1% [repeated 47x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster]
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) INFO 06-08 20:51:37 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 198.1 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 11.0%, Prefix cache hit rate: 85.1% [repeated 47x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) INFO 06-08 20:51:42 [loggers.py:259] Engine 000: Avg prompt throughput: 209.6 tokens/s, Avg generation throughput: 201.0 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 12.5%, Prefix cache hit rate: 84.9% [repeated 47x across cluster]
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) INFO 06-08 20:51:48 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 258.2 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 11.5%, Prefix cache hit rate: 84.9% [repeated 47x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) INFO 06-08 20:51:53 [loggers.py:259] Engine 000: Avg prompt throughput: 23.6 tokens/s, Avg generation throughput: 225.9 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 12.7%, Prefix cache hit rate: 85.4% [repeated 47x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) Provider List: https://docs.litellm.ai/docs/providers [repeated 11x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) INFO 06-08 20:51:58 [loggers.py:259] Engine 000: Avg prompt throughput: 35.0 tokens/s, Avg generation throughput: 236.4 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 12.6%, Prefix cache hit rate: 85.6% [repeated 47x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster]
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) INFO 06-08 20:52:03 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 191.5 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 12.3%, Prefix cache hit rate: 85.6% [repeated 47x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster]
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) INFO 06-08 20:52:08 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 184.7 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 12.6%, Prefix cache hit rate: 85.6% [repeated 47x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 30x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) INFO 06-08 20:52:13 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 155.7 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 12.1%, Prefix cache hit rate: 85.6% [repeated 48x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 19x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) INFO 06-08 20:52:18 [loggers.py:259] Engine 000: Avg prompt throughput: 235.2 tokens/s, Avg generation throughput: 169.3 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 13.2%, Prefix cache hit rate: 84.9% [repeated 48x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) INFO 06-08 20:52:23 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 141.5 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 12.5%, Prefix cache hit rate: 84.9% [repeated 48x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(AsyncVLLMInferenceEngine pid=2794923, ip=10.128.32.52) INFO 06-08 20:52:28 [loggers.py:259] Engine 000: Avg prompt throughput: 165.8 tokens/s, Avg generation throughput: 164.2 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 14.5%, Prefix cache hit rate: 85.3% [repeated 48x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=3947796, ip=10.128.32.58) INFO 06-08 20:52:33 [loggers.py:259] Engine 000: Avg prompt throughput: 27.5 tokens/s, Avg generation throughput: 56.6 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.5%, Prefix cache hit rate: 86.3% [repeated 48x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster]
(AsyncVLLMInferenceEngine pid=3947796, ip=10.128.32.58) INFO 06-08 20:52:38 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 91.5 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.6%, Prefix cache hit rate: 86.3% [repeated 47x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=3947796, ip=10.128.32.58) INFO 06-08 20:52:43 [loggers.py:259] Engine 000: Avg prompt throughput: 20.9 tokens/s, Avg generation throughput: 91.6 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.3%, Prefix cache hit rate: 86.6% [repeated 48x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(AsyncVLLMInferenceEngine pid=768476, ip=10.128.32.59) INFO 06-08 20:52:48 [loggers.py:259] Engine 000: Avg prompt throughput: 13.9 tokens/s, Avg generation throughput: 213.6 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 12.4%, Prefix cache hit rate: 93.7% [repeated 49x across cluster]
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=768476, ip=10.128.32.59) INFO 06-08 20:52:53 [loggers.py:259] Engine 000: Avg prompt throughput: 9.6 tokens/s, Avg generation throughput: 163.8 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 11.4%, Prefix cache hit rate: 93.9% [repeated 48x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199) [fd-monitor] [20:52:54] OK: 266 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=547199) [fd-monitor] [20:52:54] OK: RSS 1.15 GiB | node mem 256.7/858.0 GiB used (29.9%), avail 601.3 GiB
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 19x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=768476, ip=10.128.32.59) INFO 06-08 20:52:58 [loggers.py:259] Engine 000: Avg prompt throughput: 9.6 tokens/s, Avg generation throughput: 148.5 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 13.6%, Prefix cache hit rate: 94.1% [repeated 48x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 29x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56) [fd-monitor] [20:53:01] OK: 408 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=321702, ip=10.128.32.56) [fd-monitor] [20:53:01] OK: RSS 1.17 GiB | node mem 408.0/858.0 GiB used (47.6%), avail 450.0 GiB
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=768476, ip=10.128.32.59) INFO 06-08 20:53:03 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 179.0 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 11.8%, Prefix cache hit rate: 94.1% [repeated 48x across cluster]
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(AsyncVLLMInferenceEngine pid=3893064, ip=10.128.32.61) INFO 06-08 20:53:08 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 253.0 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 20.3%, Prefix cache hit rate: 87.7% [repeated 49x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54) [fd-monitor] [20:53:10] OK: 379 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) [fd-monitor] [20:53:10] OK: RSS 1.19 GiB | node mem 386.4/858.0 GiB used (45.0%), avail 471.6 GiB
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster]
(AsyncVLLMInferenceEngine pid=3893064, ip=10.128.32.61) INFO 06-08 20:53:13 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 252.6 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 20.6%, Prefix cache hit rate: 87.7% [repeated 48x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(AsyncVLLMInferenceEngine pid=3893064, ip=10.128.32.61) INFO 06-08 20:53:18 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 252.6 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 21.0%, Prefix cache hit rate: 87.7% [repeated 48x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50) [fd-monitor] [20:53:18] OK: 275 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) [fd-monitor] [20:53:18] OK: RSS 1.15 GiB | node mem 423.9/858.0 GiB used (49.4%), avail 434.1 GiB
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=3893064, ip=10.128.32.61) INFO 06-08 20:53:23 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 252.3 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 21.3%, Prefix cache hit rate: 87.7% [repeated 48x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) INFO 06-08 20:53:28 [loggers.py:259] Engine 000: Avg prompt throughput: 60.3 tokens/s, Avg generation throughput: 165.2 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 14.1%, Prefix cache hit rate: 82.7% [repeated 48x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(skyrl_entrypoint pid=546644) [fd-monitor] [20:53:32] OK: 339 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072)
(skyrl_entrypoint pid=546644) [fd-monitor] [20:53:32] OK: RSS 1.79 GiB | node mem 256.7/858.0 GiB used (29.9%), avail 601.3 GiB
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) INFO 06-08 20:53:33 [loggers.py:259] Engine 000: Avg prompt throughput: 63.3 tokens/s, Avg generation throughput: 191.4 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 14.0%, Prefix cache hit rate: 82.6% [repeated 48x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 36x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) INFO 06-08 20:53:38 [loggers.py:259] Engine 000: Avg prompt throughput: 56.1 tokens/s, Avg generation throughput: 196.8 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 16.4%, Prefix cache hit rate: 83.8% [repeated 48x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster]
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) INFO 06-08 20:53:43 [loggers.py:259] Engine 000: Avg prompt throughput: 187.6 tokens/s, Avg generation throughput: 233.2 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 17.2%, Prefix cache hit rate: 84.5% [repeated 48x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 41x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) INFO 06-08 20:53:48 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 205.9 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 14.7%, Prefix cache hit rate: 84.5% [repeated 48x across cluster]
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) INFO 06-08 20:53:53 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 170.6 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 14.9%, Prefix cache hit rate: 84.5% [repeated 49x across cluster]
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster]
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) INFO 06-08 20:53:58 [loggers.py:259] Engine 000: Avg prompt throughput: 64.9 tokens/s, Avg generation throughput: 191.6 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 15.8%, Prefix cache hit rate: 84.5% [repeated 48x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) INFO 06-08 20:54:03 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 213.4 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 16.1%, Prefix cache hit rate: 84.5% [repeated 48x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) INFO 06-08 20:54:08 [loggers.py:259] Engine 000: Avg prompt throughput: 52.8 tokens/s, Avg generation throughput: 254.3 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 17.0%, Prefix cache hit rate: 84.7% [repeated 48x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 31x across cluster]
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) INFO 06-08 20:54:13 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 217.7 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 16.0%, Prefix cache hit rate: 84.7% [repeated 48x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster]
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) INFO 06-08 20:54:18 [loggers.py:259] Engine 000: Avg prompt throughput: 106.0 tokens/s, Avg generation throughput: 193.1 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 17.0%, Prefix cache hit rate: 84.7% [repeated 48x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) INFO 06-08 20:54:23 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 213.7 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 17.3%, Prefix cache hit rate: 84.7% [repeated 48x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) INFO 06-08 20:54:28 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 213.5 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 17.6%, Prefix cache hit rate: 84.7% [repeated 48x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster]
(AsyncVLLMInferenceEngine pid=2730089, ip=10.128.32.53) INFO 06-08 20:54:33 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 284.3 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 24.4%, Prefix cache hit rate: 85.8% [repeated 48x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster]
(AsyncVLLMInferenceEngine pid=2730089, ip=10.128.32.53) INFO 06-08 20:54:38 [loggers.py:259] Engine 000: Avg prompt throughput: 33.5 tokens/s, Avg generation throughput: 317.3 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 25.1%, Prefix cache hit rate: 85.8% [repeated 48x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
(RolloutCoordinator pid=2690439, ip=10.128.32.54) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(AsyncVLLMInferenceEngine pid=2730089, ip=10.128.32.53) INFO 06-08 20:54:43 [loggers.py:259] Engine 000: Avg prompt throughput: 68.9 tokens/s, Avg generation throughput: 367.4 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 25.7%, Prefix cache hit rate: 85.7% [repeated 48x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
(RolloutCoordinator pid=2690439, ip=10.128.32.54) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(AsyncVLLMInferenceEngine pid=2730089, ip=10.128.32.53) INFO 06-08 20:54:48 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 378.3 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 26.2%, Prefix cache hit rate: 85.7% [repeated 48x across cluster]
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) Provider List: https://docs.litellm.ai/docs/providers [repeated 31x across cluster]
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(AsyncVLLMInferenceEngine pid=2730089, ip=10.128.32.53) INFO 06-08 20:54:53 [loggers.py:259] Engine 000: Avg prompt throughput: 60.8 tokens/s, Avg generation throughput: 398.5 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 26.9%, Prefix cache hit rate: 85.5% [repeated 48x across cluster]
(RolloutCoordinator pid=547199) [fd-monitor] [20:54:54] OK: 299 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=547199) [fd-monitor] [20:54:54] OK: RSS 1.20 GiB | node mem 256.7/858.0 GiB used (29.9%), avail 601.2 GiB
(RolloutCoordinator pid=3120643, ip=10.128.32.50) Provider List: https://docs.litellm.ai/docs/providers [repeated 2x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) INFO 06-08 20:54:58 [loggers.py:259] Engine 000: Avg prompt throughput: 73.8 tokens/s, Avg generation throughput: 305.6 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 20.3%, Prefix cache hit rate: 84.5% [repeated 48x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56) [fd-monitor] [20:55:01] OK: 431 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=321702, ip=10.128.32.56) [fd-monitor] [20:55:01] OK: RSS 1.21 GiB | node mem 408.0/858.0 GiB used (47.6%), avail 449.9 GiB
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) INFO 06-08 20:55:03 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 297.3 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 20.5%, Prefix cache hit rate: 84.5% [repeated 48x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) INFO 06-08 20:55:08 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 298.0 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 20.9%, Prefix cache hit rate: 84.5% [repeated 48x across cluster]
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) [fd-monitor] [20:55:10] OK: 407 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) [fd-monitor] [20:55:10] OK: RSS 1.20 GiB | node mem 386.4/858.0 GiB used (45.0%), avail 471.6 GiB
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
(RolloutCoordinator pid=2690439, ip=10.128.32.54) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(RolloutCoordinator pid=3120643, ip=10.128.32.50)
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) INFO 06-08 20:55:13 [loggers.py:259] Engine 000: Avg prompt throughput: 45.8 tokens/s, Avg generation throughput: 305.9 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 21.6%, Prefix cache hit rate: 84.5% [repeated 48x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster]
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=547199)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) [fd-monitor] [20:55:18] OK: 414 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072)
(RolloutCoordinator pid=3120643, ip=10.128.32.50) [fd-monitor] [20:55:18] OK: RSS 1.20 GiB | node mem 423.8/858.0 GiB used (49.4%), avail 434.1 GiB
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) INFO 06-08 20:55:18 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 310.2 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 21.8%, Prefix cache hit rate: 84.5% [repeated 48x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
(RolloutCoordinator pid=2690439, ip=10.128.32.54) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(AsyncVLLMInferenceEngine pid=2998900, ip=10.128.32.55) INFO 06-08 20:55:23 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 299.4 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 22.2%, Prefix cache hit rate: 84.5% [repeated 48x across cluster]
(AsyncVLLMInferenceEngine pid=2730089, ip=10.128.32.53) INFO 06-08 20:55:28 [loggers.py:259] Engine 000: Avg prompt throughput: 198.0 tokens/s, Avg generation throughput: 404.3 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 30.2%, Prefix cache hit rate: 85.2% [repeated 49x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(skyrl_entrypoint pid=546644) [fd-monitor] [20:55:32] OK: 458 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072)
(skyrl_entrypoint pid=546644) [fd-monitor] [20:55:32] OK: RSS 1.80 GiB | node mem 256.7/858.0 GiB used (29.9%), avail 601.2 GiB
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(AsyncVLLMInferenceEngine pid=2730089, ip=10.128.32.53) INFO 06-08 20:55:33 [loggers.py:259] Engine 000: Avg prompt throughput: 240.9 tokens/s, Avg generation throughput: 419.3 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.6%, Prefix cache hit rate: 84.8% [repeated 48x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=2730089, ip=10.128.32.53) INFO 06-08 20:55:38 [loggers.py:259] Engine 000: Avg prompt throughput: 236.9 tokens/s, Avg generation throughput: 396.2 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.1%, Prefix cache hit rate: 84.9% [repeated 48x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=2730089, ip=10.128.32.53) INFO 06-08 20:55:43 [loggers.py:259] Engine 000: Avg prompt throughput: 87.2 tokens/s, Avg generation throughput: 425.9 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.6%, Prefix cache hit rate: 84.7% [repeated 48x across cluster]
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
(RolloutCoordinator pid=2690439, ip=10.128.32.54) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56) Provider List: https://docs.litellm.ai/docs/providers [repeated 11x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=2690439, ip=10.128.32.54)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(RolloutCoordinator pid=321702, ip=10.128.32.56)
(AsyncVLLMInferenceEngine pid=2730089, ip=10.128.32.53) INFO 06-08 20:55:48 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 378.0 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.1%, Prefix cache hit rate: 84.7% [repeated 48x across cluster]
(RolloutCoordinator pid=321702, ip=10.128.32.56) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
(RolloutCoordinator pid=321702, ip=10.128.32.56) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.
(RolloutCoordinator pid=2690439, ip=10.128.32.54) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster]
(AsyncVLLMInferenceEngine pid=3893191, ip=10.128.32.61) INFO 06-08 20:55:53 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 187.7 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 13.3%, Prefix cache hit rate: 87.7% [repeated 49x across cluster]
(AsyncVLLMInferenceEngine pid=2916173, ip=10.128.32.51) INFO 06-08 20:55:54 [loggers.py:259] Engine 000: Avg prompt throughput: 552.6 tokens/s, Avg generation throughput: 360.7 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 20.3%, Prefix cache hit rate: 85.9% [repeated 4x across cluster]
Error executing job with overrides: ['+terminal_bench_config=terminal_bench', 'trainer.strategy=fsdp2', 'trainer.algorithm.advantage_estimator=rloo_n', 'trainer.algorithm.use_kl_loss=false', 'trainer.algorithm.kl_loss_coef=0.0', 'trainer.algorithm.eps_clip_low=0.2', 'trainer.algorithm.eps_clip_high=0.05', 'trainer.algorithm.loss_reduction=seq_mean_token_sum_norm_global', 'trainer.epochs=2', 'trainer.max_steps=80', 'trainer.update_epochs_per_batch=1', 'trainer.train_batch_size=64', 'trainer.policy_mini_batch_size=64', 'trainer.eval_batch_size=64', 'trainer.micro_forward_batch_size_per_gpu=4', 'trainer.micro_train_batch_size_per_gpu=1', 'trainer.max_prompt_length=999999', 'trainer.eval_interval=999999', 'trainer.eval_before_train=false', 'trainer.ckpt_interval=2', 'trainer.resume_mode=latest', 'trainer.hf_save_interval=5', '++trainer.hf_hub_repo_id=laion/ablation-pymethods2test-seqnorm', '++trainer.hf_hub_private=false', '++trainer.hf_hub_revision=main', '++trainer.enable_db_registration=false', 'trainer.project_name=OpenThoughts-Agent', 'trainer.log_level=INFO', 'trainer.tracker_commit_each_step=true', 'trainer.logger=console', 'trainer.run_name=ablation-pymethods2test-seqnorm', 'trainer.ckpt_path=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/ablation-pymethods2test-seqnorm/checkpoints', 'trainer.export_path=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/ablation-pymethods2test-seqnorm/exports', 'trainer.policy.optimizer_config.lr=8e-6', 'trainer.policy.optimizer_config.weight_decay=0.0', 'trainer.policy.optimizer_config.adam_betas=[0.9,0.999]', 'trainer.policy.optimizer_config.max_grad_norm=0.9', 'trainer.policy.fsdp_config.cpu_offload=false', 'trainer.policy.fsdp_config.reshard_after_forward=true', 'trainer.policy.fsdp_config.fsdp_size=4', 'trainer.policy.model.path=/e/data1/datasets/playground/ot-baf/hf_hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', 'trainer.ref.fsdp_config.cpu_offload=false', 'trainer.ref.fsdp_config.reshard_after_forward=true', 'trainer.ref.fsdp_config.fsdp_size=4', 'trainer.placement.colocate_all=false', 'trainer.placement.policy_num_nodes=2', 'trainer.placement.ref_num_nodes=2', 'trainer.placement.policy_num_gpus_per_node=4', 'trainer.placement.ref_num_gpus_per_node=4', 'trainer.fully_async.max_staleness_steps=16', 'trainer.fully_async.num_parallel_generation_workers=338', 'generator.backend=vllm', 'generator.timeout_multiplier=1.0', 'generator.model_dtype=bfloat16', 'generator.inference_engine_tensor_parallel_size=1', 'generator.num_inference_engines=48', 'generator.n_samples_per_prompt=8', 'generator.eval_n_samples_per_prompt=8', 'generator.gpu_memory_utilization=0.75', 'generator.max_num_seqs=24', 'generator.max_num_batched_tokens=65536', 'generator.enable_prefix_caching=true', 'generator.enable_chunked_prefill=true', 'generator.run_engines_locally=true', 'generator.weight_sync_backend=nccl', 'generator.async_engine=true', 'generator.batched=false', 'generator.enable_http_endpoint=true', 'generator.enable_ray_prometheus_stats=false', 'generator.vllm_stats_interval=1', 'generator.append_eos_token_after_stop_str_in_multi_turn=true', 'generator.max_turns=999999', 'generator.sampling_params.max_generate_length=4096', 'generator.sampling_params.temperature=0.7', 'generator.sampling_params.top_p=0.95', 'generator.sampling_params.top_k=20', '++generator.engine_init_kwargs.max_model_len=32768', '++generator.engine_init_kwargs.custom_chat_template_chat_completion_path=chat_templates/qwen3_thinking_acc.jinja2', '++generator.engine_init_kwargs.served_model_name=0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', 'data.train_data=["/e/scratch/jureap59/feuer1/tasks/exp_rpt_pymethods2test-large"]', 'data.val_data=[]', '+terminal_bench_config.trials_dir=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/ablation-pymethods2test-seqnorm/trace_jobs', '+terminal_bench_config.harbor.name=terminus-2', '+terminal_bench_config.harbor.max_episodes=999999', '+terminal_bench_config.harbor.enable_summarize=false', '+terminal_bench_config.harbor.store_all_messages=true', '+terminal_bench_config.harbor.trajectory_config.raw_content=true', '+terminal_bench_config.harbor.enable_episode_logging=false', '+terminal_bench_config.harbor.record_terminal_session=false', '+terminal_bench_config.harbor.enable_pane_logging=false', '+terminal_bench_config.harbor.strict_json_parser=true', '+terminal_bench_config.harbor.interleaved_thinking=true', '+terminal_bench_config.harbor.extra_body.chat_template_kwargs.enable_thinking=true', '+terminal_bench_config.harbor.override_timeout_sec=900', '+terminal_bench_config.harbor.override_cpus=1', '+terminal_bench_config.harbor.override_memory_mb=2048', '+terminal_bench_config.harbor.override_storage_mb=2048', '+terminal_bench_config.harbor.auto_snapshot=true', '+terminal_bench_config.harbor.verifier_override_timeout_sec=120', '+terminal_bench_config.harbor.max_retries=3', '+terminal_bench_config.harbor.min_wait_sec=60.0', '+terminal_bench_config.harbor.max_wait_sec=600.0', '+terminal_bench_config.harbor.wait_multiplier=2.0', '+terminal_bench_config.harbor.exclude_exceptions=["VerifierTimeoutError","VerifierRuntimeError","RewardFileNotFoundError","RewardFileEmptyError","VerifierOutputParseError"]', '+terminal_bench_config.harbor.n_concurrent_trials=675', '+terminal_bench_config.harbor.log_level=INFO', '+terminal_bench_config.harbor.enable_reward_shaping=false', '+terminal_bench_config.harbor.enable_error_classification=true', '+terminal_bench_config.harbor.mask_exceptions=["DaytonaError","EnvironmentStartTimeoutError","NetworkError","ConnectionError","RewardFileNotFoundError","RewardFileEmptyError","AgentEnvironmentTimeoutError","ContextLengthExceededError"]', '+terminal_bench_config.harbor.default_error_treatment=zero', '+terminal_bench_config.harbor.passthrough_exceptions=["AgentTimeoutError"]', '+terminal_bench_config.harbor.zero_exceptions=[]', '+terminal_bench_config.model_info.max_input_tokens=32000', '+terminal_bench_config.model_info.max_output_tokens=4096', '+terminal_bench_config.archiving.enabled=false', '+terminal_bench_config.trace_upload.enabled=true', '+terminal_bench_config.trace_upload.repo_org=DCAgent', '+terminal_bench_config.trace_upload.episodes=last', '+terminal_bench_config.trace_upload.dataset_type=SFT', '+terminal_bench_config.trace_upload.cleanup=true']
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/examples/terminal_bench/entrypoints/main_tbench.py", line 142, in <module>
main()
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/main.py", line 94, in decorated_main
_run_hydra(
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/_internal/utils.py", line 394, in _run_hydra
_run_app(
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/_internal/utils.py", line 457, in _run_app
run_and_report(
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/_internal/utils.py", line 223, in run_and_report
raise ex
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/_internal/utils.py", line 220, in run_and_report
return func()
^^^^^^
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/_internal/utils.py", line 458, in <lambda>
lambda: hydra.run(
^^^^^^^^^^
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/_internal/hydra.py", line 132, in run
_ = ret.return_value
^^^^^^^^^^^^^^^^
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/core/utils.py", line 260, in return_value
raise self._return_value
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/core/utils.py", line 186, in run_job
ret.return_value = task_function(task_cfg)
^^^^^^^^^^^^^^^^^^^^^^^
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/examples/terminal_bench/entrypoints/main_tbench.py", line 132, in main
ray.get(skyrl_entrypoint.remote(cfg))
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/ray/_private/auto_init_hook.py", line 22, in auto_init_wrapper
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/ray/_private/client_mode_hook.py", line 104, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/ray/_private/worker.py", line 2967, in get
values, debugger_breakpoint = worker.get_objects(
^^^^^^^^^^^^^^^^^^^
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/ray/_private/worker.py", line 1017, in get_objects
raise value
ray.exceptions.NodeDiedError: Task failed because the node it was running on is dead or unavailable. Node IP: 10.128.32.49, node ID: 4cba01b979f7f003d04abf087b90b4e01890df9fe091283ac00c4962. This can happen if the node was preempted, had a hardware failure, or its raylet crashed unexpectedly. To see node death information, use `ray list nodes --filter node_id=4cba01b979f7f003d04abf087b90b4e01890df9fe091283ac00c4962`, check the Ray dashboard cluster page, search the node ID in the GCS logs, or use `ray logs raylet.out -ip 10.128.32.49`.
Collecting Ray logs from worker jpbo-042-02...
WARNING: Failed to collect logs from jpbo-042-02
Collecting Ray logs from worker jpbo-042-03...
Stopping Ray cluster...
WARNING: Failed to collect logs from jpbo-042-03
Collecting Ray logs from worker jpbo-042-04...
WARNING: Failed to collect logs from jpbo-042-04
Collecting Ray logs from worker jpbo-042-05...
WARNING: Failed to collect logs from jpbo-042-05
Collecting Ray logs from worker jpbo-042-06...
Ray cluster stopped
[RLJobRunner] Crash detected (exit!=0) — preserving Ray logs to /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/ablation-pymethods2test-seqnorm/ray_logs BEFORE trace upload (so a wall-clock kill can't lose crash evidence)...
WARNING: Failed to collect logs from jpbo-042-06
Collecting Ray logs from worker jpbo-042-07...
Collecting Ray logs from worker jpbo-042-02...
WARNING: Failed to collect logs from jpbo-042-07
Collecting Ray logs from worker jpbo-042-08...
WARNING: Failed to collect logs from jpbo-042-02
Collecting Ray logs from worker jpbo-042-03...
WARNING: Failed to collect logs from jpbo-042-03
Collecting Ray logs from worker jpbo-042-04...
WARNING: Failed to collect logs from jpbo-042-04
Collecting Ray logs from worker jpbo-042-05...
WARNING: Failed to collect logs from jpbo-042-08
Collecting Ray logs from worker jpbo-042-09...
WARNING: Failed to collect logs from jpbo-042-05
Collecting Ray logs from worker jpbo-042-06...
WARNING: Failed to collect logs from jpbo-042-06
Collecting Ray logs from worker jpbo-042-07...
WARNING: Failed to collect logs from jpbo-042-07
Collecting Ray logs from worker jpbo-042-08...
WARNING: Failed to collect logs from jpbo-042-08
Collecting Ray logs from worker jpbo-042-09...
WARNING: Failed to collect logs from jpbo-042-09
Collecting Ray logs from worker jpbo-042-10...
WARNING: Failed to collect logs from jpbo-042-09
Collecting Ray logs from worker jpbo-042-10...
WARNING: Failed to collect logs from jpbo-042-10
WARNING: Failed to collect logs from jpbo-042-10
Collecting Ray logs from worker jpbo-042-11...
Collecting Ray logs from worker jpbo-042-11...
WARNING: Failed to collect logs from jpbo-042-11
Collecting Ray logs from worker jpbo-042-12...
WARNING: Failed to collect logs from jpbo-042-11
Collecting Ray logs from worker jpbo-042-12...
WARNING: Failed to collect logs from jpbo-042-12
Collecting Ray logs from worker jpbo-042-13...
WARNING: Failed to collect logs from jpbo-042-12
Collecting Ray logs from worker jpbo-042-13...
WARNING: Failed to collect logs from jpbo-042-13
Collecting Ray logs from worker jpbo-042-14...
WARNING: Failed to collect logs from jpbo-042-13
Collecting Ray logs from worker jpbo-042-14...
WARNING: Failed to collect logs from jpbo-042-14
Ray log preservation complete
WARNING: Failed to collect logs from jpbo-042-14
[RLJobRunner] Crash-time Ray log preservation complete.
[RLJobRunner] Launching trace upload (training exit code: 1):
repo_id: DCAgent/ablation-pymethods2test-seqnorm
job_dir: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/ablation-pymethods2test-seqnorm
episodes: last
log: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/logs/ablation-pymethods2test-seqnorm_trace_upload.log
[RLJobRunner] Waiting for trace upload to complete...