The following have been reloaded with a version change: 1) GCCcore/.14.3.0 => GCCcore/14.3.0 Lmod is automatically replacing "GCC/14.3.0" with "nvidia-compilers/25.9-CUDA-13". Deactivating conda environment: /e/scratch/jureap59/feuer1/miniforge3/envs/otagent Activating RL environment: /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl Python executable: /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/bin/python Python path check: /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/bin/python [ray] RAY_TMPDIR=/tmp/ray/ray_632523 [triton_cache] Triton cache: /tmp/triton_cache_feuer1_632523 [triton_cache] TorchInductor cache: /tmp/torchinductor_cache_feuer1_632523 [proxy] ✓ Found proxychains binary at /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 [proxy] Setting up SSH tunnel to jpbl-s01-01 [proxy] SSH key: /e/home/jusers/feuer1/jupiter/.ssh/authorized_keys/id_ed25519_jsc [proxy] Tunnel port: 7003 [proxy] Node IP: 10.128.16.35 (workers will connect here) [proxy] ✓ SSH tunnel started successfully [proxy] ✓ Generated proxychains config at /e/home/jusers/feuer1/jupiter/.proxychains/proxychains_632523.conf [proxy] - Internal traffic (10.x.x.x, 172.x.x.x, 169.254.x.x) → DIRECT [proxy] - External traffic (internet) → PROXY via tunnel [proxy] ✓ Daytona timeout settings configured [proxy] Testing proxy connectivity... [proxychains] config file found: /e/home/jusers/feuer1/jupiter/.proxychains/proxychains_632523.conf [proxychains] preloading /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/lib/libproxychains4.so [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [proxy] ✓ Proxy connectivity test passed (huggingface.co reachable via wrapped binary) [proxy] ⚠ Tunnel not accessible at 10.128.16.35:7003 (workers may fail) [proxy] ✓ Proxy setup complete (using wrapped binary for Ray workers) [container_runtime] Using cloud backend: daytona (no local container setup) === Universal RL Training Runner === Config: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/configs/ablation-pymethods2test-seqnorm_rl_config.json Working directory: /e/scratch/jureap59/feuer1/OpenThoughts-Agent Python: /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/bin/python Python version: Python 3.12.12 UV_USE_IO_URING: 0 Proxy: DISABLED (direct internet or not configured) ======================================== === RLJobRunner: ablation-pymethods2test-seqnorm === [wandb_utils] Fixing permissions on: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/wandb [wandb_utils] WandB directory ready: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/wandb HF_TOKEN=****pDbg HF_HUB_CACHE=/e/data1/datasets/playground/ot-baf/hf_hub SUPABASE_URL=https://rpzmyuapoqilpghynmza.s... (direct Supabase config) Environment configured: TENSOR_PARALLEL_SIZE=1 NUM_INFERENCE_ENGINES=56 POLICY_NUM_NODES=14 WANDB_DIR=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/wandb Starting Ray cluster with 14 nodes, 4 GPUs/node Cleaning up existing Ray instances... === Starting Ray Cluster === Nodes: 14 GPUs per node: 4 CPUs per node: 288 Head node: jpbo-001-35 (10.128.16.35) Ray port: 6379 ============================ Starting Ray head on jpbo-001-35 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_head_jpbo-001-35.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.35 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-35 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --head --node-ip-address=10.128.16.35 --port=6379 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray head on jpbo-001-35 Starting Ray worker on jpbo-001-36 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-001-36.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.36 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-36 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.36 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 1 on jpbo-001-36 Starting Ray worker on jpbo-001-37 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-001-37.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.37 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-37 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.37 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 2 on jpbo-001-37 Starting Ray worker on jpbo-001-38 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-001-38.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.38 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-38 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.38 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 3 on jpbo-001-38 Starting Ray worker on jpbo-001-41 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-001-41.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.41 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-41 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.41 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 4 on jpbo-001-41 Starting Ray worker on jpbo-001-42 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-001-42.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.42 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-42 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.42 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 5 on jpbo-001-42 Starting Ray worker on jpbo-001-43 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-001-43.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.43 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-43 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.43 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 6 on jpbo-001-43 Starting Ray worker on jpbo-001-44 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-001-44.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.44 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-44 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.44 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 7 on jpbo-001-44 Starting Ray worker on jpbo-001-45 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-001-45.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.45 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-45 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.45 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 8 on jpbo-001-45 Starting Ray worker on jpbo-001-46 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-001-46.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.46 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-46 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.46 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 9 on jpbo-001-46 Starting Ray worker on jpbo-001-47 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-001-47.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.47 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-47 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.47 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 10 on jpbo-001-47 Starting Ray worker on jpbo-001-48 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-001-48.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.48 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-48 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.48 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 11 on jpbo-001-48 Starting Ray worker on jpbo-004-41 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-004-41.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.185 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-004-41 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.185 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 12 on jpbo-004-41 Starting Ray worker on jpbo-004-43 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-004-43.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.187 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-004-43 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.187 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 13 on jpbo-004-43 Waiting for cluster (56 GPUs, 14 nodes)... Connecting to Ray at 10.128.16.35:6379 (expecting 14 nodes, 56.0 GPUs) Ray connection established, polling for resources... [Ray wait] nodes=14/14 GPUs=56.0/56.0 resources={'GPU': 56.0, 'object_store_memory': 601295421440.0, 'memory': 10335684263936.0, 'node:10.128.16.185': 1.0, 'CPU': 4032.0, 'accelerator_type:GH200': 14.0, 'node:10.128.16.187': 1.0, 'node:10.128.16.36': 1.0, 'node:10.128.16.45': 1.0, 'node:10.128.16.35': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.16.37': 1.0, 'node:10.128.16.38': 1.0, 'node:10.128.16.43': 1.0, 'node:10.128.16.44': 1.0, 'node:10.128.16.41': 1.0, 'node:10.128.16.47': 1.0, 'node:10.128.16.46': 1.0, 'node:10.128.16.42': 1.0, 'node:10.128.16.48': 1.0} ✓ Ray cluster ready === Ray Cluster Ready === Address: 10.128.16.35:6379 Total GPUs: 56 ========================= Ray cluster ready at 10.128.16.35:6379 Total GPUs available: 56 [RLJobRunner] Pinggy check: url=False, token=False, needs_tunnel=False (agent=terminus-2, env=daytona) [RLJobRunner] No Pinggy tunnel needed, using local vLLM Running SkyRL: Python: /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/bin/python Entrypoint: examples.terminal_bench.entrypoints.main_tbench Args: 120 Hydra arguments Working dir: /e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train Using proxychains binary: /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 Executing command with srun: /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f $PROXYCHAINS_CONF_FILE /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/bin/python -m examples.terminal_bench.entrypoints.main_tbench +terminal_bench_config=terminal_bench trainer.strategy=fsdp2 trainer.algorithm.advantage_estimator=rloo_n trainer.algorithm.use_kl_loss=false trainer.algorithm.kl_loss_coef=0.0 trainer.algorithm.eps_clip_low=0.2 trainer.algorithm.eps_clip_high=0.05 trainer.algorithm.loss_reduction=seq_mean_token_sum_norm_global trainer.epochs=2 trainer.max_steps=80 trainer.update_epochs_per_batch=1 trainer.train_batch_size=64 trainer.policy_mini_batch_size=64 trainer.eval_batch_size=64 trainer.micro_forward_batch_size_per_gpu=4 trainer.micro_train_batch_size_per_gpu=1 trainer.max_prompt_length=999999 trainer.eval_interval=999999 trainer.eval_before_train=false trainer.ckpt_interval=2 trainer.resume_mode=latest trainer.hf_save_interval=5 ++trainer.hf_hub_repo_id=laion/ablation-pymethods2test-seqnorm ++trainer.hf_hub_private=false ++trainer.hf_hub_revision=main ++trainer.enable_db_registration=false trainer.project_name=OpenThoughts-Agent trainer.log_level=INFO trainer.tracker_commit_each_step=true trainer.logger=console trainer.run_name=ablation-pymethods2test-seqnorm trainer.ckpt_path=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm/checkpoints trainer.export_path=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm/exports trainer.policy.optimizer_config.lr=8e-6 trainer.policy.optimizer_config.weight_decay=0.0 trainer.policy.optimizer_config.adam_betas=[0.9,0.999] trainer.policy.optimizer_config.max_grad_norm=0.9 trainer.policy.fsdp_config.cpu_offload=false trainer.policy.fsdp_config.reshard_after_forward=true trainer.policy.fsdp_config.fsdp_size=4 trainer.policy.model.path=/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6 trainer.ref.fsdp_config.cpu_offload=false trainer.ref.fsdp_config.reshard_after_forward=true trainer.ref.fsdp_config.fsdp_size=4 trainer.placement.colocate_all=false trainer.placement.policy_num_nodes=2 trainer.placement.ref_num_nodes=2 trainer.placement.policy_num_gpus_per_node=4 trainer.placement.ref_num_gpus_per_node=4 trainer.fully_async.max_staleness_steps=16 trainer.fully_async.num_parallel_generation_workers=338 generator.backend=vllm generator.timeout_multiplier=1.0 generator.model_dtype=bfloat16 generator.inference_engine_tensor_parallel_size=1 generator.num_inference_engines=48 generator.n_samples_per_prompt=8 generator.eval_n_samples_per_prompt=8 generator.gpu_memory_utilization=0.75 generator.max_num_seqs=24 generator.max_num_batched_tokens=65536 generator.enable_prefix_caching=true generator.enable_chunked_prefill=true generator.run_engines_locally=true generator.weight_sync_backend=nccl generator.async_engine=true generator.batched=false generator.enable_http_endpoint=true generator.enable_ray_prometheus_stats=false generator.vllm_stats_interval=1 generator.append_eos_token_after_stop_str_in_multi_turn=true generator.max_turns=999999 generator.sampling_params.max_generate_length=4096 generator.sampling_params.temperature=0.7 generator.sampling_params.top_p=0.95 generator.sampling_params.top_k=20 ++generator.engine_init_kwargs.max_model_len=32768 ++generator.engine_init_kwargs.custom_chat_template_chat_completion_path=chat_templates/qwen3_thinking_acc.jinja2 ++generator.engine_init_kwargs.served_model_name=0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6 data.train_data=["/e/scratch/jureap59/feuer1/tasks/exp_rpt_pymethods2test-large"] data.val_data=[] +terminal_bench_config.trials_dir=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm/trace_jobs +terminal_bench_config.harbor.name=terminus-2 +terminal_bench_config.harbor.max_episodes=999999 +terminal_bench_config.harbor.enable_summarize=false +terminal_bench_config.harbor.store_all_messages=true +terminal_bench_config.harbor.trajectory_config.raw_content=true +terminal_bench_config.harbor.enable_episode_logging=false +terminal_bench_config.harbor.record_terminal_session=false +terminal_bench_config.harbor.enable_pane_logging=false +terminal_bench_config.harbor.strict_json_parser=true +terminal_bench_config.harbor.interleaved_thinking=true +terminal_bench_config.harbor.extra_body.chat_template_kwargs.enable_thinking=true +terminal_bench_config.harbor.override_timeout_sec=900 +terminal_bench_config.harbor.override_cpus=1 +terminal_bench_config.harbor.override_memory_mb=2048 +terminal_bench_config.harbor.override_storage_mb=2048 +terminal_bench_config.harbor.auto_snapshot=true +terminal_bench_config.harbor.verifier_override_timeout_sec=120 +terminal_bench_config.harbor.max_retries=3 +terminal_bench_config.harbor.min_wait_sec=60.0 +terminal_bench_config.harbor.max_wait_sec=600.0 +terminal_bench_config.harbor.wait_multiplier=2.0 +terminal_bench_config.harbor.exclude_exceptions=["VerifierTimeoutError","VerifierRuntimeError","RewardFileNotFoundError","RewardFileEmptyError","VerifierOutputParseError"] +terminal_bench_config.harbor.n_concurrent_trials=675 +terminal_bench_config.harbor.log_level=INFO +terminal_bench_config.harbor.enable_reward_shaping=false +terminal_bench_config.harbor.enable_error_classification=true +terminal_bench_config.harbor.mask_exceptions=["DaytonaError","EnvironmentStartTimeoutError","NetworkError","ConnectionError","RewardFileNotFoundError","RewardFileEmptyError","AgentEnvironmentTimeoutError","ContextLengthExceededError"] +terminal_bench_config.harbor.default_error_treatment=zero +terminal_bench_config.harbor.passthrough_exceptions=["AgentTimeoutError"] +terminal_bench_config.harbor.zero_exceptions=[] +terminal_bench_config.model_info.max_input_tokens=32000 +terminal_bench_config.model_info.max_output_tokens=4096 +terminal_bench_config.archiving.enabled=false +terminal_bench_config.trace_upload.enabled=true +terminal_bench_config.trace_upload.repo_org=DCAgent +terminal_bench_config.trace_upload.episodes=last +terminal_bench_config.trace_upload.dataset_type=SFT +terminal_bench_config.trace_upload.cleanup=true [proxychains] config file found: /e/home/jusers/feuer1/jupiter/.proxychains/proxychains_632523.conf [proxychains] preloading /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/lib/libproxychains4.so [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e 2026-06-06 16:20:19.925 | INFO | skyrl_train.utils.utils:prepare_runtime_environment:623 - Exporting wandb api key to ray runtime env 2026-06-06 16:20:19.926 | INFO | skyrl_train.utils.utils:prepare_runtime_environment:642 - Exporting RAY_ADDRESS to ray runtime env 2026-06-06 16:20:19.926 | INFO | skyrl_train.utils.utils:prepare_runtime_environment:667 - Exporting `NCCL_SOCKET_IFNAME` to ray runtime env: ib0 2026-06-06 16:20:19.926 | INFO | skyrl_train.utils.utils:prepare_runtime_environment:667 - Exporting `NCCL_SOCKET_FAMILY` to ray runtime env: AF_INET 2026-06-06 16:20:19.926 | INFO | skyrl_train.utils.utils:prepare_runtime_environment:667 - Exporting `NCCL_DEBUG` to ray runtime env: WARN 2026-06-06 16:20:19,927 INFO worker.py:1680 -- Using address 10.128.16.35:6379 set in the environment variable RAY_ADDRESS 2026-06-06 16:20:19,957 INFO worker.py:1821 -- Connecting to existing Ray cluster at address: 10.128.16.35:6379... 2026-06-06 16:20:19,967 INFO worker.py:2007 -- Connected to Ray cluster. /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (pid=1391421) [2026-06-06 16:20:20,091 E 1391421 1392523] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 (raylet, ip=10.128.16.44) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e 2026-06-06 16:20:22.366 | INFO | skyrl_train.utils.ppo_utils:sync_registries:546 - Synced registries to ray actor (pid=1391454) [2026-06-06 16:20:28,287 E 1391454 1393044] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 [repeated 9x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (raylet) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 17x across cluster] (skyrl_entrypoint pid=1407951) 2026-06-06 16:20:29.474 | INFO  | skyrl_train.entrypoints.main_base:_configure_log_level:211 - SkyRL log level set to: INFO (skyrl_entrypoint pid=1407951) The tokenizer you are loading from '/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. (skyrl_entrypoint pid=1407951) 2026-06-06 16:20:29.745 | INFO  | examples.terminal_bench.dataset:_load_data_files:40 - Loading data from: /e/scratch/jureap59/feuer1/tasks/exp_rpt_pymethods2test-large (skyrl_entrypoint pid=1407951) 2026-06-06 16:20:31.621 | INFO  | examples.terminal_bench.dataset:_load_data_files:50 - Found 5000 valid task directories out of 5000 total directories (skyrl_entrypoint pid=1407951) 2026-06-06 16:20:31.621 | INFO  | examples.terminal_bench.dataset:__init__:27 - TerminalBenchTaskDataset initialized with 5000 task paths (skyrl_entrypoint pid=1407951) 2026-06-06 16:20:31.632 | INFO  | skyrl_train.entrypoints.main_base:_setup_trainer:394 - data: (skyrl_entrypoint pid=1407951) train_data: (skyrl_entrypoint pid=1407951) - /e/scratch/jureap59/feuer1/tasks/exp_rpt_pymethods2test-large (skyrl_entrypoint pid=1407951) val_data: [] (skyrl_entrypoint pid=1407951) trainer: (skyrl_entrypoint pid=1407951) placement: (skyrl_entrypoint pid=1407951) colocate_all: false (skyrl_entrypoint pid=1407951) colocate_policy_ref: true (skyrl_entrypoint pid=1407951) policy_num_nodes: 2 (skyrl_entrypoint pid=1407951) policy_num_gpus_per_node: 4 (skyrl_entrypoint pid=1407951) critic_num_nodes: 1 (skyrl_entrypoint pid=1407951) critic_num_gpus_per_node: 4 (skyrl_entrypoint pid=1407951) ref_num_nodes: 2 (skyrl_entrypoint pid=1407951) ref_num_gpus_per_node: 4 (skyrl_entrypoint pid=1407951) policy_strict_spread_pg: false (skyrl_entrypoint pid=1407951) sequence_parallel_backend: ulysses (skyrl_entrypoint pid=1407951) strategy: fsdp2 (skyrl_entrypoint pid=1407951) policy: (skyrl_entrypoint pid=1407951) model: (skyrl_entrypoint pid=1407951) path: /e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6 (skyrl_entrypoint pid=1407951) lora: (skyrl_entrypoint pid=1407951) rank: 0 (skyrl_entrypoint pid=1407951) alpha: 16 (skyrl_entrypoint pid=1407951) dropout: 0 (skyrl_entrypoint pid=1407951) lora_sync_path: /tmp/skyrl_lora_sync (skyrl_entrypoint pid=1407951) target_modules: all-linear (skyrl_entrypoint pid=1407951) exclude_modules: null (skyrl_entrypoint pid=1407951) deepspeed_config: ${deepspeed_config.train} (skyrl_entrypoint pid=1407951) optimizer_config: (skyrl_entrypoint pid=1407951) optimizer: AdamW (skyrl_entrypoint pid=1407951) lr: 8.0e-06 (skyrl_entrypoint pid=1407951) adam_betas: (skyrl_entrypoint pid=1407951) - 0.9 (skyrl_entrypoint pid=1407951) - 0.999 (skyrl_entrypoint pid=1407951) weight_decay: 0.0 (skyrl_entrypoint pid=1407951) max_grad_norm: 0.9 (skyrl_entrypoint pid=1407951) offload_after_step: true (skyrl_entrypoint pid=1407951) num_warmup_steps: 0 (skyrl_entrypoint pid=1407951) scheduler: constant_with_warmup (skyrl_entrypoint pid=1407951) optimizer_kwargs: {} (skyrl_entrypoint pid=1407951) fsdp_config: (skyrl_entrypoint pid=1407951) cpu_offload: false (skyrl_entrypoint pid=1407951) reshard_after_forward: true (skyrl_entrypoint pid=1407951) fsdp_size: 4 (skyrl_entrypoint pid=1407951) expert_model_parallel_size: 1 (skyrl_entrypoint pid=1407951) expert_tensor_parallel_size: 1 (skyrl_entrypoint pid=1407951) moe_token_dispatcher_type: alltoall (skyrl_entrypoint pid=1407951) moe_router_replay: false (skyrl_entrypoint pid=1407951) moe_grouped_gemm: false (skyrl_entrypoint pid=1407951) ep_comm_backend: torch (skyrl_entrypoint pid=1407951) deepep_num_sms: 20 (skyrl_entrypoint pid=1407951) deepep_token_chunk_size: null (skyrl_entrypoint pid=1407951) sequence_parallel_size: 1 (skyrl_entrypoint pid=1407951) use_torch_compile: false (skyrl_entrypoint pid=1407951) record_memory: false (skyrl_entrypoint pid=1407951) megatron_config: (skyrl_entrypoint pid=1407951) tensor_model_parallel_size: 1 (skyrl_entrypoint pid=1407951) pipeline_model_parallel_size: 1 (skyrl_entrypoint pid=1407951) context_parallel_size: 1 (skyrl_entrypoint pid=1407951) expert_model_parallel_size: 1 (skyrl_entrypoint pid=1407951) expert_tensor_parallel_size: null (skyrl_entrypoint pid=1407951) ddp_config: (skyrl_entrypoint pid=1407951) grad_reduce_in_fp32: true (skyrl_entrypoint pid=1407951) overlap_grad_reduce: false (skyrl_entrypoint pid=1407951) overlap_param_gather: false (skyrl_entrypoint pid=1407951) average_in_collective: true (skyrl_entrypoint pid=1407951) model_config_kwargs: {} (skyrl_entrypoint pid=1407951) torch_profiler_config: (skyrl_entrypoint pid=1407951) enable: false (skyrl_entrypoint pid=1407951) ranks: [] (skyrl_entrypoint pid=1407951) save_path: null (skyrl_entrypoint pid=1407951) optimizer_config_kwargs: (skyrl_entrypoint pid=1407951) overlap_cpu_optimizer_d2h_h2d: false (skyrl_entrypoint pid=1407951) use_precision_aware_optimizer: false (skyrl_entrypoint pid=1407951) optimizer_cpu_offload: false (skyrl_entrypoint pid=1407951) optimizer_offload_fraction: 0.0 (skyrl_entrypoint pid=1407951) transformer_config_kwargs: (skyrl_entrypoint pid=1407951) recompute_granularity: full (skyrl_entrypoint pid=1407951) recompute_modules: (skyrl_entrypoint pid=1407951) - core_attn (skyrl_entrypoint pid=1407951) recompute_method: uniform (skyrl_entrypoint pid=1407951) recompute_num_layers: 1 (skyrl_entrypoint pid=1407951) empty_cuda_cache: true (skyrl_entrypoint pid=1407951) ref: (skyrl_entrypoint pid=1407951) model: (skyrl_entrypoint pid=1407951) path: ${trainer.policy.model.path} (skyrl_entrypoint pid=1407951) sequence_parallel_size: 1 (skyrl_entrypoint pid=1407951) deepspeed_config: ${deepspeed_config.eval} (skyrl_entrypoint pid=1407951) fsdp_config: (skyrl_entrypoint pid=1407951) cpu_offload: false (skyrl_entrypoint pid=1407951) reshard_after_forward: true (skyrl_entrypoint pid=1407951) fsdp_size: 4 (skyrl_entrypoint pid=1407951) expert_model_parallel_size: 1 (skyrl_entrypoint pid=1407951) expert_tensor_parallel_size: 1 (skyrl_entrypoint pid=1407951) moe_token_dispatcher_type: alltoall (skyrl_entrypoint pid=1407951) moe_router_replay: false (skyrl_entrypoint pid=1407951) moe_grouped_gemm: false (skyrl_entrypoint pid=1407951) ep_comm_backend: torch (skyrl_entrypoint pid=1407951) deepep_num_sms: 20 (skyrl_entrypoint pid=1407951) deepep_token_chunk_size: null (skyrl_entrypoint pid=1407951) megatron_config: (skyrl_entrypoint pid=1407951) tensor_model_parallel_size: 1 (skyrl_entrypoint pid=1407951) pipeline_model_parallel_size: 1 (skyrl_entrypoint pid=1407951) context_parallel_size: 1 (skyrl_entrypoint pid=1407951) expert_model_parallel_size: 1 (skyrl_entrypoint pid=1407951) expert_tensor_parallel_size: 1 (skyrl_entrypoint pid=1407951) model_config_kwargs: {} (skyrl_entrypoint pid=1407951) transformer_config_kwargs: {} (skyrl_entrypoint pid=1407951) critic: (skyrl_entrypoint pid=1407951) model: (skyrl_entrypoint pid=1407951) path: null (skyrl_entrypoint pid=1407951) lora: (skyrl_entrypoint pid=1407951) rank: 0 (skyrl_entrypoint pid=1407951) alpha: 16 (skyrl_entrypoint pid=1407951) dropout: 0 (skyrl_entrypoint pid=1407951) target_modules: all-linear (skyrl_entrypoint pid=1407951) exclude_modules: null (skyrl_entrypoint pid=1407951) deepspeed_config: ${deepspeed_config.train} (skyrl_entrypoint pid=1407951) optimizer_config: (skyrl_entrypoint pid=1407951) optimizer: AdamW (skyrl_entrypoint pid=1407951) lr: 5.0e-06 (skyrl_entrypoint pid=1407951) adam_betas: (skyrl_entrypoint pid=1407951) - 0.9 (skyrl_entrypoint pid=1407951) - 0.999 (skyrl_entrypoint pid=1407951) weight_decay: 0.01 (skyrl_entrypoint pid=1407951) max_grad_norm: 1.0 (skyrl_entrypoint pid=1407951) offload_after_step: true (skyrl_entrypoint pid=1407951) num_warmup_steps: 0 (skyrl_entrypoint pid=1407951) scheduler: constant_with_warmup (skyrl_entrypoint pid=1407951) optimizer_kwargs: {} (skyrl_entrypoint pid=1407951) fsdp_config: (skyrl_entrypoint pid=1407951) cpu_offload: false (skyrl_entrypoint pid=1407951) reshard_after_forward: true (skyrl_entrypoint pid=1407951) fsdp_size: -1 (skyrl_entrypoint pid=1407951) expert_model_parallel_size: 1 (skyrl_entrypoint pid=1407951) expert_tensor_parallel_size: 1 (skyrl_entrypoint pid=1407951) moe_token_dispatcher_type: alltoall (skyrl_entrypoint pid=1407951) moe_router_replay: false (skyrl_entrypoint pid=1407951) moe_grouped_gemm: false (skyrl_entrypoint pid=1407951) ep_comm_backend: torch (skyrl_entrypoint pid=1407951) deepep_num_sms: 20 (skyrl_entrypoint pid=1407951) deepep_token_chunk_size: null (skyrl_entrypoint pid=1407951) sequence_parallel_size: 1 (skyrl_entrypoint pid=1407951) algorithm: (skyrl_entrypoint pid=1407951) advantage_estimator: rloo_n (skyrl_entrypoint pid=1407951) kl_ctrl: (skyrl_entrypoint pid=1407951) type: fixed (skyrl_entrypoint pid=1407951) kl_target: 0.1 (skyrl_entrypoint pid=1407951) horizon: 10000 (skyrl_entrypoint pid=1407951) kl_estimator_type: k3 (skyrl_entrypoint pid=1407951) use_kl_estimator_k3: false (skyrl_entrypoint pid=1407951) use_abs_kl: false (skyrl_entrypoint pid=1407951) use_kl_in_reward: false (skyrl_entrypoint pid=1407951) use_kl_loss: false (skyrl_entrypoint pid=1407951) kl_loss_coef: 0.0 (skyrl_entrypoint pid=1407951) use_entropy_loss: false (skyrl_entrypoint pid=1407951) entropy_loss_coef: 0.01 (skyrl_entrypoint pid=1407951) advantage_batch_normalize: false (skyrl_entrypoint pid=1407951) value_head_prefix: value_head (skyrl_entrypoint pid=1407951) policy_loss_type: regular (skyrl_entrypoint pid=1407951) loss_reduction: seq_mean_token_sum_norm_global (skyrl_entrypoint pid=1407951) grpo_norm_by_std: true (skyrl_entrypoint pid=1407951) rloo_n_min_group_size: 4 (skyrl_entrypoint pid=1407951) rloo_n_filter_zero_reward_groups: true (skyrl_entrypoint pid=1407951) lambd: 1.0 (skyrl_entrypoint pid=1407951) gamma: 1.0 (skyrl_entrypoint pid=1407951) eps_clip_low: 0.2 (skyrl_entrypoint pid=1407951) eps_clip_high: 0.05 (skyrl_entrypoint pid=1407951) clip_ratio_c: 3.0 (skyrl_entrypoint pid=1407951) tis_imp_ratio_cap: -1.0 (skyrl_entrypoint pid=1407951) use_tis: false (skyrl_entrypoint pid=1407951) sapo: (skyrl_entrypoint pid=1407951) tau_pos: 1.0 (skyrl_entrypoint pid=1407951) tau_neg: 1.05 (skyrl_entrypoint pid=1407951) value_clip: 0.2 (skyrl_entrypoint pid=1407951) dynamic_sampling: (skyrl_entrypoint pid=1407951) type: null (skyrl_entrypoint pid=1407951) max_sample_batches: 30 (skyrl_entrypoint pid=1407951) min_replace_ratio: 0.3 (skyrl_entrypoint pid=1407951) clip_cov: (skyrl_entrypoint pid=1407951) clip_ratio: 0.0002 (skyrl_entrypoint pid=1407951) clip_cov_lb: 1.0 (skyrl_entrypoint pid=1407951) clip_cov_ub: 5.0 (skyrl_entrypoint pid=1407951) kl_cov: (skyrl_entrypoint pid=1407951) kl_cov_frac: 0.2 (skyrl_entrypoint pid=1407951) ppo_kl_coef: 1.0 (skyrl_entrypoint pid=1407951) cispo: (skyrl_entrypoint pid=1407951) cispo_eps_clip_low: 0 (skyrl_entrypoint pid=1407951) cispo_eps_clip_high: 5 (skyrl_entrypoint pid=1407951) z_clip: (skyrl_entrypoint pid=1407951) enabled: false (skyrl_entrypoint pid=1407951) alpha: 0.97 (skyrl_entrypoint pid=1407951) z_thresh: 2.5 (skyrl_entrypoint pid=1407951) warmup_steps: 25 (skyrl_entrypoint pid=1407951) mode: zscore (skyrl_entrypoint pid=1407951) clip_option: adaptive_scaling (skyrl_entrypoint pid=1407951) clip_factor: 1.0 (skyrl_entrypoint pid=1407951) skip_update_on_spike: false (skyrl_entrypoint pid=1407951) stale_clip: (skyrl_entrypoint pid=1407951) enabled: false (skyrl_entrypoint pid=1407951) alpha: 0.3 (skyrl_entrypoint pid=1407951) entropy_threshold: 0.15 (skyrl_entrypoint pid=1407951) entropy_window: 10 (skyrl_entrypoint pid=1407951) min_lr_scale: 0.1 (skyrl_entrypoint pid=1407951) max_seq_len: 1004095 (skyrl_entrypoint pid=1407951) fully_async: (skyrl_entrypoint pid=1407951) max_staleness_steps: 16 (skyrl_entrypoint pid=1407951) num_parallel_generation_workers: 338 (skyrl_entrypoint pid=1407951) gradient_checkpointing: true (skyrl_entrypoint pid=1407951) gradient_checkpointing_use_reentrant: false (skyrl_entrypoint pid=1407951) seed: 42 (skyrl_entrypoint pid=1407951) resume_mode: latest (skyrl_entrypoint pid=1407951) resume_path: null (skyrl_entrypoint pid=1407951) ckpt_path: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm/checkpoints (skyrl_entrypoint pid=1407951) max_ckpts_to_keep: -1 (skyrl_entrypoint pid=1407951) ckpt_interval: 2 (skyrl_entrypoint pid=1407951) hf_save_interval: 5 (skyrl_entrypoint pid=1407951) hf_upload_mode: latest (skyrl_entrypoint pid=1407951) export_path: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm/exports (skyrl_entrypoint pid=1407951) bf16: true (skyrl_entrypoint pid=1407951) epochs: 2 (skyrl_entrypoint pid=1407951) max_steps: 80 (skyrl_entrypoint pid=1407951) update_epochs_per_batch: 1 (skyrl_entrypoint pid=1407951) train_batch_size: 64 (skyrl_entrypoint pid=1407951) policy_mini_batch_size: 64 (skyrl_entrypoint pid=1407951) critic_mini_batch_size: 256 (skyrl_entrypoint pid=1407951) micro_train_batch_size_per_gpu: 1 (skyrl_entrypoint pid=1407951) micro_forward_batch_size_per_gpu: 4 (skyrl_entrypoint pid=1407951) update_ref_every_epoch: false (skyrl_entrypoint pid=1407951) use_sample_packing: true (skyrl_entrypoint pid=1407951) eval_batch_size: 64 (skyrl_entrypoint pid=1407951) eval_before_train: false (skyrl_entrypoint pid=1407951) eval_interval: 999999 (skyrl_entrypoint pid=1407951) max_prompt_length: 999999 (skyrl_entrypoint pid=1407951) flash_attn: true (skyrl_entrypoint pid=1407951) disable_fast_tokenizer: false (skyrl_entrypoint pid=1407951) target_modules: null (skyrl_entrypoint pid=1407951) exclude_modules: null (skyrl_entrypoint pid=1407951) project_name: OpenThoughts-Agent (skyrl_entrypoint pid=1407951) run_name: ablation-pymethods2test-seqnorm (skyrl_entrypoint pid=1407951) logger: console (skyrl_entrypoint pid=1407951) tracker_commit_each_step: true (skyrl_entrypoint pid=1407951) dump_data_batch: false (skyrl_entrypoint pid=1407951) dump_eval_results: true (skyrl_entrypoint pid=1407951) log_level: INFO (skyrl_entrypoint pid=1407951) rope_scaling: null (skyrl_entrypoint pid=1407951) rope_theta: null (skyrl_entrypoint pid=1407951) step_wise_training: false (skyrl_entrypoint pid=1407951) hf_hub_repo_id: laion/ablation-pymethods2test-seqnorm (skyrl_entrypoint pid=1407951) hf_hub_private: false (skyrl_entrypoint pid=1407951) hf_hub_revision: main (skyrl_entrypoint pid=1407951) enable_db_registration: false (skyrl_entrypoint pid=1407951) generator: (skyrl_entrypoint pid=1407951) model_name: ${trainer.policy.model.path} (skyrl_entrypoint pid=1407951) model_dtype: bfloat16 (skyrl_entrypoint pid=1407951) timeout_multiplier: 1.0 (skyrl_entrypoint pid=1407951) run_engines_locally: true (skyrl_entrypoint pid=1407951) num_inference_engines: 48 (skyrl_entrypoint pid=1407951) backend: vllm (skyrl_entrypoint pid=1407951) weight_sync_backend: nccl (skyrl_entrypoint pid=1407951) fuse_weights: false (skyrl_entrypoint pid=1407951) weight_transfer_threshold_cuda_ipc_GB: 1.0 (skyrl_entrypoint pid=1407951) inference_engine_tensor_parallel_size: 1 (skyrl_entrypoint pid=1407951) inference_engine_pipeline_parallel_size: 1 (skyrl_entrypoint pid=1407951) inference_engine_expert_parallel_size: 1 (skyrl_entrypoint pid=1407951) inference_engine_data_parallel_size: 1 (skyrl_entrypoint pid=1407951) n_samples_per_prompt: 8 (skyrl_entrypoint pid=1407951) async_engine: true (skyrl_entrypoint pid=1407951) batched: false (skyrl_entrypoint pid=1407951) max_input_length: ${trainer.max_prompt_length} (skyrl_entrypoint pid=1407951) vllm_v1_disable_multiproc: true (skyrl_entrypoint pid=1407951) enable_prefix_caching: true (skyrl_entrypoint pid=1407951) enable_chunked_prefill: true (skyrl_entrypoint pid=1407951) max_num_batched_tokens: 65536 (skyrl_entrypoint pid=1407951) enforce_eager: true (skyrl_entrypoint pid=1407951) fully_sharded_loras: false (skyrl_entrypoint pid=1407951) enable_ray_prometheus_stats: false (skyrl_entrypoint pid=1407951) vllm_stats_interval: 1 (skyrl_entrypoint pid=1407951) gpu_memory_utilization: 0.75 (skyrl_entrypoint pid=1407951) max_num_seqs: 24 (skyrl_entrypoint pid=1407951) remote_inference_engine_urls: (skyrl_entrypoint pid=1407951) - 127.0.0.1:8001 (skyrl_entrypoint pid=1407951) enable_http_endpoint: true (skyrl_entrypoint pid=1407951) http_endpoint_host: 127.0.0.1 (skyrl_entrypoint pid=1407951) http_endpoint_port: 8000 (skyrl_entrypoint pid=1407951) max_turns: 999999 (skyrl_entrypoint pid=1407951) chat_template: (skyrl_entrypoint pid=1407951) source: name (skyrl_entrypoint pid=1407951) name_or_path: null (skyrl_entrypoint pid=1407951) chat_template_kwargs: {} (skyrl_entrypoint pid=1407951) engine_init_kwargs: (skyrl_entrypoint pid=1407951) max_model_len: 32768 (skyrl_entrypoint pid=1407951) custom_chat_template_chat_completion_path: chat_templates/qwen3_thinking_acc.jinja2 (skyrl_entrypoint pid=1407951) served_model_name: 0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6 (skyrl_entrypoint pid=1407951) override_existing_update_group: disable (skyrl_entrypoint pid=1407951) sampling_params: (skyrl_entrypoint pid=1407951) max_generate_length: 4096 (skyrl_entrypoint pid=1407951) repetition_penalty: 1.0 (skyrl_entrypoint pid=1407951) temperature: 0.7 (skyrl_entrypoint pid=1407951) top_p: 0.95 (skyrl_entrypoint pid=1407951) min_p: 0.0 (skyrl_entrypoint pid=1407951) top_k: 20 (skyrl_entrypoint pid=1407951) logprobs: null (skyrl_entrypoint pid=1407951) stop: null (skyrl_entrypoint pid=1407951) use_conversation_multi_turn: true (skyrl_entrypoint pid=1407951) append_eos_token_after_stop_str_in_multi_turn: true (skyrl_entrypoint pid=1407951) eval_sampling_params: (skyrl_entrypoint pid=1407951) max_generate_length: ${generator.sampling_params.max_generate_length} (skyrl_entrypoint pid=1407951) repetition_penalty: 1.0 (skyrl_entrypoint pid=1407951) temperature: 0.0 (skyrl_entrypoint pid=1407951) top_p: 1.0 (skyrl_entrypoint pid=1407951) min_p: 0.0 (skyrl_entrypoint pid=1407951) top_k: -1 (skyrl_entrypoint pid=1407951) logprobs: null (skyrl_entrypoint pid=1407951) stop: null (skyrl_entrypoint pid=1407951) eval_n_samples_per_prompt: 8 (skyrl_entrypoint pid=1407951) zero_reward_on_non_stop: false (skyrl_entrypoint pid=1407951) apply_overlong_filtering: false (skyrl_entrypoint pid=1407951) rope_scaling: ${trainer.rope_scaling} (skyrl_entrypoint pid=1407951) rope_theta: ${trainer.rope_theta} (skyrl_entrypoint pid=1407951) teacher: (skyrl_entrypoint pid=1407951) model_path: null (skyrl_entrypoint pid=1407951) top_k_logprobs: 256 (skyrl_entrypoint pid=1407951) num_inference_engines: 1 (skyrl_entrypoint pid=1407951) inference_engine_tensor_parallel_size: 1 (skyrl_entrypoint pid=1407951) inference_engine_pipeline_parallel_size: 1 (skyrl_entrypoint pid=1407951) gpu_memory_utilization: 0.9 (skyrl_entrypoint pid=1407951) enforce_eager: false (skyrl_entrypoint pid=1407951) backend: vllm (skyrl_entrypoint pid=1407951) engine_init_kwargs: {} (skyrl_entrypoint pid=1407951) environment: (skyrl_entrypoint pid=1407951) env_class: gsm8k (skyrl_entrypoint pid=1407951) skyrl_gym: (skyrl_entrypoint pid=1407951) max_env_workers: 32 (skyrl_entrypoint pid=1407951) text2sql: (skyrl_entrypoint pid=1407951) db_path: /home/ray/default/sql_data (skyrl_entrypoint pid=1407951) llm_as_a_judge: (skyrl_entrypoint pid=1407951) model: gpt-4o-mini (skyrl_entrypoint pid=1407951) base_url: null (skyrl_entrypoint pid=1407951) search: (skyrl_entrypoint pid=1407951) log_requests: false (skyrl_entrypoint pid=1407951) search_url: http://127.0.0.1:8000/retrieve (skyrl_entrypoint pid=1407951) topk: 3 (skyrl_entrypoint pid=1407951) timeout: 30 (skyrl_entrypoint pid=1407951) rollout: (skyrl_entrypoint pid=1407951) fanout: (skyrl_entrypoint pid=1407951) enabled: true (skyrl_entrypoint pid=1407951) num_coordinators: 4 (skyrl_entrypoint pid=1407951) cpus_per_coordinator: 8 (skyrl_entrypoint pid=1407951) deepspeed_config: (skyrl_entrypoint pid=1407951) train: (skyrl_entrypoint pid=1407951) zero_optimization: (skyrl_entrypoint pid=1407951) stage: 3 (skyrl_entrypoint pid=1407951) offload_param: (skyrl_entrypoint pid=1407951) device: none (skyrl_entrypoint pid=1407951) offload_optimizer: (skyrl_entrypoint pid=1407951) device: none (skyrl_entrypoint pid=1407951) pin_memory: true (skyrl_entrypoint pid=1407951) sub_group_size: auto (skyrl_entrypoint pid=1407951) reduce_bucket_size: auto (skyrl_entrypoint pid=1407951) stage3_param_persistence_threshold: auto (skyrl_entrypoint pid=1407951) stage3_prefetch_bucket_size: auto (skyrl_entrypoint pid=1407951) stage3_max_live_parameters: auto (skyrl_entrypoint pid=1407951) stage3_max_reuse_distance: auto (skyrl_entrypoint pid=1407951) round_robin_gradients: true (skyrl_entrypoint pid=1407951) zero_hpz_partition_size: 1 (skyrl_entrypoint pid=1407951) zero_quantized_weights: false (skyrl_entrypoint pid=1407951) zero_quantized_gradients: false (skyrl_entrypoint pid=1407951) torch_autocast: (skyrl_entrypoint pid=1407951) enabled: true (skyrl_entrypoint pid=1407951) dtype: bfloat16 (skyrl_entrypoint pid=1407951) disable_trace_cache: false (skyrl_entrypoint pid=1407951) data_types: (skyrl_entrypoint pid=1407951) grad_accum_dtype: fp32 (skyrl_entrypoint pid=1407951) gradient_clipping: 1.0 (skyrl_entrypoint pid=1407951) wall_clock_breakdown: false (skyrl_entrypoint pid=1407951) prescale_gradient: false (skyrl_entrypoint pid=1407951) eval: (skyrl_entrypoint pid=1407951) zero_optimization: (skyrl_entrypoint pid=1407951) stage: 3 (skyrl_entrypoint pid=1407951) stage3_param_persistence_threshold: auto (skyrl_entrypoint pid=1407951) offload_param: (skyrl_entrypoint pid=1407951) device: cpu (skyrl_entrypoint pid=1407951) pin_memory: true (skyrl_entrypoint pid=1407951) torch_autocast: (skyrl_entrypoint pid=1407951) enabled: true (skyrl_entrypoint pid=1407951) dtype: bfloat16 (skyrl_entrypoint pid=1407951) gradient_clipping: 1.0 (skyrl_entrypoint pid=1407951) prescale_gradient: false (skyrl_entrypoint pid=1407951) wall_clock_breakdown: false (skyrl_entrypoint pid=1407951) terminal_bench_config: (skyrl_entrypoint pid=1407951) trials_dir: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm/trace_jobs (skyrl_entrypoint pid=1407951) harbor: (skyrl_entrypoint pid=1407951) name: terminus-2 (skyrl_entrypoint pid=1407951) max_episodes: 999999 (skyrl_entrypoint pid=1407951) enable_summarize: false (skyrl_entrypoint pid=1407951) store_all_messages: true (skyrl_entrypoint pid=1407951) trajectory_config: (skyrl_entrypoint pid=1407951) raw_content: true (skyrl_entrypoint pid=1407951) enable_episode_logging: false (skyrl_entrypoint pid=1407951) record_terminal_session: false (skyrl_entrypoint pid=1407951) enable_pane_logging: false (skyrl_entrypoint pid=1407951) strict_json_parser: true (skyrl_entrypoint pid=1407951) interleaved_thinking: true (skyrl_entrypoint pid=1407951) extra_body: (skyrl_entrypoint pid=1407951) chat_template_kwargs: (skyrl_entrypoint pid=1407951) enable_thinking: true (skyrl_entrypoint pid=1407951) override_timeout_sec: 900 (skyrl_entrypoint pid=1407951) override_cpus: 1 (skyrl_entrypoint pid=1407951) override_memory_mb: 2048 (skyrl_entrypoint pid=1407951) override_storage_mb: 2048 (skyrl_entrypoint pid=1407951) auto_snapshot: true (skyrl_entrypoint pid=1407951) verifier_override_timeout_sec: 120 (skyrl_entrypoint pid=1407951) max_retries: 3 (skyrl_entrypoint pid=1407951) min_wait_sec: 60.0 (skyrl_entrypoint pid=1407951) max_wait_sec: 600.0 (skyrl_entrypoint pid=1407951) wait_multiplier: 2.0 (skyrl_entrypoint pid=1407951) exclude_exceptions: (skyrl_entrypoint pid=1407951) - VerifierTimeoutError (skyrl_entrypoint pid=1407951) - VerifierRuntimeError (skyrl_entrypoint pid=1407951) - RewardFileNotFoundError (skyrl_entrypoint pid=1407951) - RewardFileEmptyError (skyrl_entrypoint pid=1407951) - VerifierOutputParseError (skyrl_entrypoint pid=1407951) n_concurrent_trials: 675 (skyrl_entrypoint pid=1407951) log_level: INFO (skyrl_entrypoint pid=1407951) enable_reward_shaping: false (skyrl_entrypoint pid=1407951) enable_error_classification: true (skyrl_entrypoint pid=1407951) mask_exceptions: (skyrl_entrypoint pid=1407951) - DaytonaError (skyrl_entrypoint pid=1407951) - EnvironmentStartTimeoutError (skyrl_entrypoint pid=1407951) - NetworkError (skyrl_entrypoint pid=1407951) - ConnectionError (skyrl_entrypoint pid=1407951) - RewardFileNotFoundError (skyrl_entrypoint pid=1407951) - RewardFileEmptyError (skyrl_entrypoint pid=1407951) - AgentEnvironmentTimeoutError (skyrl_entrypoint pid=1407951) - ContextLengthExceededError (skyrl_entrypoint pid=1407951) default_error_treatment: zero (skyrl_entrypoint pid=1407951) passthrough_exceptions: (skyrl_entrypoint pid=1407951) - AgentTimeoutError (skyrl_entrypoint pid=1407951) zero_exceptions: [] (skyrl_entrypoint pid=1407951) model_info: (skyrl_entrypoint pid=1407951) max_input_tokens: 32000 (skyrl_entrypoint pid=1407951) max_output_tokens: 4096 (skyrl_entrypoint pid=1407951) archiving: (skyrl_entrypoint pid=1407951) enabled: false (skyrl_entrypoint pid=1407951) trace_upload: (skyrl_entrypoint pid=1407951) enabled: true (skyrl_entrypoint pid=1407951) repo_org: DCAgent (skyrl_entrypoint pid=1407951) episodes: last (skyrl_entrypoint pid=1407951) dataset_type: SFT (skyrl_entrypoint pid=1407951) cleanup: true (skyrl_entrypoint pid=1407951)  (skyrl_entrypoint pid=1407951) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: (skyrl_entrypoint pid=1407951) No module named 'vllm._version' (skyrl_entrypoint pid=1407951) from .version import __version__, __version_tuple__ # isort:skip (skyrl_entrypoint pid=1407951) W0606 16:20:35.985000 1407951 envs/rl/lib/python3.12/site-packages/torch/utils/cpp_extension.py:117] No CUDA runtime is found, using CUDA_HOME='/e/software/default/stages/2026/software/CUDA/13' (pid=1391705) [2026-06-06 16:20:31,962 E 1391705 1406826] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 [repeated 268x across cluster] (raylet) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e (pid=1088873, ip=10.128.16.43) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: (pid=1088873, ip=10.128.16.43) No module named 'vllm._version' (pid=1088873, ip=10.128.16.43) from .version import __version__, __version_tuple__ # isort:skip (raylet, ip=10.128.16.48) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 41x across cluster] (pid=1378319, ip=10.128.16.37) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: [repeated 4x across cluster] (pid=1378319, ip=10.128.16.37) No module named 'vllm._version' [repeated 4x across cluster] (pid=1378319, ip=10.128.16.37) from .version import __version__, __version_tuple__ # isort:skip [repeated 4x across cluster] (raylet, ip=10.128.16.44) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 111x across cluster] [2026-06-06 16:20:50,151 E 1407598 1407925] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 (RegistryActor pid=1379010, ip=10.128.16.44) [2026-06-06 16:20:51,058 E 1379010 1379050] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) 2026-06-06 16:20:51.613 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:123 - setup_envvars_for_vllm: distributed_executor_backend=uni, SKYRL_ENABLE_NUMA_AFFINITY=1, CUDA_VISIBLE_DEVICES=0, VLLM_ENABLE_V1_MULTIPROCESSING= (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) 2026-06-06 16:20:51.614 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:133 - setup_envvars_for_vllm: numa_enabled=True (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) 2026-06-06 16:20:51.614 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:136 - setup_envvars_for_vllm: set VLLM_ENABLE_V1_MULTIPROCESSING=0 for NUMA affinity (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) 2026-06-06 16:20:52.242 | INFO | skyrl_train.utils.numa:set_numa_affinity_for_gpu:340 - NUMA affinity: bound process to CPUs 0-71 (NUMA node 0) for GPU 0 (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) 2026-06-06 16:20:52.266 | DEBUG | skyrl_train.utils.numa:_set_membind_via_libnuma:375 - NUMA affinity: set memory preferred to NUMA node 0 (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) 2026-06-06 16:20:52.266 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:584 - BaseVLLMInferenceEngine: vllm_v1_disable_multiproc=True, vllm.__version__=dev, VLLM_ENABLE_V1_MULTIPROCESSING=0 (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) 2026-06-06 16:20:52.266 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:592 - BaseVLLMInferenceEngine: set VLLM_ENABLE_V1_MULTIPROCESSING=0 (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) 2026-06-06 16:20:52.266 | WARNING | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1150 - OpenAI API sampling params overridden: temperature=0.7, top_p=0.95, top_k=20 (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) 2026-06-06 16:20:52.285 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1202 - Engine startup stagger: sleeping 1.56s (attempt 1/5) to avoid port collisions (pid=1943797, ip=10.128.16.187) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: [repeated 20x across cluster] (pid=1943797, ip=10.128.16.187) No module named 'vllm._version' [repeated 20x across cluster] (pid=1943797, ip=10.128.16.187) from .version import __version__, __version_tuple__ # isort:skip [repeated 20x across cluster] (raylet, ip=10.128.16.45) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 81x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) The tokenizer you are loading from '/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. (skyrl_entrypoint pid=1407951) [2026-06-06 16:20:53,496 E 1407951 1407994] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) 2026-06-06 16:20:56.763 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:123 - setup_envvars_for_vllm: distributed_executor_backend=uni, SKYRL_ENABLE_NUMA_AFFINITY=1, CUDA_VISIBLE_DEVICES=1, VLLM_ENABLE_V1_MULTIPROCESSING= [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) 2026-06-06 16:20:56.764 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:133 - setup_envvars_for_vllm: numa_enabled=True [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) 2026-06-06 16:20:56.764 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:136 - setup_envvars_for_vllm: set VLLM_ENABLE_V1_MULTIPROCESSING=0 for NUMA affinity [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) 2026-06-06 16:20:57.620 | INFO | skyrl_train.utils.numa:set_numa_affinity_for_gpu:340 - NUMA affinity: bound process to CPUs 0-71 (NUMA node 0) for GPU 0 (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) 2026-06-06 16:20:57.643 | DEBUG | skyrl_train.utils.numa:_set_membind_via_libnuma:375 - NUMA affinity: set memory preferred to NUMA node 0 (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) 2026-06-06 16:20:57.643 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:584 - BaseVLLMInferenceEngine: vllm_v1_disable_multiproc=True, vllm.__version__=dev, VLLM_ENABLE_V1_MULTIPROCESSING=0 (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) 2026-06-06 16:20:57.644 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:592 - BaseVLLMInferenceEngine: set VLLM_ENABLE_V1_MULTIPROCESSING=0 (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) 2026-06-06 16:20:57.644 | WARNING | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1150 - OpenAI API sampling params overridden: temperature=0.7, top_p=0.95, top_k=20 (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) 2026-06-06 16:20:57.676 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1202 - Engine startup stagger: sleeping 2.41s (attempt 1/5) to avoid port collisions (pid=1449091, ip=10.128.16.42) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: [repeated 17x across cluster] (pid=1449091, ip=10.128.16.42) No module named 'vllm._version' [repeated 17x across cluster] (pid=1449091, ip=10.128.16.42) from .version import __version__, __version_tuple__ # isort:skip [repeated 17x across cluster] (raylet, ip=10.128.16.41) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 116x across cluster] (skyrl_entrypoint pid=1407951) [2026-06-06 16:20:59] INFO inference_engine_client_http_endpoint.py:350: Starting server on 0.0.0.0:8000 (skyrl_entrypoint pid=1407951) [2026-06-06 16:20:59] INFO inference_engine_client_http_endpoint.py:242: Starting inference HTTP endpoint... (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) The tokenizer you are loading from '/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. (skyrl_entrypoint pid=1407951) [2026-06-06 16:21:00] INFO inference_engine_client_http_endpoint.py:229: Server ready after 2 attempts (2 seconds) (skyrl_entrypoint pid=1407951) 2026-06-06 16:21:00.276 | INFO  | skyrl_train.inference_engines.inference_engine_client:_spin_up_http_endpoint:969 - InferenceEngineClient HTTP endpoint started on 127.0.0.1:8000 (skyrl_entrypoint pid=1407951) 2026-06-06 16:21:00.276 | INFO  | skyrl_train.inference_engines.inference_engine_client:__init__:61 - InferenceEngineClient initialized with 48 engines. (skyrl_entrypoint pid=1407951) 2026-06-06 16:21:00.278 | INFO  | examples.terminal_bench.terminal_bench_generator:_configure_harbor_logging:225 - Harbor logging level set to INFO (skyrl_entrypoint pid=1407951) 2026-06-06 16:21:00.279 | INFO  | examples.terminal_bench.terminal_bench_generator:__init__:125 - TerminalBenchGenerator initialized with HarborConfigBuilder. Exposed fields: ['name', 'max_episodes', 'enable_summarize', 'store_all_messages', 'trajectory_config', 'enable_episode_logging', 'record_terminal_session', 'enable_pane_logging', 'strict_json_parser', 'interleaved_thinking', 'extra_body', 'override_timeout_sec', 'override_cpus', 'override_memory_mb', 'override_storage_mb', 'auto_snapshot', 'verifier_override_timeout_sec', 'max_retries', 'min_wait_sec', 'max_wait_sec', 'wait_multiplier', 'exclude_exceptions', 'n_concurrent_trials', 'log_level', 'enable_reward_shaping', 'enable_error_classification', 'mask_exceptions', 'default_error_treatment', 'passthrough_exceptions', 'zero_exceptions']. Retry config: max_retries=3, backoff=60.0-600.0s. Concurrent trials: 675. Reward shaping: enabled=False, shaper=pass_ratio. Error classification: enabled=True (skyrl_entrypoint pid=1407951) 2026-06-06 16:21:00.280 | INFO  | examples.terminal_bench.terminal_bench_generator:__init__:141 - TerminalBenchGenerator initialized with custom chat template read from: chat_templates/qwen3_thinking_acc.jinja2 (skyrl_entrypoint pid=1407951) 2026-06-06 16:21:00.280 | INFO  | skyrl_train.utils.trainer_utils:build_dataloader:656 - Total steps: 156 (skyrl_entrypoint pid=1407951) 2026-06-06 16:21:00.280 | INFO  | skyrl_train.fully_async_trainer:_build_train_dataloader_and_compute_training_steps:357 - Length of train_dataloader: 5000 (skyrl_entrypoint pid=1407951) 2026-06-06 16:21:00.280 | INFO  | skyrl_train.fully_async_trainer:_build_train_dataloader_and_compute_training_steps:358 - Number of steps per epoch: 78 (skyrl_entrypoint pid=1407951) 2026-06-06 16:21:00.280 | INFO  | skyrl_train.fully_async_trainer:_build_train_dataloader_and_compute_training_steps:359 - Total training steps: 80 (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) 2026-06-06 16:21:01.105 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:123 - setup_envvars_for_vllm: distributed_executor_backend=uni, SKYRL_ENABLE_NUMA_AFFINITY=1, CUDA_VISIBLE_DEVICES=0, VLLM_ENABLE_V1_MULTIPROCESSING= [repeated 19x across cluster] (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) 2026-06-06 16:21:01.106 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:133 - setup_envvars_for_vllm: numa_enabled=True [repeated 19x across cluster] (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) 2026-06-06 16:21:01.106 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:136 - setup_envvars_for_vllm: set VLLM_ENABLE_V1_MULTIPROCESSING=0 for NUMA affinity [repeated 19x across cluster] (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) 2026-06-06 16:21:01.902 | INFO | skyrl_train.utils.numa:set_numa_affinity_for_gpu:340 - NUMA affinity: bound process to CPUs 216-287 (NUMA node 3) for GPU 3 [repeated 23x across cluster] (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) 2026-06-06 16:21:01.926 | DEBUG | skyrl_train.utils.numa:_set_membind_via_libnuma:375 - NUMA affinity: set memory preferred to NUMA node 3 [repeated 23x across cluster] (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) 2026-06-06 16:21:01.926 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:584 - BaseVLLMInferenceEngine: vllm_v1_disable_multiproc=True, vllm.__version__=dev, VLLM_ENABLE_V1_MULTIPROCESSING=0 [repeated 23x across cluster] (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) 2026-06-06 16:21:01.926 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:592 - BaseVLLMInferenceEngine: set VLLM_ENABLE_V1_MULTIPROCESSING=0 [repeated 23x across cluster] (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) 2026-06-06 16:21:01.926 | WARNING | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1150 - OpenAI API sampling params overridden: temperature=0.7, top_p=0.95, top_k=20 [repeated 23x across cluster] (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) 2026-06-06 16:21:01.958 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1202 - Engine startup stagger: sleeping 2.63s (attempt 1/5) to avoid port collisions [repeated 23x across cluster] (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: [repeated 22x across cluster] (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) No module named 'vllm._version' [repeated 22x across cluster] (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) from .version import __version__, __version_tuple__ # isort:skip [repeated 22x across cluster] (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 55x across cluster] (AsyncVLLMInferenceEngine pid=2121712, ip=10.128.16.185) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 42x across cluster] (AsyncVLLMInferenceEngine pid=2121712, ip=10.128.16.185) The tokenizer you are loading from '/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. [repeated 21x across cluster] (pid=1420556, ip=10.128.16.47) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) (EngineCore_DP0 pid=1089211) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/torch/__init__.py:1617: UserWarning: Please use the new API settings to control TF32 behavior, such as torch.backends.cudnn.conv.fp32_precision = 'tf32' or torch.backends.cuda.matmul.fp32_precision = 'ieee'. Old settings, e.g, torch.backends.cuda.matmul.allow_tf32 = True, torch.backends.cudnn.allow_tf32 = True, allowTF32CuDNN() and allowTF32CuBLAS() will be deprecated after Pytorch 2.9. Please see https://pytorch.org/docs/main/notes/cuda.html#tensorfloat-32-tf32-on-ampere-and-later-devices (Triggered internally at /pytorch/aten/src/ATen/Context.cpp:80.) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) (EngineCore_DP0 pid=1089211) _C._set_float32_matmul_precision(precision) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) [W606 16:21:06.942052529 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [jpbo-001-43.jupiter.internal]:59133 (errno: 97 - Address family not supported by protocol). (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) [W606 16:21:06.942633991 Utils.hpp:166] Warning: Environment variable NCCL_BLOCKING_WAIT is deprecated; use TORCH_NCCL_BLOCKING_WAIT instead (function operator()) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) [rank0]:[W606 16:21:06.945059998 Utils.hpp:137] Warning: Environment variable TORCH_NCCL_TRACE_BUFFER_SIZE is deprecated; use TORCH_FR_BUFFER_SIZE instead (function operator()) (AsyncVLLMInferenceEngine pid=1449091, ip=10.128.16.42) 2026-06-06 16:21:06.881 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:123 - setup_envvars_for_vllm: distributed_executor_backend=uni, SKYRL_ENABLE_NUMA_AFFINITY=1, CUDA_VISIBLE_DEVICES=0, VLLM_ENABLE_V1_MULTIPROCESSING= [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1449091, ip=10.128.16.42) 2026-06-06 16:21:06.882 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:133 - setup_envvars_for_vllm: numa_enabled=True [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1449091, ip=10.128.16.42) 2026-06-06 16:21:06.882 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:136 - setup_envvars_for_vllm: set VLLM_ENABLE_V1_MULTIPROCESSING=0 for NUMA affinity [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) (EngineCore_DP0 pid=1089211) Loading safetensors checkpoint shards: 0% Completed | 0/4 [00:00 [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) 2026-06-06 16:21:10.936 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:133 - setup_envvars_for_vllm: numa_enabled=True [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) 2026-06-06 16:21:10.936 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:136 - setup_envvars_for_vllm: set VLLM_ENABLE_V1_MULTIPROCESSING=0 for NUMA affinity [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) (EngineCore_DP0 pid=1456098) _C._set_float32_matmul_precision(precision) (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) (EngineCore_DP0 pid=1456103) _C._set_float32_matmul_precision(precision) (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) (EngineCore_DP0 pid=1089257) Loading safetensors checkpoint shards: 0% Completed | 0/4 [00:00 10.128.16.35 (routable head IP) for coordinator litellm base_url connectivity (skyrl_entrypoint pid=1407951) 2026-06-06 16:21:44.101 | INFO  | examples.terminal_bench.rollout_coordinator:__init__:405 - [RolloutDispatcher] configured num_coordinators=4, cpus_per_coordinator=8 (skyrl_entrypoint pid=1407951) 2026-06-06 16:21:44.149 | INFO  | examples.terminal_bench.rollout_coordinator:startup:434 - [RolloutDispatcher] PlacementGroup ready: 4 bundles x 8 CPU (SPREAD) (raylet, ip=10.128.16.42) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e (FSDPPolicyWorkerBase pid=1408114) [2026-06-06 16:21:45,260 E 1408114 1408305] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) (EngineCore_DP0 pid=1449467) The tokenizer you are loading from '/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) (EngineCore_DP0 pid=1420371) (FSDPPolicyWorkerBase pid=1408117) [rank4]:[W606 16:21:41.021454127 Utils.hpp:137] Warning: Environment variable TORCH_NCCL_TRACE_BUFFER_SIZE is deprecated; use TORCH_FR_BUFFER_SIZE instead (function operator()) [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) (EngineCore_DP0 pid=1420367) (AsyncVLLMInferenceEngine pid=1420132, ip=10.128.16.41) (EngineCore_DP0 pid=1420359) (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) (EngineCore_DP0 pid=1420383) (RolloutCoordinator pid=1449700, ip=10.128.16.42) 2026-06-06 16:21:49.314 | INFO | examples.terminal_bench.rollout_coordinator:_scale_terminal_bench_cfg:121 - [RolloutCoordinator] scaled n_concurrent_trials 675 -> 168 (// 4) (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) (EngineCore_DP0 pid=1420383) Loading safetensors checkpoint shards: 100% Completed | 4/4 [00:17<00:00, 4.41s/it] [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1449221, ip=10.128.16.42) 2026-06-06 16:21:43.946 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1259 - Initializing OpenAIServingChat with custom_chat_template read from: chat_templates/qwen3_thinking_acc.jinja2 [repeated 11x across cluster] (raylet, ip=10.128.16.42) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 5x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) 2026-06-06 16:21:49.525 | INFO | examples.terminal_bench.terminal_bench_generator:_configure_harbor_logging:225 - Harbor logging level set to INFO (RolloutCoordinator pid=1449700, ip=10.128.16.42) 2026-06-06 16:21:49.525 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:125 - TerminalBenchGenerator initialized with HarborConfigBuilder. Exposed fields: ['name', 'max_episodes', 'enable_summarize', 'store_all_messages', 'trajectory_config', 'enable_episode_logging', 'record_terminal_session', 'enable_pane_logging', 'strict_json_parser', 'interleaved_thinking', 'extra_body', 'override_timeout_sec', 'override_cpus', 'override_memory_mb', 'override_storage_mb', 'auto_snapshot', 'verifier_override_timeout_sec', 'max_retries', 'min_wait_sec', 'max_wait_sec', 'wait_multiplier', 'exclude_exceptions', 'n_concurrent_trials', 'log_level', 'enable_reward_shaping', 'enable_error_classification', 'mask_exceptions', 'default_error_treatment', 'passthrough_exceptions', 'zero_exceptions']. Retry config: max_retries=3, backoff=60.0-600.0s. Concurrent trials: 168. Reward shaping: enabled=False, shaper=pass_ratio. Error classification: enabled=True (RolloutCoordinator pid=1449700, ip=10.128.16.42) 2026-06-06 16:21:49.525 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:141 - TerminalBenchGenerator initialized with custom chat template read from: chat_templates/qwen3_thinking_acc.jinja2 (RolloutCoordinator pid=1449700, ip=10.128.16.42) 2026-06-06 16:21:49.525 | INFO | examples.terminal_bench.rollout_coordinator:__init__:246 - [RolloutCoordinator 0/4] constructed (http=10.128.16.35:8000) (RolloutCoordinator pid=1449700, ip=10.128.16.42) 2026-06-06 16:21:49.528 | INFO | examples.terminal_bench.terminal_bench_generator:_create_orchestrator:281 - QueueOrchestrator created and started with n_concurrent_trials=168, rollback_hook registered for ContextLengthExceededError/AgentTimeoutError (RolloutCoordinator pid=1449700, ip=10.128.16.42) 2026-06-06 16:21:49.528 | INFO | examples.terminal_bench.terminal_bench_generator:startup:238 - TerminalBenchGenerator startup complete. Shared orchestrator ready with n_concurrent_trials=168 (RolloutCoordinator pid=1449700, ip=10.128.16.42) 2026-06-06 16:21:49.528 | INFO | examples.terminal_bench.rollout_coordinator:startup:254 - [RolloutCoordinator 0] startup complete (skyrl_entrypoint pid=1407951) 2026-06-06 16:21:49.529 | INFO  | examples.terminal_bench.rollout_coordinator:startup:469 - [RolloutDispatcher] coordinator 1/4 started (FSDPPolicyWorkerBase pid=1420632, ip=10.128.16.47) [2026-06-06 16:21:45,399 E 1420632 1420800] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) (EngineCore_DP0 pid=1420371) The tokenizer you are loading from '/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) 2026-06-06 16:21:57.740 | INFO | examples.terminal_bench.rollout_coordinator:_scale_terminal_bench_cfg:121 - [RolloutCoordinator] scaled n_concurrent_trials 675 -> 168 (// 4) (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) 2026-06-06 16:21:51.255 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1259 - Initializing OpenAIServingChat with custom_chat_template read from: chat_templates/qwen3_thinking_acc.jinja2 [repeated 4x across cluster] (raylet, ip=10.128.16.36) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) (EngineCore_DP0 pid=1420383) The tokenizer you are loading from '/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. [repeated 3x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) 2026-06-06 16:21:57.952 | INFO | examples.terminal_bench.terminal_bench_generator:_configure_harbor_logging:225 - Harbor logging level set to INFO (RolloutCoordinator pid=1427486, ip=10.128.16.36) 2026-06-06 16:21:57.953 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:125 - TerminalBenchGenerator initialized with HarborConfigBuilder. Exposed fields: ['name', 'max_episodes', 'enable_summarize', 'store_all_messages', 'trajectory_config', 'enable_episode_logging', 'record_terminal_session', 'enable_pane_logging', 'strict_json_parser', 'interleaved_thinking', 'extra_body', 'override_timeout_sec', 'override_cpus', 'override_memory_mb', 'override_storage_mb', 'auto_snapshot', 'verifier_override_timeout_sec', 'max_retries', 'min_wait_sec', 'max_wait_sec', 'wait_multiplier', 'exclude_exceptions', 'n_concurrent_trials', 'log_level', 'enable_reward_shaping', 'enable_error_classification', 'mask_exceptions', 'default_error_treatment', 'passthrough_exceptions', 'zero_exceptions']. Retry config: max_retries=3, backoff=60.0-600.0s. Concurrent trials: 168. Reward shaping: enabled=False, shaper=pass_ratio. Error classification: enabled=True (RolloutCoordinator pid=1427486, ip=10.128.16.36) 2026-06-06 16:21:57.953 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:141 - TerminalBenchGenerator initialized with custom chat template read from: chat_templates/qwen3_thinking_acc.jinja2 (RolloutCoordinator pid=1427486, ip=10.128.16.36) 2026-06-06 16:21:57.953 | INFO | examples.terminal_bench.rollout_coordinator:__init__:246 - [RolloutCoordinator 1/4] constructed (http=10.128.16.35:8000) (skyrl_entrypoint pid=1407951) 2026-06-06 16:21:57.957 | INFO  | examples.terminal_bench.rollout_coordinator:startup:469 - [RolloutDispatcher] coordinator 2/4 started (RolloutCoordinator pid=1427486, ip=10.128.16.36) 2026-06-06 16:21:57.956 | INFO | examples.terminal_bench.terminal_bench_generator:_create_orchestrator:281 - QueueOrchestrator created and started with n_concurrent_trials=168, rollback_hook registered for ContextLengthExceededError/AgentTimeoutError (RolloutCoordinator pid=1427486, ip=10.128.16.36) 2026-06-06 16:21:57.956 | INFO | examples.terminal_bench.terminal_bench_generator:startup:238 - TerminalBenchGenerator startup complete. Shared orchestrator ready with n_concurrent_trials=168 (RolloutCoordinator pid=1427486, ip=10.128.16.36) 2026-06-06 16:21:57.956 | INFO | examples.terminal_bench.rollout_coordinator:startup:254 - [RolloutCoordinator 1] startup complete (RolloutCoordinator pid=1237433, ip=10.128.16.45) 2026-06-06 16:22:06.013 | INFO | examples.terminal_bench.rollout_coordinator:_scale_terminal_bench_cfg:121 - [RolloutCoordinator] scaled n_concurrent_trials 675 -> 168 (// 4) (raylet, ip=10.128.16.45) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 6x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) The tokenizer you are loading from '/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. (RolloutCoordinator pid=1237433, ip=10.128.16.45) The tokenizer you are loading from '/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. (RolloutCoordinator pid=1237433, ip=10.128.16.45) 2026-06-06 16:22:06.234 | INFO | examples.terminal_bench.terminal_bench_generator:_configure_harbor_logging:225 - Harbor logging level set to INFO (RolloutCoordinator pid=1237433, ip=10.128.16.45) 2026-06-06 16:22:06.234 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:125 - TerminalBenchGenerator initialized with HarborConfigBuilder. Exposed fields: ['name', 'max_episodes', 'enable_summarize', 'store_all_messages', 'trajectory_config', 'enable_episode_logging', 'record_terminal_session', 'enable_pane_logging', 'strict_json_parser', 'interleaved_thinking', 'extra_body', 'override_timeout_sec', 'override_cpus', 'override_memory_mb', 'override_storage_mb', 'auto_snapshot', 'verifier_override_timeout_sec', 'max_retries', 'min_wait_sec', 'max_wait_sec', 'wait_multiplier', 'exclude_exceptions', 'n_concurrent_trials', 'log_level', 'enable_reward_shaping', 'enable_error_classification', 'mask_exceptions', 'default_error_treatment', 'passthrough_exceptions', 'zero_exceptions']. Retry config: max_retries=3, backoff=60.0-600.0s. Concurrent trials: 168. Reward shaping: enabled=False, shaper=pass_ratio. Error classification: enabled=True (RolloutCoordinator pid=1237433, ip=10.128.16.45) 2026-06-06 16:22:06.235 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:141 - TerminalBenchGenerator initialized with custom chat template read from: chat_templates/qwen3_thinking_acc.jinja2 (RolloutCoordinator pid=1237433, ip=10.128.16.45) 2026-06-06 16:22:06.235 | INFO | examples.terminal_bench.rollout_coordinator:__init__:246 - [RolloutCoordinator 2/4] constructed (http=10.128.16.35:8000) (skyrl_entrypoint pid=1407951) 2026-06-06 16:22:06.238 | INFO  | examples.terminal_bench.rollout_coordinator:startup:469 - [RolloutDispatcher] coordinator 3/4 started (RolloutCoordinator pid=1237433, ip=10.128.16.45) 2026-06-06 16:22:06.237 | INFO | examples.terminal_bench.terminal_bench_generator:_create_orchestrator:281 - QueueOrchestrator created and started with n_concurrent_trials=168, rollback_hook registered for ContextLengthExceededError/AgentTimeoutError (RolloutCoordinator pid=1237433, ip=10.128.16.45) 2026-06-06 16:22:06.237 | INFO | examples.terminal_bench.terminal_bench_generator:startup:238 - TerminalBenchGenerator startup complete. Shared orchestrator ready with n_concurrent_trials=168 (RolloutCoordinator pid=1237433, ip=10.128.16.45) 2026-06-06 16:22:06.238 | INFO | examples.terminal_bench.rollout_coordinator:startup:254 - [RolloutCoordinator 2] startup complete (RolloutCoordinator pid=2122385, ip=10.128.16.185) 2026-06-06 16:22:14.193 | INFO | examples.terminal_bench.rollout_coordinator:_scale_terminal_bench_cfg:121 - [RolloutCoordinator] scaled n_concurrent_trials 675 -> 168 (// 4) (raylet, ip=10.128.16.185) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 6x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) The tokenizer you are loading from '/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. (RolloutCoordinator pid=2122385, ip=10.128.16.185) 2026-06-06 16:22:14.420 | INFO | examples.terminal_bench.terminal_bench_generator:_configure_harbor_logging:225 - Harbor logging level set to INFO (RolloutCoordinator pid=2122385, ip=10.128.16.185) 2026-06-06 16:22:14.420 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:125 - TerminalBenchGenerator initialized with HarborConfigBuilder. Exposed fields: ['name', 'max_episodes', 'enable_summarize', 'store_all_messages', 'trajectory_config', 'enable_episode_logging', 'record_terminal_session', 'enable_pane_logging', 'strict_json_parser', 'interleaved_thinking', 'extra_body', 'override_timeout_sec', 'override_cpus', 'override_memory_mb', 'override_storage_mb', 'auto_snapshot', 'verifier_override_timeout_sec', 'max_retries', 'min_wait_sec', 'max_wait_sec', 'wait_multiplier', 'exclude_exceptions', 'n_concurrent_trials', 'log_level', 'enable_reward_shaping', 'enable_error_classification', 'mask_exceptions', 'default_error_treatment', 'passthrough_exceptions', 'zero_exceptions']. Retry config: max_retries=3, backoff=60.0-600.0s. Concurrent trials: 168. Reward shaping: enabled=False, shaper=pass_ratio. Error classification: enabled=True (RolloutCoordinator pid=2122385, ip=10.128.16.185) 2026-06-06 16:22:14.420 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:141 - TerminalBenchGenerator initialized with custom chat template read from: chat_templates/qwen3_thinking_acc.jinja2 (RolloutCoordinator pid=2122385, ip=10.128.16.185) 2026-06-06 16:22:14.420 | INFO | examples.terminal_bench.rollout_coordinator:__init__:246 - [RolloutCoordinator 3/4] constructed (http=10.128.16.35:8000) (RolloutCoordinator pid=2122385, ip=10.128.16.185) 2026-06-06 16:22:14.423 | INFO | examples.terminal_bench.terminal_bench_generator:_create_orchestrator:281 - QueueOrchestrator created and started with n_concurrent_trials=168, rollback_hook registered for ContextLengthExceededError/AgentTimeoutError (RolloutCoordinator pid=2122385, ip=10.128.16.185) 2026-06-06 16:22:14.423 | INFO | examples.terminal_bench.terminal_bench_generator:startup:238 - TerminalBenchGenerator startup complete. Shared orchestrator ready with n_concurrent_trials=168 (RolloutCoordinator pid=2122385, ip=10.128.16.185) 2026-06-06 16:22:14.423 | INFO | examples.terminal_bench.rollout_coordinator:startup:254 - [RolloutCoordinator 3] startup complete (skyrl_entrypoint pid=1407951) 2026-06-06 16:22:14.424 | INFO  | examples.terminal_bench.rollout_coordinator:startup:469 - [RolloutDispatcher] coordinator 4/4 started (skyrl_entrypoint pid=1407951) 2026-06-06 16:22:14.424 | INFO  | examples.terminal_bench.rollout_coordinator:startup:477 - [RolloutDispatcher] 4 coordinators started (skyrl_entrypoint pid=1407951) 2026-06-06 16:22:14.424 | INFO  | skyrl_train.fully_async_trainer:train:485 - Generator startup complete (skyrl_entrypoint pid=1407951) 2026-06-06 16:22:14.424 | INFO  | skyrl_train.fully_async_trainer:_train_loop:510 - Started: 'load_checkpoints' (skyrl_entrypoint pid=1407951) 2026-06-06 16:22:14.425 | INFO  | skyrl_train.trainer:load_checkpoints:1628 - No checkpoint found, starting training from scratch (skyrl_entrypoint pid=1407951) 2026-06-06 16:22:14.425 | INFO  | skyrl_train.fully_async_trainer:_train_loop:512 - Resumed training from global_step 0 (skyrl_entrypoint pid=1407951) 2026-06-06 16:22:14.425 | INFO  | skyrl_train.fully_async_trainer:_train_loop:510 - Finished: 'load_checkpoints', time cost: 0.00s (skyrl_entrypoint pid=1407951) 2026-06-06 16:22:14.425 | INFO  | skyrl_train.fully_async_trainer:_train_loop:544 - Started: 'init_weight_sync_state' (FSDPPolicyWorkerBase pid=1420556, ip=10.128.16.47) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: (FSDPPolicyWorkerBase pid=1420556, ip=10.128.16.47) No module named 'vllm._version' (FSDPPolicyWorkerBase pid=1420556, ip=10.128.16.47) from .version import __version__, __version_tuple__ # isort:skip (RolloutCoordinator pid=1449700, ip=10.128.16.42) [2026-06-06 16:22:14,915 E 1449700 1449740] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 (FSDPPolicyWorkerBase pid=1420556, ip=10.128.16.47) 2026-06-06 16:22:18.821 | INFO  | logging:info:2216 - [weight-sync] Using master_addr=10.128.16.47, master_port=52813 (FSDPPolicyWorkerBase pid=1420556, ip=10.128.16.47) [rank0]:[W606 16:22:18.088095115 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [jpbo-001-47.jupiter.internal]:52813 (errno: 97 - Address family not supported by protocol). (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) (EngineCore_DP0 pid=1446247) 2026-06-06 16:22:18.976 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:init_weight_update_communicator:189 - torch.distributed.get_rank(): 0, rank_offset: 33, rank: 33, world_size: 49, group_name: skyrl (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) (EngineCore_DP0 pid=1446273) 2026-06-06 16:22:18.979 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:init_weight_update_communicator:200 - init_weight_update_communicator: master_address=10.128.16.47, master_port=52813, (FSDPPolicyWorkerBase pid=1420556, ip=10.128.16.47) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/torch/distributed/distributed_c10d.py:4876: UserWarning: barrier(): using the device under current context. You can specify `device_id` in `init_process_group` to mute this warning. (FSDPPolicyWorkerBase pid=1420556, ip=10.128.16.47) warnings.warn( # warn only once (skyrl_entrypoint pid=1407951) 2026-06-06 16:22:19.047 | INFO  | skyrl_train.trainer:init_weight_sync_state:847 - Initialized weight sync state for policy model and inference engines. (skyrl_entrypoint pid=1407951) 2026-06-06 16:22:19.047 | INFO  | skyrl_train.fully_async_trainer:_train_loop:544 - Finished: 'init_weight_sync_state', time cost: 4.62s (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) [rank0]:[W606 16:22:21.537153903 Utils.hpp:137] Warning: Environment variable TORCH_NCCL_TRACE_BUFFER_SIZE is deprecated; use TORCH_FR_BUFFER_SIZE instead (function operator()) (skyrl_entrypoint pid=1407951) 2026-06-06 16:22:19.047 | INFO  | skyrl_train.fully_async_trainer:_train_loop:548 - Started: 'sync_weights_to_inference_engines' [repeated 4x across cluster] (FSDPPolicyWorkerBase pid=1408115) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: [repeated 7x across cluster] (FSDPPolicyWorkerBase pid=1408115) No module named 'vllm._version' [repeated 7x across cluster] (FSDPPolicyWorkerBase pid=1408115) from .version import __version__, __version_tuple__ # isort:skip [repeated 7x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) [2026-06-06 16:22:22,446 E 1427486 1427526] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 (skyrl_entrypoint pid=1407951) 2026-06-06 16:22:26.753 | INFO  | skyrl_train.fully_async_trainer:_train_loop:548 - Finished: 'sync_weights_to_inference_engines', time cost: 7.71s (skyrl_entrypoint pid=1407951) 2026-06-06 16:22:26.754 | INFO  | skyrl_train.callbacks.builtin:on_train_begin:270 - HFHubUploadCallback initialized: repo=laion/ablation-pymethods2test-seqnorm, upload_steps=5, export_path=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm/exports (skyrl_entrypoint pid=1407951) Training Step Progress: 0%| | 0/80 [00:00, value=RayTaskError(ConfigAttributeError)(ConfigAttributeError("Key 'global_loss_denom' is not in struct\n full_key: trainer.algorithm.global_loss_denom\n object_type=dict")), traceback=), 'extra': {}, 'file': (name='fully_async_trainer.py', path='/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/skyrl_train/fully_async_trainer.py'), 'function': 'train', 'level': (name='ERROR', no=40, icon='❌'), 'line': 493, 'message': 'Train loop failed at global_step 1: \x1b[36mray::FSDPPolicyWorkerBase.ppo_train()\x1b[39m (pid=1408117, ip=10.128.16.35, actor_id=b46e5c16ece895cee8fda57602000000, repr=)\n File "/e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/concurrent/futures/_base.py", line 449, in result\n return self.__get_result()\n ^^^^^^^^^^^^^^^^^^^\n File "/e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result\n raise self._exception\n ^^^^^^^^^^^^^^^^^^^^^\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/skyrl_train/workers/worker.py", line 752, in ppo_train\n self.cfg.trainer.algorithm.global_loss_denom = (\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 337, in __setattr__\n raise e\n File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 334, in __setattr__\n self.__set_impl(key, value)\n File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 318, in __set_impl\n self._set_item_impl(key, value)\n File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/basecontainer.py", line 549, in _set_item_impl\n self._validate_set(key, value)\n File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 180, in _validate_set\n target = self._get_node(key) if key is not None else self\n ^^^^^^^^^^^^^^^^^^^\n File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 475, in _get_node\n self._validate_get(key)\n File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 164, in _validate_get\n self._format_and_raise(\n File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/base.py", line 231, in _format_and_raise\n format_and_raise(\n File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/_utils.py", line 899, in format_and_raise\n _raise(ex, cause)\n File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/_utils.py", line 797, in _raise\n raise ex.with_traceback(sys.exc_info()[2]) # set env var OC_CAUSE=1 for full trace\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nomegaconf.errors.ConfigAttributeError: Key \'global_loss_denom\' is not in struct\n full_key: trainer.algorithm.global_loss_denom\n object_type=dict', 'module': 'fully_async_trainer', 'name': 'skyrl_train.fully_async_trainer', 'process': (id=1407951, name='MainProcess'), 'thread': (id=70368902602464, name='MainThread'), 'time': datetime(2026, 6, 6, 17, 5, 56, 206931, tzinfo=datetime.timezone(datetime.timedelta(seconds=7200), 'CEST'))} (skyrl_entrypoint pid=1407951) Traceback (most recent call last): (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/skyrl_train/fully_async_trainer.py", line 491, in train (skyrl_entrypoint pid=1407951) await self._train_loop() (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/skyrl_train/fully_async_trainer.py", line 642, in _train_loop (skyrl_entrypoint pid=1407951) status = await self._run_training(training_input) (skyrl_entrypoint pid=1407951) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/skyrl_train/fully_async_trainer.py", line 868, in _run_training (skyrl_entrypoint pid=1407951) status = await asyncio.to_thread(self.train_critic_and_policy, training_input) (skyrl_entrypoint pid=1407951) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/asyncio/threads.py", line 25, in to_thread (skyrl_entrypoint pid=1407951) return await loop.run_in_executor(None, func_call) (skyrl_entrypoint pid=1407951) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/concurrent/futures/thread.py", line 59, in run (skyrl_entrypoint pid=1407951) result = self.fn(*self.args, **self.kwargs) (skyrl_entrypoint pid=1407951) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/skyrl_train/trainer.py", line 1434, in train_critic_and_policy (skyrl_entrypoint pid=1407951) policy_statuses = ray.get(self.policy_model.async_run_ray_method("mesh", "ppo_train", data)) (skyrl_entrypoint pid=1407951) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/ray/_private/auto_init_hook.py", line 22, in auto_init_wrapper (skyrl_entrypoint pid=1407951) return fn(*args, **kwargs) (skyrl_entrypoint pid=1407951) ^^^^^^^^^^^^^^^^^^^ (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/ray/_private/client_mode_hook.py", line 104, in wrapper (skyrl_entrypoint pid=1407951) return func(*args, **kwargs) (skyrl_entrypoint pid=1407951) ^^^^^^^^^^^^^^^^^^^^^ (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/ray/_private/worker.py", line 2967, in get (skyrl_entrypoint pid=1407951) values, debugger_breakpoint = worker.get_objects( (skyrl_entrypoint pid=1407951) ^^^^^^^^^^^^^^^^^^^ (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/ray/_private/worker.py", line 1015, in get_objects (skyrl_entrypoint pid=1407951) raise value.as_instanceof_cause() (skyrl_entrypoint pid=1407951) ray.exceptions.RayTaskError(ConfigAttributeError): ray::FSDPPolicyWorkerBase.ppo_train() (pid=1408117, ip=10.128.16.35, actor_id=b46e5c16ece895cee8fda57602000000, repr=) (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/concurrent/futures/_base.py", line 449, in result (skyrl_entrypoint pid=1407951) return self.__get_result() (skyrl_entrypoint pid=1407951) ^^^^^^^^^^^^^^^^^^^ (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (skyrl_entrypoint pid=1407951) raise self._exception (skyrl_entrypoint pid=1407951) ^^^^^^^^^^^^^^^^^^^^^ (skyrl_entrypoint pid=1407951) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (skyrl_entrypoint pid=1407951) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/skyrl_train/workers/worker.py", line 752, in ppo_train (skyrl_entrypoint pid=1407951) self.cfg.trainer.algorithm.global_loss_denom = ( (skyrl_entrypoint pid=1407951) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 337, in __setattr__ (skyrl_entrypoint pid=1407951) raise e (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 334, in __setattr__ (skyrl_entrypoint pid=1407951) self.__set_impl(key, value) (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 318, in __set_impl (skyrl_entrypoint pid=1407951) self._set_item_impl(key, value) (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/basecontainer.py", line 549, in _set_item_impl (skyrl_entrypoint pid=1407951) self._validate_set(key, value) (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 180, in _validate_set (skyrl_entrypoint pid=1407951) target = self._get_node(key) if key is not None else self (skyrl_entrypoint pid=1407951) ^^^^^^^^^^^^^^^^^^^ (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 475, in _get_node (skyrl_entrypoint pid=1407951) self._validate_get(key) (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 164, in _validate_get (skyrl_entrypoint pid=1407951) self._format_and_raise( (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/base.py", line 231, in _format_and_raise (skyrl_entrypoint pid=1407951) format_and_raise( (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/_utils.py", line 899, in format_and_raise (skyrl_entrypoint pid=1407951) _raise(ex, cause) (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/_utils.py", line 797, in _raise (skyrl_entrypoint pid=1407951) raise ex.with_traceback(sys.exc_info()[2]) # set env var OC_CAUSE=1 for full trace (skyrl_entrypoint pid=1407951) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (skyrl_entrypoint pid=1407951) omegaconf.errors.ConfigAttributeError: Key 'global_loss_denom' is not in struct (skyrl_entrypoint pid=1407951) full_key: trainer.algorithm.global_loss_denom (skyrl_entrypoint pid=1407951) object_type=dict (skyrl_entrypoint pid=1407951) (skyrl_entrypoint pid=1407951) During handling of the above exception, another exception occurred: (skyrl_entrypoint pid=1407951) (skyrl_entrypoint pid=1407951) Traceback (most recent call last): (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/loguru/_handler.py", line 204, in emit (skyrl_entrypoint pid=1407951) self._queue.put(str_record) (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/multiprocessing/queues.py", line 393, in put (skyrl_entrypoint pid=1407951) obj = _ForkingPickler.dumps(obj) (skyrl_entrypoint pid=1407951) ^^^^^^^^^^^^^^^^^^^^^^^^^^ (skyrl_entrypoint pid=1407951) File "/e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/multiprocessing/reduction.py", line 51, in dumps (skyrl_entrypoint pid=1407951) cls(buf, protocol).dump(obj) (skyrl_entrypoint pid=1407951) _pickle.PicklingError: Can't pickle : attribute lookup RayTaskError(ConfigAttributeError) on ray.exceptions failed (skyrl_entrypoint pid=1407951) --- End of logging error --- (skyrl_entrypoint pid=1407951) 2026-06-06 17:05:56.211 | WARNING  | skyrl_train.fully_async_trainer:_cancel_generator_tasks:391 - Cancelling 338 orphaned generator tasks from abnormal train loop exit (skyrl_entrypoint pid=1407951) 2026-06-06 17:05:57.215 | INFO  | skyrl_train.inference_engines.inference_engine_client_http_endpoint:shutdown_server:203 - Server shut down after 2 seconds (RolloutCoordinator pid=1237433, ip=10.128.16.45) Installing tmux from source... (skyrl_entrypoint pid=1407951) 2026-06-06 17:06:02.215 | WARNING  | skyrl_train.inference_engines.inference_engine_client:shutdown_http_endpoint:1000 - HTTP endpoint thread did not exit within 5s (skyrl_entrypoint pid=1407951) 2026-06-06 17:06:02.216 | INFO  | skyrl_train.trainer:_guarded_sync:226 - HTTP endpoint shutdown complete (RolloutCoordinator pid=1449700, ip=10.128.16.42) 2026-06-06 17:06:02.216 | INFO | examples.terminal_bench.terminal_bench_generator:shutdown:352 - Shutting down shared QueueOrchestrator... (RolloutCoordinator pid=1449700, ip=10.128.16.42) 2026-06-06 17:06:02.217 | INFO | examples.terminal_bench.terminal_bench_generator:shutdown:354 - QueueOrchestrator shutdown complete (RolloutCoordinator pid=1449700, ip=10.128.16.42) 2026-06-06 17:06:02.217 | INFO | examples.terminal_bench.rollout_coordinator:shutdown:258 - [RolloutCoordinator 0] shutdown complete (skyrl_entrypoint pid=1407951) 2026-06-06 17:06:02.332 | INFO  | skyrl_train.trainer:_guarded_async:215 - Generator shutdown complete (RolloutCoordinator pid=1237433, ip=10.128.16.45) Error closing AsyncDaytona client: Task cb=[_run_until_complete_cb() at /e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/asyncio/base_events.py:181]> got Future > attached to a different loop 2026-06-06 17:06:02.876 | ERROR | __main__:main:134 - Training failed: ray::skyrl_entrypoint() (pid=1407951, ip=10.128.16.35) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/examples/terminal_bench/entrypoints/main_tbench.py", line 105, in skyrl_entrypoint exp.run() File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/skyrl_train/entrypoints/main_base.py", line 472, in run asyncio.run(trainer.train()) File "/e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/asyncio/runners.py", line 195, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/asyncio/base_events.py", line 691, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/skyrl_train/fully_async_trainer.py", line 491, in train await self._train_loop() File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/skyrl_train/fully_async_trainer.py", line 642, in _train_loop status = await self._run_training(training_input) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/skyrl_train/fully_async_trainer.py", line 868, in _run_training status = await asyncio.to_thread(self.train_critic_and_policy, training_input) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/asyncio/threads.py", line 25, in to_thread return await loop.run_in_executor(None, func_call) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/concurrent/futures/thread.py", line 59, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/skyrl_train/trainer.py", line 1434, in train_critic_and_policy policy_statuses = ray.get(self.policy_model.async_run_ray_method("mesh", "ppo_train", data)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^ ray.exceptions.RayTaskError(ConfigAttributeError): ray::FSDPPolicyWorkerBase.ppo_train() (pid=1408117, ip=10.128.16.35, actor_id=b46e5c16ece895cee8fda57602000000, repr=) File "/e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/concurrent/futures/_base.py", line 449, in result return self.__get_result() ^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result raise self._exception ^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/skyrl_train/workers/worker.py", line 752, in ppo_train self.cfg.trainer.algorithm.global_loss_denom = ( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 337, in __setattr__ raise e File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 334, in __setattr__ self.__set_impl(key, value) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 318, in __set_impl self._set_item_impl(key, value) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/basecontainer.py", line 549, in _set_item_impl self._validate_set(key, value) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 180, in _validate_set target = self._get_node(key) if key is not None else self ^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 475, in _get_node self._validate_get(key) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 164, in _validate_get self._format_and_raise( File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/base.py", line 231, in _format_and_raise format_and_raise( File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/_utils.py", line 899, in format_and_raise _raise(ex, cause) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/_utils.py", line 797, in _raise raise ex.with_traceback(sys.exc_info()[2]) # set env var OC_CAUSE=1 for full trace ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ omegaconf.errors.ConfigAttributeError: Key 'global_loss_denom' is not in struct full_key: trainer.algorithm.global_loss_denom object_type=dict 2026-06-06 17:06:02.876 | INFO | __main__:main:137 - Shutting down Ray on head node... (RolloutCoordinator pid=1427486, ip=10.128.16.36) 2026-06-06 17:06:02.323 | INFO | examples.terminal_bench.terminal_bench_generator:shutdown:352 - Shutting down shared QueueOrchestrator... [repeated 3x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) 2026-06-06 17:06:02.324 | INFO | examples.terminal_bench.terminal_bench_generator:shutdown:354 - QueueOrchestrator shutdown complete [repeated 3x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) 2026-06-06 17:06:02.324 | INFO | examples.terminal_bench.rollout_coordinator:shutdown:258 - [RolloutCoordinator 1] shutdown complete [repeated 3x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Error closing AsyncDaytona client: Task cb=[_run_until_complete_cb() at /e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/asyncio/base_events.py:181]> got Future > attached to a different loop [repeated 3x across cluster] (skyrl_entrypoint pid=1407951) [fd-monitor] Started monitoring (every 120s) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:20:29] OK: 50 / 131,072 FDs open (0.0% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:20:29] OK: RSS 1.38 GiB | node mem 125.1/858.0 GiB used (14.6%), avail 732.8 GiB (skyrl_entrypoint pid=1407951) ⚙️ Running in WANDB offline mode (skyrl_entrypoint pid=1407951) INFO 06-06 16:20:35 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s (pid=1088873, ip=10.128.16.43) INFO 06-06 16:20:50 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) WARNING 06-06 16:20:53 [arg_utils.py:1256] The global random seed is set to 42. Since VLLM_ENABLE_V1_MULTIPROCESSING is set to False, this may affect the random state of the Python process that launched vLLM. (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:20:53 [model.py:529] Resolved architecture: Qwen3ForCausalLM (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:20:53 [model.py:1549] Using max model len 32768 (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:20:53 [arg_utils.py:1469] Using ray runtime env (env vars redacted): {'env_vars': {'NCCL_CUMEM_ENABLE': '***', 'NCCL_DEBUG': '***', 'NCCL_SOCKET_FAMILY': '***', 'NCCL_SOCKET_IFNAME': '***', 'RAY_ADDRESS': '***', 'VLLM_ALLOW_INSECURE_SERIALIZATION': '***', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': '***', 'VLLM_DISABLE_COMPILE_CACHE': '***', 'WANDB_API_KEY': '***'}} (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:20:53 [scheduler.py:224] Chunked prefill is enabled with max_num_batched_tokens=65536. (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:20:53 [vllm.py:690] Asynchronous scheduling is enabled. (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) WARNING 06-06 16:20:53 [vllm.py:728] Enforce eager set, overriding optimization level to -O0 (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:20:53 [vllm.py:846] Cudagraph is disabled under eager mode (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) WARNING 06-06 16:20:54 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) WARNING 06-06 16:20:54 [system_utils.py:140] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (pid=1455735, ip=10.128.16.48) INFO 06-06 16:20:54 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) WARNING 06-06 16:20:59 [arg_utils.py:1256] The global random seed is set to 54. Since VLLM_ENABLE_V1_MULTIPROCESSING is set to False, this may affect the random state of the Python process that launched vLLM. (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:20:59 [model.py:529] Resolved architecture: Qwen3ForCausalLM (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:20:59 [model.py:1549] Using max model len 32768 (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:20:59 [arg_utils.py:1469] Using ray runtime env (env vars redacted): {'env_vars': {'NCCL_CUMEM_ENABLE': '***', 'NCCL_DEBUG': '***', 'NCCL_SOCKET_FAMILY': '***', 'NCCL_SOCKET_IFNAME': '***', 'RAY_ADDRESS': '***', 'VLLM_ALLOW_INSECURE_SERIALIZATION': '***', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': '***', 'VLLM_DISABLE_COMPILE_CACHE': '***', 'WANDB_API_KEY': '***'}} (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:20:59 [scheduler.py:224] Chunked prefill is enabled with max_num_batched_tokens=65536. (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:20:59 [vllm.py:690] Asynchronous scheduling is enabled. (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) WARNING 06-06 16:20:59 [vllm.py:728] Enforce eager set, overriding optimization level to -O0 (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:20:59 [vllm.py:846] Cudagraph is disabled under eager mode (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) WARNING 06-06 16:20:59 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) WARNING 06-06 16:20:59 [system_utils.py:140] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (pid=1943797, ip=10.128.16.187) INFO 06-06 16:20:59 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s [repeated 23x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) WARNING 06-06 16:20:59 [arg_utils.py:1256] The global random seed is set to 57. Since VLLM_ENABLE_V1_MULTIPROCESSING is set to False, this may affect the random state of the Python process that launched vLLM. (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) INFO 06-06 16:20:59 [model.py:529] Resolved architecture: Qwen3ForCausalLM (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) INFO 06-06 16:20:59 [model.py:1549] Using max model len 32768 (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) INFO 06-06 16:20:59 [arg_utils.py:1469] Using ray runtime env (env vars redacted): {'env_vars': {'NCCL_CUMEM_ENABLE': '***', 'NCCL_DEBUG': '***', 'NCCL_SOCKET_FAMILY': '***', 'NCCL_SOCKET_IFNAME': '***', 'RAY_ADDRESS': '***', 'VLLM_ALLOW_INSECURE_SERIALIZATION': '***', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': '***', 'VLLM_DISABLE_COMPILE_CACHE': '***', 'WANDB_API_KEY': '***'}} (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) INFO 06-06 16:20:59 [scheduler.py:224] Chunked prefill is enabled with max_num_batched_tokens=65536. (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) INFO 06-06 16:20:59 [vllm.py:690] Asynchronous scheduling is enabled. (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) WARNING 06-06 16:20:59 [vllm.py:728] Enforce eager set, overriding optimization level to -O0 (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) INFO 06-06 16:20:59 [vllm.py:846] Cudagraph is disabled under eager mode (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) (EngineCore_DP0 pid=1089211) INFO 06-06 16:21:01 [core.py:97] Initializing a V1 LLM engine (vdev) with config: model='/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', speculative_config=None, tokenizer='/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=42, served_model_name=0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': , 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'splitting_ops': [], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [65536], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': , 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': 0, 'dynamic_shapes_config': {'type': , 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': True, 'static_all_moe_layers': []} (AsyncVLLMInferenceEngine pid=2121712, ip=10.128.16.185) WARNING 06-06 16:21:04 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 [repeated 21x across cluster] (AsyncVLLMInferenceEngine pid=2121712, ip=10.128.16.185) WARNING 06-06 16:21:04 [system_utils.py:140] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 21x across cluster] (pid=1236954, ip=10.128.16.45) INFO 06-06 16:21:04 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) WARNING 06-06 16:21:04 [arg_utils.py:1256] The global random seed is set to 63. Since VLLM_ENABLE_V1_MULTIPROCESSING is set to False, this may affect the random state of the Python process that launched vLLM. [repeated 22x across cluster] (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) INFO 06-06 16:21:04 [model.py:529] Resolved architecture: Qwen3ForCausalLM [repeated 22x across cluster] (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) INFO 06-06 16:21:04 [model.py:1549] Using max model len 32768 [repeated 22x across cluster] (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) INFO 06-06 16:21:04 [arg_utils.py:1469] Using ray runtime env (env vars redacted): {'env_vars': {'NCCL_CUMEM_ENABLE': '***', 'NCCL_DEBUG': '***', 'NCCL_SOCKET_FAMILY': '***', 'NCCL_SOCKET_IFNAME': '***', 'RAY_ADDRESS': '***', 'VLLM_ALLOW_INSECURE_SERIALIZATION': '***', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': '***', 'VLLM_DISABLE_COMPILE_CACHE': '***', 'WANDB_API_KEY': '***'}} [repeated 22x across cluster] (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) INFO 06-06 16:21:04 [scheduler.py:224] Chunked prefill is enabled with max_num_batched_tokens=65536. [repeated 22x across cluster] (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) INFO 06-06 16:21:04 [vllm.py:690] Asynchronous scheduling is enabled. [repeated 22x across cluster] (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) WARNING 06-06 16:21:04 [vllm.py:728] Enforce eager set, overriding optimization level to -O0 [repeated 22x across cluster] (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) INFO 06-06 16:21:04 [vllm.py:846] Cudagraph is disabled under eager mode [repeated 22x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) (EngineCore_DP0 pid=1089211) INFO 06-06 16:21:05 [worker_base.py:289] Injected into for extended collective_rpc calls ['_apply_fp8_weight_loader_patches', '_is_fp8_model', '_quantize_weights_for_fp8', '_restore_param_subclasses', '_undo_param_subclasses', 'begin_weight_update', 'destroy_weights_update_group', 'end_weight_update', 'init_weight_update_communicator', 'load_weights', 'read_named_weights', 'set_numa_affinity', 'test_rpc'] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) (EngineCore_DP0 pid=1089211) INFO 06-06 16:21:06 [parallel_state.py:1234] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.128.16.43:59133 backend=nccl (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) (EngineCore_DP0 pid=1089211) INFO 06-06 16:21:06 [parallel_state.py:1445] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) (EngineCore_DP0 pid=1089211) INFO 06-06 16:21:06 [gpu_model_runner.py:4125] Starting to load model /e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6... (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) (EngineCore_DP0 pid=1089239) INFO 06-06 16:21:07 [core.py:97] Initializing a V1 LLM engine (vdev) with config: model='/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', speculative_config=None, tokenizer='/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=57, served_model_name=0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': , 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'splitting_ops': [], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [65536], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': , 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': 0, 'dynamic_shapes_config': {'type': , 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': True, 'static_all_moe_layers': []} (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) (EngineCore_DP0 pid=1089211) INFO 06-06 16:21:07 [cuda.py:367] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION']. (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) WARNING 06-06 16:21:09 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) WARNING 06-06 16:21:09 [system_utils.py:140] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) INFO 06-06 16:21:10 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) WARNING 06-06 16:21:09 [arg_utils.py:1256] The global random seed is set to 73. Since VLLM_ENABLE_V1_MULTIPROCESSING is set to False, this may affect the random state of the Python process that launched vLLM. [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) INFO 06-06 16:21:09 [model.py:529] Resolved architecture: Qwen3ForCausalLM [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) INFO 06-06 16:21:09 [model.py:1549] Using max model len 32768 [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) INFO 06-06 16:21:09 [arg_utils.py:1469] Using ray runtime env (env vars redacted): {'env_vars': {'NCCL_CUMEM_ENABLE': '***', 'NCCL_DEBUG': '***', 'NCCL_SOCKET_FAMILY': '***', 'NCCL_SOCKET_IFNAME': '***', 'RAY_ADDRESS': '***', 'VLLM_ALLOW_INSECURE_SERIALIZATION': '***', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': '***', 'VLLM_DISABLE_COMPILE_CACHE': '***', 'WANDB_API_KEY': '***'}} [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) INFO 06-06 16:21:09 [scheduler.py:224] Chunked prefill is enabled with max_num_batched_tokens=65536. [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) INFO 06-06 16:21:09 [vllm.py:690] Asynchronous scheduling is enabled. [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) WARNING 06-06 16:21:09 [vllm.py:728] Enforce eager set, overriding optimization level to -O0 [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) INFO 06-06 16:21:09 [vllm.py:846] Cudagraph is disabled under eager mode [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) (EngineCore_DP0 pid=1089244) INFO 06-06 16:21:10 [worker_base.py:289] Injected into for extended collective_rpc calls ['_apply_fp8_weight_loader_patches', '_is_fp8_model', '_quantize_weights_for_fp8', '_restore_param_subclasses', '_undo_param_subclasses', 'begin_weight_update', 'destroy_weights_update_group', 'end_weight_update', 'init_weight_update_communicator', 'load_weights', 'read_named_weights', 'set_numa_affinity', 'test_rpc'] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) (EngineCore_DP0 pid=1089239) INFO 06-06 16:21:11 [parallel_state.py:1234] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.128.16.43:51725 backend=nccl (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) (EngineCore_DP0 pid=1089244) INFO 06-06 16:21:11 [parallel_state.py:1445] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) (EngineCore_DP0 pid=1089239) INFO 06-06 16:21:11 [gpu_model_runner.py:4125] Starting to load model /e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6... [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) (EngineCore_DP0 pid=2122131) INFO 06-06 16:21:12 [core.py:97] Initializing a V1 LLM engine (vdev) with config: model='/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', speculative_config=None, tokenizer='/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=64, served_model_name=0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': , 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'splitting_ops': [], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [65536], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': , 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': 0, 'dynamic_shapes_config': {'type': , 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': True, 'static_all_moe_layers': []} [repeated 18x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) (EngineCore_DP0 pid=1089257) INFO 06-06 16:21:12 [cuda.py:367] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION']. [repeated 3x across cluster] (pid=1420556, ip=10.128.16.47) ⚙️ Running in WANDB offline mode (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) WARNING 06-06 16:21:14 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) WARNING 06-06 16:21:15 [system_utils.py:140] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) INFO 06-06 16:21:13 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) WARNING 06-06 16:21:15 [arg_utils.py:1256] The global random seed is set to 89. Since VLLM_ENABLE_V1_MULTIPROCESSING is set to False, this may affect the random state of the Python process that launched vLLM. [repeated 13x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) INFO 06-06 16:21:15 [model.py:529] Resolved architecture: Qwen3ForCausalLM [repeated 13x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) INFO 06-06 16:21:15 [model.py:1549] Using max model len 32768 [repeated 13x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) INFO 06-06 16:21:15 [arg_utils.py:1469] Using ray runtime env (env vars redacted): {'env_vars': {'NCCL_CUMEM_ENABLE': '***', 'NCCL_DEBUG': '***', 'NCCL_SOCKET_FAMILY': '***', 'NCCL_SOCKET_IFNAME': '***', 'RAY_ADDRESS': '***', 'VLLM_ALLOW_INSECURE_SERIALIZATION': '***', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': '***', 'VLLM_DISABLE_COMPILE_CACHE': '***', 'WANDB_API_KEY': '***'}} [repeated 13x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) INFO 06-06 16:21:15 [scheduler.py:224] Chunked prefill is enabled with max_num_batched_tokens=65536. [repeated 13x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) INFO 06-06 16:21:15 [vllm.py:690] Asynchronous scheduling is enabled. [repeated 13x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) WARNING 06-06 16:21:15 [vllm.py:728] Enforce eager set, overriding optimization level to -O0 [repeated 13x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) INFO 06-06 16:21:15 [vllm.py:846] Cudagraph is disabled under eager mode [repeated 13x across cluster] (AsyncVLLMInferenceEngine pid=2121712, ip=10.128.16.185) (EngineCore_DP0 pid=2122135) INFO 06-06 16:21:15 [worker_base.py:289] Injected into for extended collective_rpc calls ['_apply_fp8_weight_loader_patches', '_is_fp8_model', '_quantize_weights_for_fp8', '_restore_param_subclasses', '_undo_param_subclasses', 'begin_weight_update', 'destroy_weights_update_group', 'end_weight_update', 'init_weight_update_communicator', 'load_weights', 'read_named_weights', 'set_numa_affinity', 'test_rpc'] [repeated 14x across cluster] (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) (EngineCore_DP0 pid=1379440) INFO 06-06 16:21:16 [parallel_state.py:1234] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.128.16.44:48075 backend=nccl [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1379081, ip=10.128.16.44) (EngineCore_DP0 pid=1379456) INFO 06-06 16:21:16 [parallel_state.py:1445] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) (EngineCore_DP0 pid=1379452) INFO 06-06 16:21:16 [gpu_model_runner.py:4125] Starting to load model /e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6... [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) (EngineCore_DP0 pid=1427240) INFO 06-06 16:21:17 [core.py:97] Initializing a V1 LLM engine (vdev) with config: model='/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', speculative_config=None, tokenizer='/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=68, served_model_name=0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': , 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'splitting_ops': [], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [65536], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': , 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': 0, 'dynamic_shapes_config': {'type': , 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': True, 'static_all_moe_layers': []} [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) (EngineCore_DP0 pid=1089211) INFO 06-06 16:21:18 [default_loader.py:293] Loading weights took 9.95 seconds (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) (EngineCore_DP0 pid=1379440) INFO 06-06 16:21:17 [cuda.py:367] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION']. [repeated 9x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) (EngineCore_DP0 pid=1089211) INFO 06-06 16:21:18 [gpu_model_runner.py:4222] Model loading took 15.27 GiB memory and 11.078706 seconds (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) WARNING 06-06 16:21:15 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) WARNING 06-06 16:21:15 [system_utils.py:140] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) INFO 06-06 16:21:19 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s [repeated 19x across cluster] (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) (EngineCore_DP0 pid=1237192) INFO 06-06 16:21:20 [worker_base.py:289] Injected into for extended collective_rpc calls ['_apply_fp8_weight_loader_patches', '_is_fp8_model', '_quantize_weights_for_fp8', '_restore_param_subclasses', '_undo_param_subclasses', 'begin_weight_update', 'destroy_weights_update_group', 'end_weight_update', 'init_weight_update_communicator', 'load_weights', 'read_named_weights', 'set_numa_affinity', 'test_rpc'] [repeated 14x across cluster] (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) (EngineCore_DP0 pid=1446247) INFO 06-06 16:21:21 [parallel_state.py:1234] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.128.16.46:34477 backend=nccl [repeated 17x across cluster] (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) (EngineCore_DP0 pid=1446247) INFO 06-06 16:21:21 [parallel_state.py:1445] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A [repeated 17x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) (EngineCore_DP0 pid=1089211) INFO 06-06 16:21:21 [gpu_worker.py:373] Available KV cache memory: 49.32 GiB (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) (EngineCore_DP0 pid=1089211) INFO 06-06 16:21:21 [kv_cache_utils.py:1307] GPU KV cache size: 359,104 tokens (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) (EngineCore_DP0 pid=1089211) INFO 06-06 16:21:21 [kv_cache_utils.py:1312] Maximum concurrency for 32,768 tokens per request: 10.96x (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) (EngineCore_DP0 pid=1089211) INFO 06-06 16:21:21 [kernel_warmup.py:44] Skipping FlashInfer autotune because it is disabled. (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) (EngineCore_DP0 pid=1089211) INFO 06-06 16:21:21 [core.py:278] init engine (profile, create kv cache, warmup model) took 3.38 seconds (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) (EngineCore_DP0 pid=1944169) INFO 06-06 16:21:21 [gpu_model_runner.py:4125] Starting to load model /e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6... [repeated 15x across cluster] (pid=1420632, ip=10.128.16.47) ⚙️ Running in WANDB offline mode (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) (EngineCore_DP0 pid=1089211) WARNING 06-06 16:21:22 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) (EngineCore_DP0 pid=1089211) INFO 06-06 16:21:22 [vllm.py:690] Asynchronous scheduling is enabled. (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) (EngineCore_DP0 pid=1089211) WARNING 06-06 16:21:22 [vllm.py:735] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored. (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) (EngineCore_DP0 pid=1089211) INFO 06-06 16:21:22 [vllm.py:846] Cudagraph is disabled under eager mode (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) WARNING 06-06 16:21:22 [model.py:1350] Default vLLM sampling parameters have been overridden by the model's `generation_config.json`: `{'temperature': 0.6, 'top_k': 20, 'top_p': 0.95}`. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`. (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) (EngineCore_DP0 pid=1446273) INFO 06-06 16:21:20 [core.py:97] Initializing a V1 LLM engine (vdev) with config: model='/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', speculative_config=None, tokenizer='/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=75, served_model_name=0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': , 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'splitting_ops': [], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [65536], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': , 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': 0, 'dynamic_shapes_config': {'type': , 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': True, 'static_all_moe_layers': []} [repeated 13x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) (EngineCore_DP0 pid=1089239) INFO 06-06 16:21:18 [default_loader.py:293] Loading weights took 5.77 seconds [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) (EngineCore_DP0 pid=1944169) INFO 06-06 16:21:21 [cuda.py:367] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION']. [repeated 15x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) (EngineCore_DP0 pid=1089244) INFO 06-06 16:21:18 [gpu_model_runner.py:4222] Model loading took 15.27 GiB memory and 6.363103 seconds [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) INFO 06-06 16:21:22 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) (EngineCore_DP0 pid=1420367) INFO 06-06 16:21:25 [worker_base.py:289] Injected into for extended collective_rpc calls ['_apply_fp8_weight_loader_patches', '_is_fp8_model', '_quantize_weights_for_fp8', '_restore_param_subclasses', '_undo_param_subclasses', 'begin_weight_update', 'destroy_weights_update_group', 'end_weight_update', 'init_weight_update_communicator', 'load_weights', 'read_named_weights', 'set_numa_affinity', 'test_rpc'] [repeated 14x across cluster] (AsyncVLLMInferenceEngine pid=1449221, ip=10.128.16.42) (EngineCore_DP0 pid=1449457) INFO 06-06 16:21:25 [parallel_state.py:1234] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.128.16.42:42579 backend=nccl [repeated 15x across cluster] (AsyncVLLMInferenceEngine pid=1449221, ip=10.128.16.42) (EngineCore_DP0 pid=1449457) INFO 06-06 16:21:25 [parallel_state.py:1445] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A [repeated 15x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) (EngineCore_DP0 pid=1089244) INFO 06-06 16:21:21 [gpu_worker.py:373] Available KV cache memory: 49.32 GiB [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) (EngineCore_DP0 pid=1089244) INFO 06-06 16:21:21 [kv_cache_utils.py:1307] GPU KV cache size: 359,104 tokens [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) (EngineCore_DP0 pid=1089244) INFO 06-06 16:21:21 [kv_cache_utils.py:1312] Maximum concurrency for 32,768 tokens per request: 10.96x [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) (EngineCore_DP0 pid=1089239) INFO 06-06 16:21:21 [kernel_warmup.py:44] Skipping FlashInfer autotune because it is disabled. [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) (EngineCore_DP0 pid=1089244) INFO 06-06 16:21:21 [core.py:278] init engine (profile, create kv cache, warmup model) took 3.40 seconds [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1449221, ip=10.128.16.42) (EngineCore_DP0 pid=1449457) INFO 06-06 16:21:26 [gpu_model_runner.py:4125] Starting to load model /e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6... [repeated 16x across cluster] (pid=1408117) ⚙️ Running in WANDB offline mode [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) (EngineCore_DP0 pid=1944143) WARNING 06-06 16:21:27 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) (EngineCore_DP0 pid=1944143) INFO 06-06 16:21:27 [vllm.py:690] Asynchronous scheduling is enabled. [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) (EngineCore_DP0 pid=1944143) WARNING 06-06 16:21:27 [vllm.py:735] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored. [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) (EngineCore_DP0 pid=1944143) INFO 06-06 16:21:27 [vllm.py:846] Cudagraph is disabled under eager mode [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) WARNING 06-06 16:21:22 [model.py:1350] Default vLLM sampling parameters have been overridden by the model's `generation_config.json`: `{'temperature': 0.6, 'top_k': 20, 'top_p': 0.95}`. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`. [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) (EngineCore_DP0 pid=1420383) INFO 06-06 16:21:23 [core.py:97] Initializing a V1 LLM engine (vdev) with config: model='/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', speculative_config=None, tokenizer='/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=89, served_model_name=0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': , 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'splitting_ops': [], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [65536], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': , 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': 0, 'dynamic_shapes_config': {'type': , 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': True, 'static_all_moe_layers': []} [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) (EngineCore_DP0 pid=1944169) INFO 06-06 16:21:26 [default_loader.py:293] Loading weights took 4.42 seconds [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1449221, ip=10.128.16.42) (EngineCore_DP0 pid=1449457) INFO 06-06 16:21:26 [cuda.py:367] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION']. [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) (EngineCore_DP0 pid=1944169) INFO 06-06 16:21:26 [gpu_model_runner.py:4222] Model loading took 15.27 GiB memory and 4.739795 seconds [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) (EngineCore_DP0 pid=1420383) INFO 06-06 16:21:28 [worker_base.py:289] Injected into for extended collective_rpc calls ['_apply_fp8_weight_loader_patches', '_is_fp8_model', '_quantize_weights_for_fp8', '_restore_param_subclasses', '_undo_param_subclasses', 'begin_weight_update', 'destroy_weights_update_group', 'end_weight_update', 'init_weight_update_communicator', 'load_weights', 'read_named_weights', 'set_numa_affinity', 'test_rpc'] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) (EngineCore_DP0 pid=1420383) INFO 06-06 16:21:28 [parallel_state.py:1234] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.128.16.41:37163 backend=nccl [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) (EngineCore_DP0 pid=1420383) INFO 06-06 16:21:28 [parallel_state.py:1445] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) (EngineCore_DP0 pid=1944169) INFO 06-06 16:21:29 [gpu_worker.py:373] Available KV cache memory: 49.32 GiB [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) (EngineCore_DP0 pid=1944169) INFO 06-06 16:21:29 [kv_cache_utils.py:1307] GPU KV cache size: 359,104 tokens [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) (EngineCore_DP0 pid=1944169) INFO 06-06 16:21:29 [kv_cache_utils.py:1312] Maximum concurrency for 32,768 tokens per request: 10.96x [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) (EngineCore_DP0 pid=1944169) INFO 06-06 16:21:29 [kernel_warmup.py:44] Skipping FlashInfer autotune because it is disabled. [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) (EngineCore_DP0 pid=1944169) INFO 06-06 16:21:29 [core.py:278] init engine (profile, create kv cache, warmup model) took 2.42 seconds [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) (EngineCore_DP0 pid=1420383) INFO 06-06 16:21:28 [gpu_model_runner.py:4125] Starting to load model /e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6... [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) (EngineCore_DP0 pid=1944169) WARNING 06-06 16:21:29 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) (EngineCore_DP0 pid=1944169) INFO 06-06 16:21:29 [vllm.py:690] Asynchronous scheduling is enabled. [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) (EngineCore_DP0 pid=1944169) WARNING 06-06 16:21:29 [vllm.py:735] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored. [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) (EngineCore_DP0 pid=1944169) INFO 06-06 16:21:29 [vllm.py:846] Cudagraph is disabled under eager mode [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) WARNING 06-06 16:21:29 [model.py:1350] Default vLLM sampling parameters have been overridden by the model's `generation_config.json`: `{'temperature': 0.6, 'top_k': 20, 'top_p': 0.95}`. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`. [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) (EngineCore_DP0 pid=2122144) INFO 06-06 16:21:32 [default_loader.py:293] Loading weights took 13.14 seconds [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) (EngineCore_DP0 pid=1420383) INFO 06-06 16:21:29 [cuda.py:367] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION']. [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) (EngineCore_DP0 pid=2122144) INFO 06-06 16:21:32 [gpu_model_runner.py:4222] Model loading took 15.27 GiB memory and 14.260447 seconds [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) (EngineCore_DP0 pid=1446255) INFO 06-06 16:21:36 [gpu_worker.py:373] Available KV cache memory: 49.32 GiB [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) (EngineCore_DP0 pid=1446255) INFO 06-06 16:21:36 [kv_cache_utils.py:1307] GPU KV cache size: 359,104 tokens [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) (EngineCore_DP0 pid=1446255) INFO 06-06 16:21:36 [kv_cache_utils.py:1312] Maximum concurrency for 32,768 tokens per request: 10.96x [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) (EngineCore_DP0 pid=1446255) INFO 06-06 16:21:36 [kernel_warmup.py:44] Skipping FlashInfer autotune because it is disabled. [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) (EngineCore_DP0 pid=1446273) INFO 06-06 16:21:36 [core.py:278] init engine (profile, create kv cache, warmup model) took 2.79 seconds [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) (EngineCore_DP0 pid=1378689) WARNING 06-06 16:21:37 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 [repeated 24x across cluster] (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) (EngineCore_DP0 pid=1378689) INFO 06-06 16:21:37 [vllm.py:690] Asynchronous scheduling is enabled. [repeated 24x across cluster] (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) (EngineCore_DP0 pid=1378689) WARNING 06-06 16:21:37 [vllm.py:735] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored. [repeated 24x across cluster] (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) (EngineCore_DP0 pid=1378689) INFO 06-06 16:21:37 [vllm.py:846] Cudagraph is disabled under eager mode [repeated 24x across cluster] (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) WARNING 06-06 16:21:38 [model.py:1350] Default vLLM sampling parameters have been overridden by the model's `generation_config.json`: `{'temperature': 0.6, 'top_k': 20, 'top_p': 0.95}`. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`. [repeated 24x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) (EngineCore_DP0 pid=1427244) INFO 06-06 16:21:39 [default_loader.py:293] Loading weights took 14.77 seconds [repeated 9x across cluster] (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) (EngineCore_DP0 pid=1378689) INFO 06-06 16:21:34 [gpu_model_runner.py:4222] Model loading took 15.27 GiB memory and 16.067231 seconds [repeated 8x across cluster] (FSDPPolicyWorkerBase pid=1420556, ip=10.128.16.47) NCCL version 2.27.7+cuda13.0 (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) (EngineCore_DP0 pid=1378689) INFO 06-06 16:21:36 [gpu_worker.py:373] Available KV cache memory: 49.32 GiB [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) (EngineCore_DP0 pid=1378679) INFO 06-06 16:21:36 [kv_cache_utils.py:1307] GPU KV cache size: 359,104 tokens [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) (EngineCore_DP0 pid=1378679) INFO 06-06 16:21:36 [kv_cache_utils.py:1312] Maximum concurrency for 32,768 tokens per request: 10.96x [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) (EngineCore_DP0 pid=1378685) INFO 06-06 16:21:37 [kernel_warmup.py:44] Skipping FlashInfer autotune because it is disabled. [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) (EngineCore_DP0 pid=1378689) INFO 06-06 16:21:37 [core.py:278] init engine (profile, create kv cache, warmup model) took 2.80 seconds [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1449221, ip=10.128.16.42) (EngineCore_DP0 pid=1449457) WARNING 06-06 16:21:43 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1449221, ip=10.128.16.42) (EngineCore_DP0 pid=1449457) INFO 06-06 16:21:43 [vllm.py:690] Asynchronous scheduling is enabled. [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1449221, ip=10.128.16.42) (EngineCore_DP0 pid=1449457) WARNING 06-06 16:21:43 [vllm.py:735] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored. [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1449221, ip=10.128.16.42) (EngineCore_DP0 pid=1449457) INFO 06-06 16:21:43 [vllm.py:846] Cudagraph is disabled under eager mode [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1449221, ip=10.128.16.42) WARNING 06-06 16:21:43 [model.py:1350] Default vLLM sampling parameters have been overridden by the model's `generation_config.json`: `{'temperature': 0.6, 'top_k': 20, 'top_p': 0.95}`. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`. [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) (EngineCore_DP0 pid=1420371) INFO 06-06 16:21:47 [default_loader.py:293] Loading weights took 17.68 seconds [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1449221, ip=10.128.16.42) (EngineCore_DP0 pid=1449457) INFO 06-06 16:21:40 [gpu_model_runner.py:4222] Model loading took 15.27 GiB memory and 13.900228 seconds [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) (EngineCore_DP0 pid=1449467) INFO 06-06 16:21:42 [gpu_worker.py:373] Available KV cache memory: 49.32 GiB [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) (EngineCore_DP0 pid=1449467) INFO 06-06 16:21:42 [kv_cache_utils.py:1307] GPU KV cache size: 359,104 tokens [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) (EngineCore_DP0 pid=1449467) INFO 06-06 16:21:42 [kv_cache_utils.py:1312] Maximum concurrency for 32,768 tokens per request: 10.96x [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) (EngineCore_DP0 pid=1449467) INFO 06-06 16:21:43 [kernel_warmup.py:44] Skipping FlashInfer autotune because it is disabled. [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1449221, ip=10.128.16.42) (EngineCore_DP0 pid=1449457) INFO 06-06 16:21:43 [core.py:278] init engine (profile, create kv cache, warmup model) took 2.80 seconds [repeated 12x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] Started monitoring (every 120s) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:21:49] OK: 46 / 131,072 FDs open (0.0% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:21:49] OK: RSS 0.78 GiB | node mem 432.7/858.0 GiB used (50.4%), avail 425.3 GiB (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) (EngineCore_DP0 pid=1420383) WARNING 06-06 16:21:51 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) (EngineCore_DP0 pid=1420383) INFO 06-06 16:21:51 [vllm.py:690] Asynchronous scheduling is enabled. [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) (EngineCore_DP0 pid=1420383) WARNING 06-06 16:21:51 [vllm.py:735] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored. [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) (EngineCore_DP0 pid=1420383) INFO 06-06 16:21:51 [vllm.py:846] Cudagraph is disabled under eager mode [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) WARNING 06-06 16:21:51 [model.py:1350] Default vLLM sampling parameters have been overridden by the model's `generation_config.json`: `{'temperature': 0.6, 'top_k': 20, 'top_p': 0.95}`. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`. [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) (EngineCore_DP0 pid=1420383) INFO 06-06 16:21:47 [default_loader.py:293] Loading weights took 17.66 seconds [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) (EngineCore_DP0 pid=1420383) INFO 06-06 16:21:47 [gpu_model_runner.py:4222] Model loading took 15.27 GiB memory and 18.374420 seconds [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) (EngineCore_DP0 pid=1420383) INFO 06-06 16:21:50 [gpu_worker.py:373] Available KV cache memory: 49.32 GiB [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) (EngineCore_DP0 pid=1420383) INFO 06-06 16:21:50 [kv_cache_utils.py:1307] GPU KV cache size: 359,104 tokens [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) (EngineCore_DP0 pid=1420383) INFO 06-06 16:21:50 [kv_cache_utils.py:1312] Maximum concurrency for 32,768 tokens per request: 10.96x [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) (EngineCore_DP0 pid=1420383) INFO 06-06 16:21:50 [kernel_warmup.py:44] Skipping FlashInfer autotune because it is disabled. [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) (EngineCore_DP0 pid=1420383) INFO 06-06 16:21:50 [core.py:278] init engine (profile, create kv cache, warmup model) took 2.83 seconds [repeated 4x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] Started monitoring (every 120s) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:21:57] OK: 46 / 131,072 FDs open (0.0% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:21:57] OK: RSS 0.78 GiB | node mem 430.6/858.0 GiB used (50.2%), avail 427.3 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] Started monitoring (every 120s) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:22:06] OK: 46 / 131,072 FDs open (0.0% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:22:06] OK: RSS 0.78 GiB | node mem 422.9/858.0 GiB used (49.3%), avail 435.0 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] Started monitoring (every 120s) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:22:14] OK: 46 / 131,072 FDs open (0.0% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:22:14] OK: RSS 0.78 GiB | node mem 398.6/858.0 GiB used (46.5%), avail 459.4 GiB (FSDPPolicyWorkerBase pid=1408115) INFO 06-06 16:22:17 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) (EngineCore_DP0 pid=1092677) INFO 06-06 16:22:26 [block_pool.py:452] Successfully reset prefix cache (FSDPPolicyWorkerBase pid=1420632, ip=10.128.16.47) INFO 06-06 16:22:17 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s [repeated 7x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:22:29] OK: 104 / 131,072 FDs open (0.1% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:22:29] OK: RSS 1.70 GiB | node mem 195.3/858.0 GiB used (22.8%), avail 662.7 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) (EngineCore_DP0 pid=1944165) INFO 06-06 16:22:26 [block_pool.py:452] Successfully reset prefix cache [repeated 47x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 1008x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 960x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 47x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:22:45 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:22:45 [loggers.py:259] Engine 000: Avg prompt throughput: 14.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.3%, Prefix cache hit rate: 0.0% (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:22:50 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. [repeated 17x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:22:50 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 49.8 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.3%, Prefix cache hit rate: 0.0% [repeated 18x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) INFO 06-06 16:22:55 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. [repeated 8x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:22:55 [loggers.py:259] Engine 000: Avg prompt throughput: 10.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.3%, Prefix cache hit rate: 0.0% [repeated 23x across cluster] (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) INFO 06-06 16:22:59 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. [repeated 8x across cluster] (AsyncVLLMInferenceEngine pid=2121712, ip=10.128.16.185) INFO 06-06 16:23:00 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 48.4 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.3%, Prefix cache hit rate: 0.0% [repeated 35x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:23:05 [loggers.py:259] Engine 000: Avg prompt throughput: 130.9 tokens/s, Avg generation throughput: 129.7 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.0%, Prefix cache hit rate: 57.4% [repeated 34x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:23:10 [loggers.py:259] Engine 000: Avg prompt throughput: 33.8 tokens/s, Avg generation throughput: 87.6 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.5%, Prefix cache hit rate: 67.1% [repeated 33x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:23:15 [loggers.py:259] Engine 000: Avg prompt throughput: 113.3 tokens/s, Avg generation throughput: 122.5 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.5%, Prefix cache hit rate: 78.7% [repeated 32x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:23:20 [loggers.py:259] Engine 000: Avg prompt throughput: 36.2 tokens/s, Avg generation throughput: 117.7 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.2%, Prefix cache hit rate: 81.3% [repeated 33x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:23:25 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 98.5 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.4%, Prefix cache hit rate: 81.3% [repeated 32x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:23:30 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 91.5 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.0%, Prefix cache hit rate: 81.3% [repeated 33x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:23:35 [loggers.py:259] Engine 000: Avg prompt throughput: 138.2 tokens/s, Avg generation throughput: 73.7 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.8%, Prefix cache hit rate: 80.9% [repeated 28x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 6x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:23:40 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 96.4 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.0%, Prefix cache hit rate: 80.9% [repeated 26x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) INFO 06-06 16:23:41 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:23:45 [loggers.py:259] Engine 000: Avg prompt throughput: 50.2 tokens/s, Avg generation throughput: 84.0 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.1%, Prefix cache hit rate: 83.2% [repeated 31x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) INFO 06-06 16:23:42 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) INFO 06-06 16:23:49 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:23:49] OK: 391 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:23:49] OK: RSS 1.12 GiB | node mem 441.6/858.0 GiB used (51.5%), avail 416.4 GiB (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:23:50 [loggers.py:259] Engine 000: Avg prompt throughput: 34.3 tokens/s, Avg generation throughput: 44.3 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.3%, Prefix cache hit rate: 85.3% [repeated 36x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) INFO 06-06 16:23:51 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. [repeated 7x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 14x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:23:55 [loggers.py:259] Engine 000: Avg prompt throughput: 141.0 tokens/s, Avg generation throughput: 92.1 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.5%, Prefix cache hit rate: 85.3% [repeated 44x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:23:57] OK: 409 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:23:57] OK: RSS 1.13 GiB | node mem 439.8/858.0 GiB used (51.3%), avail 418.2 GiB (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) INFO 06-06 16:23:55 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) INFO 06-06 16:24:01 [loggers.py:259] Engine 000: Avg prompt throughput: 33.3 tokens/s, Avg generation throughput: 84.5 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.6%, Prefix cache hit rate: 55.2% [repeated 44x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) INFO 06-06 16:24:01 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:24:06] OK: 320 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:24:06] OK: RSS 1.15 GiB | node mem 431.9/858.0 GiB used (50.3%), avail 426.1 GiB (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) INFO 06-06 16:24:06 [loggers.py:259] Engine 000: Avg prompt throughput: 64.7 tokens/s, Avg generation throughput: 131.8 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.7%, Prefix cache hit rate: 57.6% [repeated 47x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) INFO 06-06 16:24:02 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) INFO 06-06 16:24:11 [loggers.py:259] Engine 000: Avg prompt throughput: 110.1 tokens/s, Avg generation throughput: 109.9 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.2%, Prefix cache hit rate: 68.8% [repeated 46x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) INFO 06-06 16:24:13 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:24:14] OK: 250 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:24:14] OK: RSS 1.13 GiB | node mem 407.2/858.0 GiB used (47.5%), avail 450.7 GiB (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) INFO 06-06 16:24:16 [loggers.py:259] Engine 000: Avg prompt throughput: 79.5 tokens/s, Avg generation throughput: 112.9 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.5%, Prefix cache hit rate: 72.2% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) INFO 06-06 16:24:21 [loggers.py:259] Engine 000: Avg prompt throughput: 100.1 tokens/s, Avg generation throughput: 119.6 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.2%, Prefix cache hit rate: 74.6% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) INFO 06-06 16:24:26 [loggers.py:259] Engine 000: Avg prompt throughput: 101.9 tokens/s, Avg generation throughput: 119.1 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.1%, Prefix cache hit rate: 79.1% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:24:29] OK: 302 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:24:29] OK: RSS 1.72 GiB | node mem 195.5/858.0 GiB used (22.8%), avail 662.5 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) INFO 06-06 16:24:31 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 123.7 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.6%, Prefix cache hit rate: 79.1% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) INFO 06-06 16:24:36 [loggers.py:259] Engine 000: Avg prompt throughput: 135.6 tokens/s, Avg generation throughput: 61.8 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.2%, Prefix cache hit rate: 79.9% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) INFO 06-06 16:24:41 [loggers.py:259] Engine 000: Avg prompt throughput: 58.8 tokens/s, Avg generation throughput: 79.8 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.0%, Prefix cache hit rate: 81.3% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 22x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) INFO 06-06 16:24:46 [loggers.py:259] Engine 000: Avg prompt throughput: 140.7 tokens/s, Avg generation throughput: 131.5 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.4%, Prefix cache hit rate: 82.0% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 19x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) INFO 06-06 16:24:51 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 145.3 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.6%, Prefix cache hit rate: 82.0% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) INFO 06-06 16:24:56 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 115.4 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.6%, Prefix cache hit rate: 82.0% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) INFO 06-06 16:25:01 [loggers.py:259] Engine 000: Avg prompt throughput: 94.5 tokens/s, Avg generation throughput: 110.5 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.0%, Prefix cache hit rate: 82.9% [repeated 49x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 37x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) INFO 06-06 16:25:06 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 133.7 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.2%, Prefix cache hit rate: 82.9% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) INFO 06-06 16:25:11 [loggers.py:259] Engine 000: Avg prompt throughput: 207.0 tokens/s, Avg generation throughput: 100.0 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.2%, Prefix cache hit rate: 82.8% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) INFO 06-06 16:25:16 [loggers.py:259] Engine 000: Avg prompt throughput: 167.4 tokens/s, Avg generation throughput: 161.3 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 5.1%, Prefix cache hit rate: 83.5% [repeated 47x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) INFO 06-06 16:25:21 [loggers.py:259] Engine 000: Avg prompt throughput: 34.1 tokens/s, Avg generation throughput: 162.2 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 5.3%, Prefix cache hit rate: 83.7% [repeated 46x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) INFO 06-06 16:25:26 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 177.9 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 5.6%, Prefix cache hit rate: 83.7% [repeated 50x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) INFO 06-06 16:25:31 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 178.2 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 5.8%, Prefix cache hit rate: 83.7% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 29x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) INFO 06-06 16:25:36 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 163.9 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 5.6%, Prefix cache hit rate: 83.7% [repeated 47x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) INFO 06-06 16:25:41 [loggers.py:259] Engine 000: Avg prompt throughput: 85.2 tokens/s, Avg generation throughput: 176.5 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 6.4%, Prefix cache hit rate: 83.7% [repeated 47x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) INFO 06-06 16:25:46 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 177.5 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 6.7%, Prefix cache hit rate: 83.7% [repeated 47x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:25:49] OK: 401 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:25:49] OK: RSS 1.19 GiB | node mem 441.6/858.0 GiB used (51.5%), avail 416.4 GiB (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) INFO 06-06 16:25:51 [loggers.py:259] Engine 000: Avg prompt throughput: 89.9 tokens/s, Avg generation throughput: 182.4 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 7.1%, Prefix cache hit rate: 83.2% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) INFO 06-06 16:25:56 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 203.1 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 6.6%, Prefix cache hit rate: 83.2% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:25:57] OK: 405 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:25:57] OK: RSS 1.14 GiB | node mem 439.9/858.0 GiB used (51.3%), avail 418.1 GiB (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) INFO 06-06 16:26:01 [loggers.py:259] Engine 000: Avg prompt throughput: 235.6 tokens/s, Avg generation throughput: 185.2 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 5.8%, Prefix cache hit rate: 83.0% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:26:06] OK: 346 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:26:06] OK: RSS 1.15 GiB | node mem 431.9/858.0 GiB used (50.3%), avail 426.1 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster] (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) INFO 06-06 16:26:06 [loggers.py:259] Engine 000: Avg prompt throughput: 302.7 tokens/s, Avg generation throughput: 241.9 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 8.7%, Prefix cache hit rate: 83.3% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) INFO 06-06 16:26:11 [loggers.py:259] Engine 000: Avg prompt throughput: 133.7 tokens/s, Avg generation throughput: 144.7 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.9%, Prefix cache hit rate: 82.3% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:26:14] OK: 260 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:26:14] OK: RSS 1.15 GiB | node mem 407.2/858.0 GiB used (47.5%), avail 450.7 GiB (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 19x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) INFO 06-06 16:26:16 [loggers.py:259] Engine 000: Avg prompt throughput: 33.1 tokens/s, Avg generation throughput: 166.0 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.3%, Prefix cache hit rate: 82.6% [repeated 47x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) INFO 06-06 16:26:21 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 192.1 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.6%, Prefix cache hit rate: 82.6% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 31x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) INFO 06-06 16:26:26 [loggers.py:259] Engine 000: Avg prompt throughput: 66.7 tokens/s, Avg generation throughput: 161.8 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.6%, Prefix cache hit rate: 82.0% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:26:29] OK: 390 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:26:29] OK: RSS 1.74 GiB | node mem 195.5/858.0 GiB used (22.8%), avail 662.5 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 32x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) INFO 06-06 16:26:31 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 340.2 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 14.5%, Prefix cache hit rate: 86.0% [repeated 49x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 21x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) INFO 06-06 16:26:36 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 340.7 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 15.0%, Prefix cache hit rate: 86.0% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 30x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) INFO 06-06 16:26:41 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 340.0 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 15.5%, Prefix cache hit rate: 86.0% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 31x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) INFO 06-06 16:26:46 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 310.6 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 15.2%, Prefix cache hit rate: 86.0% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) INFO 06-06 16:26:51 [loggers.py:259] Engine 000: Avg prompt throughput: 229.8 tokens/s, Avg generation throughput: 331.0 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 16.7%, Prefix cache hit rate: 85.6% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 31x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) INFO 06-06 16:26:56 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 340.1 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 17.2%, Prefix cache hit rate: 85.6% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 21x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) INFO 06-06 16:27:01 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 339.4 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 17.6%, Prefix cache hit rate: 85.6% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 31x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) INFO 06-06 16:27:06 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 339.3 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 18.1%, Prefix cache hit rate: 85.6% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 33x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) INFO 06-06 16:27:11 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 338.9 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 18.6%, Prefix cache hit rate: 85.6% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:27:16 [loggers.py:259] Engine 000: Avg prompt throughput: 174.0 tokens/s, Avg generation throughput: 331.9 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 14.7%, Prefix cache hit rate: 88.6% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 30x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:27:21 [loggers.py:259] Engine 000: Avg prompt throughput: 21.0 tokens/s, Avg generation throughput: 346.8 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 15.2%, Prefix cache hit rate: 88.9% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 49x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:27:26 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 333.8 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 11.3%, Prefix cache hit rate: 88.9% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 30x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:27:31 [loggers.py:259] Engine 000: Avg prompt throughput: 179.3 tokens/s, Avg generation throughput: 381.8 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 16.5%, Prefix cache hit rate: 89.4% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 21x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) INFO 06-06 16:27:37 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 243.3 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 10.3%, Prefix cache hit rate: 86.5% [repeated 49x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) INFO 06-06 16:27:42 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 196.8 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 9.1%, Prefix cache hit rate: 86.5% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 30x across cluster] (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) INFO 06-06 16:27:47 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 194.8 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 9.3%, Prefix cache hit rate: 86.5% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:27:49] OK: 412 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:27:49] OK: RSS 1.21 GiB | node mem 441.6/858.0 GiB used (51.5%), avail 416.4 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) INFO 06-06 16:27:52 [loggers.py:259] Engine 000: Avg prompt throughput: 53.0 tokens/s, Avg generation throughput: 228.0 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 11.2%, Prefix cache hit rate: 87.0% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster] (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) INFO 06-06 16:27:57 [loggers.py:259] Engine 000: Avg prompt throughput: 116.0 tokens/s, Avg generation throughput: 236.7 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 9.4%, Prefix cache hit rate: 90.4% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:27:57] OK: 291 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:27:57] OK: RSS 1.18 GiB | node mem 439.9/858.0 GiB used (51.3%), avail 418.1 GiB (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) INFO 06-06 16:28:02 [loggers.py:259] Engine 000: Avg prompt throughput: 128.5 tokens/s, Avg generation throughput: 179.5 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 8.5%, Prefix cache hit rate: 86.6% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:28:06] OK: 355 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:28:06] OK: RSS 1.17 GiB | node mem 431.9/858.0 GiB used (50.3%), avail 426.1 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 39x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) INFO 06-06 16:28:07 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 120.3 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 7.0%, Prefix cache hit rate: 86.6% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) INFO 06-06 16:28:12 [loggers.py:259] Engine 000: Avg prompt throughput: 158.1 tokens/s, Avg generation throughput: 103.7 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 9.1%, Prefix cache hit rate: 86.8% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:28:14] OK: 263 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:28:14] OK: RSS 1.15 GiB | node mem 407.3/858.0 GiB used (47.5%), avail 450.7 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) INFO 06-06 16:28:17 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 188.5 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 7.8%, Prefix cache hit rate: 86.8% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) INFO 06-06 16:28:22 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 151.4 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 8.0%, Prefix cache hit rate: 86.8% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 11x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) INFO 06-06 16:28:27 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 151.6 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 8.3%, Prefix cache hit rate: 86.8% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:28:29] OK: 399 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:28:29] OK: RSS 1.75 GiB | node mem 195.6/858.0 GiB used (22.8%), avail 662.4 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 22x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:28:32 [loggers.py:259] Engine 000: Avg prompt throughput: 132.5 tokens/s, Avg generation throughput: 375.0 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 23.8%, Prefix cache hit rate: 90.5% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:28:37 [loggers.py:259] Engine 000: Avg prompt throughput: 86.3 tokens/s, Avg generation throughput: 428.9 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 24.2%, Prefix cache hit rate: 90.5% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:28:42 [loggers.py:259] Engine 000: Avg prompt throughput: 21.1 tokens/s, Avg generation throughput: 355.5 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 21.8%, Prefix cache hit rate: 90.6% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:28:47 [loggers.py:259] Engine 000: Avg prompt throughput: 261.5 tokens/s, Avg generation throughput: 327.9 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 22.0%, Prefix cache hit rate: 90.3% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 33x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) INFO 06-06 16:28:52 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 270.8 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 8.7%, Prefix cache hit rate: 90.1% [repeated 50x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) INFO 06-06 16:28:57 [loggers.py:259] Engine 000: Avg prompt throughput: 278.6 tokens/s, Avg generation throughput: 223.4 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 7.7%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) INFO 06-06 16:29:02 [loggers.py:259] Engine 000: Avg prompt throughput: 235.0 tokens/s, Avg generation throughput: 315.4 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 11.1%, Prefix cache hit rate: 89.7% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) INFO 06-06 16:29:07 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 370.9 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 10.3%, Prefix cache hit rate: 89.7% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) INFO 06-06 16:29:12 [loggers.py:259] Engine 000: Avg prompt throughput: 32.7 tokens/s, Avg generation throughput: 310.2 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 10.9%, Prefix cache hit rate: 89.7% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) INFO 06-06 16:29:17 [loggers.py:259] Engine 000: Avg prompt throughput: 45.1 tokens/s, Avg generation throughput: 307.0 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 11.4%, Prefix cache hit rate: 89.7% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) INFO 06-06 16:29:22 [loggers.py:259] Engine 000: Avg prompt throughput: 92.5 tokens/s, Avg generation throughput: 185.5 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 11.7%, Prefix cache hit rate: 87.1% [repeated 50x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) INFO 06-06 16:29:27 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 226.2 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 12.1%, Prefix cache hit rate: 87.1% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) INFO 06-06 16:29:32 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 228.0 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 12.4%, Prefix cache hit rate: 87.1% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) INFO 06-06 16:29:37 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 227.7 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 12.7%, Prefix cache hit rate: 87.1% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) INFO 06-06 16:29:42 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 227.8 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 13.0%, Prefix cache hit rate: 87.1% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) INFO 06-06 16:29:47 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 191.1 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 12.9%, Prefix cache hit rate: 87.1% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:29:49] OK: 409 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:29:49] OK: RSS 1.21 GiB | node mem 441.6/858.0 GiB used (51.5%), avail 416.3 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) INFO 06-06 16:29:52 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 181.1 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 13.1%, Prefix cache hit rate: 87.1% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:29:57] OK: 294 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:29:57] OK: RSS 1.19 GiB | node mem 439.9/858.0 GiB used (51.3%), avail 418.0 GiB (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) INFO 06-06 16:29:57 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 469.8 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.5%, Prefix cache hit rate: 87.6% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) INFO 06-06 16:30:02 [loggers.py:259] Engine 000: Avg prompt throughput: 111.6 tokens/s, Avg generation throughput: 463.2 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.3%, Prefix cache hit rate: 87.5% [repeated 49x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 36x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:30:06] OK: 367 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:30:06] OK: RSS 1.21 GiB | node mem 431.9/858.0 GiB used (50.3%), avail 426.1 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) INFO 06-06 16:30:07 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 178.7 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 13.9%, Prefix cache hit rate: 87.1% [repeated 47x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 22x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) INFO 06-06 16:30:12 [loggers.py:259] Engine 000: Avg prompt throughput: 73.1 tokens/s, Avg generation throughput: 221.3 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 14.4%, Prefix cache hit rate: 87.4% [repeated 47x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:30:14] OK: 311 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:30:14] OK: RSS 1.17 GiB | node mem 407.3/858.0 GiB used (47.5%), avail 450.7 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) INFO 06-06 16:30:17 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 503.3 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.4%, Prefix cache hit rate: 87.5% [repeated 50x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) INFO 06-06 16:30:22 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 494.6 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.7%, Prefix cache hit rate: 87.5% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 26x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) INFO 06-06 16:30:27 [loggers.py:259] Engine 000: Avg prompt throughput: 21.8 tokens/s, Avg generation throughput: 468.7 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 39.6%, Prefix cache hit rate: 87.7% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:30:29] OK: 464 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:30:29] OK: RSS 1.77 GiB | node mem 195.6/858.0 GiB used (22.8%), avail 662.3 GiB (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) INFO 06-06 16:30:32 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 502.1 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 40.3%, Prefix cache hit rate: 87.7% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) INFO 06-06 16:30:37 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 501.5 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 41.0%, Prefix cache hit rate: 87.7% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) INFO 06-06 16:30:43 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 502.4 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 41.7%, Prefix cache hit rate: 87.7% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 22x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) INFO 06-06 16:30:48 [loggers.py:259] Engine 000: Avg prompt throughput: 206.7 tokens/s, Avg generation throughput: 569.4 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 43.0%, Prefix cache hit rate: 87.5% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) INFO 06-06 16:30:53 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 569.3 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 43.3%, Prefix cache hit rate: 87.5% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) INFO 06-06 16:30:58 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 548.5 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 44.1%, Prefix cache hit rate: 87.5% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) INFO 06-06 16:31:03 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 234.7 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 23.8%, Prefix cache hit rate: 87.0% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) INFO 06-06 16:31:08 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 234.2 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 24.2%, Prefix cache hit rate: 87.0% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) INFO 06-06 16:31:13 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 234.8 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 24.5%, Prefix cache hit rate: 87.0% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 22x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) INFO 06-06 16:31:18 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 239.5 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 22.1%, Prefix cache hit rate: 88.0% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) INFO 06-06 16:31:23 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 239.9 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 22.4%, Prefix cache hit rate: 88.0% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) INFO 06-06 16:31:28 [loggers.py:259] Engine 000: Avg prompt throughput: 22.8 tokens/s, Avg generation throughput: 237.9 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 12.8%, Prefix cache hit rate: 87.4% [repeated 49x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) INFO 06-06 16:31:33 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 244.4 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 13.2%, Prefix cache hit rate: 87.4% [repeated 49x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) INFO 06-06 16:31:38 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 197.1 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 12.0%, Prefix cache hit rate: 87.4% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) INFO 06-06 16:31:43 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 320.7 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.6%, Prefix cache hit rate: 85.8% [repeated 49x across cluster] (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) INFO 06-06 16:31:48 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 320.6 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 30.0%, Prefix cache hit rate: 85.8% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:31:49] OK: 406 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:31:49] OK: RSS 1.21 GiB | node mem 441.6/858.0 GiB used (51.5%), avail 416.3 GiB (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) INFO 06-06 16:31:53 [loggers.py:259] Engine 000: Avg prompt throughput: 67.4 tokens/s, Avg generation throughput: 364.3 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 30.6%, Prefix cache hit rate: 85.7% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:31:57] OK: 299 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:31:57] OK: RSS 1.19 GiB | node mem 439.9/858.0 GiB used (51.3%), avail 418.0 GiB (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) INFO 06-06 16:31:58 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 337.8 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 30.9%, Prefix cache hit rate: 85.7% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) INFO 06-06 16:32:03 [loggers.py:259] Engine 000: Avg prompt throughput: 192.5 tokens/s, Avg generation throughput: 368.1 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.0%, Prefix cache hit rate: 86.2% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:32:06] OK: 385 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:32:06] OK: RSS 1.22 GiB | node mem 432.0/858.0 GiB used (50.3%), avail 426.0 GiB (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) INFO 06-06 16:32:08 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 409.4 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 33.7%, Prefix cache hit rate: 86.2% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) INFO 06-06 16:32:13 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 351.9 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.3%, Prefix cache hit rate: 86.2% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:32:14] OK: 313 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:32:14] OK: RSS 1.20 GiB | node mem 407.4/858.0 GiB used (47.5%), avail 450.6 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:32:18 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) INFO 06-06 16:32:19 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 362.6 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.1%, Prefix cache hit rate: 88.8% [repeated 49x across cluster] (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) INFO 06-06 16:32:24 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 362.4 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.6%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) INFO 06-06 16:32:29 [loggers.py:259] Engine 000: Avg prompt throughput: 102.4 tokens/s, Avg generation throughput: 367.8 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 45.1%, Prefix cache hit rate: 89.4% [repeated 48x across cluster] (skyrl_entrypoint pid=1407951) [fd-monitor] [16:32:29] OK: 471 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:32:29] OK: RSS 1.77 GiB | node mem 195.6/858.0 GiB used (22.8%), avail 662.3 GiB (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) INFO 06-06 16:32:34 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 407.1 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 45.7%, Prefix cache hit rate: 89.4% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) INFO 06-06 16:32:39 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 406.3 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 46.2%, Prefix cache hit rate: 89.4% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:32:44 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) INFO 06-06 16:32:43 [loggers.py:259] Engine 000: Avg prompt throughput: 113.1 tokens/s, Avg generation throughput: 328.8 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.5%, Prefix cache hit rate: 88.3% [repeated 47x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:32:44 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) INFO 06-06 16:32:49 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 187.6 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 13.2%, Prefix cache hit rate: 89.2% [repeated 50x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) INFO 06-06 16:32:54 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 187.4 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 13.5%, Prefix cache hit rate: 89.2% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:32:54 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) INFO 06-06 16:32:59 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 187.7 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 13.8%, Prefix cache hit rate: 89.2% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:33:04 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) INFO 06-06 16:33:04 [loggers.py:259] Engine 000: Avg prompt throughput: 35.4 tokens/s, Avg generation throughput: 446.6 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 49.4%, Prefix cache hit rate: 89.3% [repeated 47x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) INFO 06-06 16:33:09 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 187.6 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 14.3%, Prefix cache hit rate: 89.2% [repeated 49x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 14x across cluster] (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) INFO 06-06 16:33:14 [loggers.py:259] Engine 000: Avg prompt throughput: 86.1 tokens/s, Avg generation throughput: 211.6 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 14.7%, Prefix cache hit rate: 89.1% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) INFO 06-06 16:33:19 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 211.4 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 14.8%, Prefix cache hit rate: 89.1% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) INFO 06-06 16:33:24 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 450.6 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 51.9%, Prefix cache hit rate: 89.3% [repeated 47x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:33:26 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) INFO 06-06 16:33:29 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 280.1 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 16.6%, Prefix cache hit rate: 89.2% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:33:27 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) INFO 06-06 16:33:34 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 280.8 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 17.0%, Prefix cache hit rate: 89.2% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:33:38 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) INFO 06-06 16:33:39 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 275.1 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 16.3%, Prefix cache hit rate: 89.2% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:33:40 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) INFO 06-06 16:33:44 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 233.8 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 16.7%, Prefix cache hit rate: 89.2% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:46 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:33:49] OK: 383 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:33:49] OK: RSS 1.22 GiB | node mem 441.6/858.0 GiB used (51.5%), avail 416.3 GiB (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 31x across cluster] (AsyncVLLMInferenceEngine pid=1420132, ip=10.128.16.41) INFO 06-06 16:33:49 [loggers.py:259] Engine 000: Avg prompt throughput: 9.4 tokens/s, Avg generation throughput: 248.9 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 24.3%, Prefix cache hit rate: 90.3% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:33:51 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) INFO 06-06 16:33:54 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 401.5 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.0%, Prefix cache hit rate: 85.5% [repeated 49x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 8x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 8x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:33:57] OK: 466 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:33:57] OK: RSS 1.21 GiB | node mem 440.0/858.0 GiB used (51.3%), avail 418.0 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 38x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) INFO 06-06 16:33:59 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 320.7 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.4%, Prefix cache hit rate: 85.5% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) INFO 06-06 16:34:04 [loggers.py:259] Engine 000: Avg prompt throughput: 72.3 tokens/s, Avg generation throughput: 353.9 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.0%, Prefix cache hit rate: 85.4% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:34:06] OK: 363 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:34:06] OK: RSS 1.23 GiB | node mem 432.0/858.0 GiB used (50.4%), avail 425.9 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) INFO 06-06 16:34:09 [loggers.py:259] Engine 000: Avg prompt throughput: 183.9 tokens/s, Avg generation throughput: 451.2 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.9%, Prefix cache hit rate: 85.3% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1378319, ip=10.128.16.37) ERROR 06-06 16:34:13 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:34:14] OK: 376 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:34:14] OK: RSS 1.21 GiB | node mem 407.4/858.0 GiB used (47.5%), avail 450.6 GiB (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) INFO 06-06 16:34:14 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 377.2 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.7%, Prefix cache hit rate: 85.3% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:34:17 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) INFO 06-06 16:34:19 [loggers.py:259] Engine 000: Avg prompt throughput: 36.0 tokens/s, Avg generation throughput: 351.9 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 39.2%, Prefix cache hit rate: 85.3% [repeated 49x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) INFO 06-06 16:34:24 [loggers.py:259] Engine 000: Avg prompt throughput: 74.6 tokens/s, Avg generation throughput: 267.9 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 16.0%, Prefix cache hit rate: 85.4% [repeated 49x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:34:25 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:34:29] OK: 564 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:34:29] OK: RSS 1.78 GiB | node mem 195.7/858.0 GiB used (22.8%), avail 662.2 GiB (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) INFO 06-06 16:34:29 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 287.1 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 16.4%, Prefix cache hit rate: 85.4% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:34:26 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) INFO 06-06 16:34:34 [loggers.py:259] Engine 000: Avg prompt throughput: 108.9 tokens/s, Avg generation throughput: 319.9 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 18.6%, Prefix cache hit rate: 85.7% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1446016, ip=10.128.16.46) ERROR 06-06 16:34:37 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) INFO 06-06 16:34:40 [loggers.py:259] Engine 000: Avg prompt throughput: 71.5 tokens/s, Avg generation throughput: 410.9 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 20.2%, Prefix cache hit rate: 85.8% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:41 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 38x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) INFO 06-06 16:34:45 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 430.8 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 20.8%, Prefix cache hit rate: 85.8% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:34:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 26x across cluster] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) INFO 06-06 16:34:50 [loggers.py:259] Engine 000: Avg prompt throughput: 45.4 tokens/s, Avg generation throughput: 424.3 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 20.0%, Prefix cache hit rate: 86.1% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:34:49 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 16x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) INFO 06-06 16:34:55 [loggers.py:259] Engine 000: Avg prompt throughput: 69.1 tokens/s, Avg generation throughput: 396.7 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 20.7%, Prefix cache hit rate: 86.0% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:34:58 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) INFO 06-06 16:35:00 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 414.9 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 20.5%, Prefix cache hit rate: 86.0% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] Error in preprocessing prompt inputs [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] Traceback (most recent call last): [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] return await asyncio.gather( [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] raise VLLMValidationError( [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:01 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 5x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) INFO 06-06 16:35:05 [loggers.py:259] Engine 000: Avg prompt throughput: 523.2 tokens/s, Avg generation throughput: 396.0 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 24.7%, Prefix cache hit rate: 85.9% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] Error in preprocessing prompt inputs [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] Traceback (most recent call last): [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] return await asyncio.gather( [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] raise VLLMValidationError( [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:35:04 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 5x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 8x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 8x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) INFO 06-06 16:35:10 [loggers.py:259] Engine 000: Avg prompt throughput: 119.0 tokens/s, Avg generation throughput: 551.2 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 25.7%, Prefix cache hit rate: 85.9% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:35:11 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) INFO 06-06 16:35:15 [loggers.py:259] Engine 000: Avg prompt throughput: 26.7 tokens/s, Avg generation throughput: 491.7 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 44.5%, Prefix cache hit rate: 89.7% [repeated 49x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1378449, ip=10.128.16.37) ERROR 06-06 16:35:16 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 33x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) INFO 06-06 16:35:20 [loggers.py:259] Engine 000: Avg prompt throughput: 12.2 tokens/s, Avg generation throughput: 426.6 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 46.2%, Prefix cache hit rate: 89.7% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) INFO 06-06 16:35:25 [loggers.py:259] Engine 000: Avg prompt throughput: 27.1 tokens/s, Avg generation throughput: 471.4 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 46.6%, Prefix cache hit rate: 89.8% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 33x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:35:30 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) INFO 06-06 16:35:30 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 465.7 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 47.3%, Prefix cache hit rate: 89.8% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 30x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:35 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) INFO 06-06 16:35:35 [loggers.py:259] Engine 000: Avg prompt throughput: 96.6 tokens/s, Avg generation throughput: 441.5 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 46.9%, Prefix cache hit rate: 89.8% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:35:36 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) INFO 06-06 16:35:40 [loggers.py:259] Engine 000: Avg prompt throughput: 41.0 tokens/s, Avg generation throughput: 436.2 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 47.6%, Prefix cache hit rate: 89.8% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) ERROR 06-06 16:35:40 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) INFO 06-06 16:35:45 [loggers.py:259] Engine 000: Avg prompt throughput: 105.8 tokens/s, Avg generation throughput: 423.1 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 49.2%, Prefix cache hit rate: 89.9% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:35:49] OK: 365 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:35:49] OK: RSS 1.23 GiB | node mem 441.6/858.0 GiB used (51.5%), avail 416.3 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:35:50 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) INFO 06-06 16:35:50 [loggers.py:259] Engine 000: Avg prompt throughput: 21.0 tokens/s, Avg generation throughput: 464.6 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 53.0%, Prefix cache hit rate: 90.0% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] Error in preprocessing prompt inputs [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] Traceback (most recent call last): [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] return await asyncio.gather( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] raise VLLMValidationError( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:52 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 3x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) INFO 06-06 16:35:55 [loggers.py:259] Engine 000: Avg prompt throughput: 112.0 tokens/s, Avg generation throughput: 564.5 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 56.0%, Prefix cache hit rate: 90.2% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:35:57] OK: 405 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:35:57] OK: RSS 1.22 GiB | node mem 440.0/858.0 GiB used (51.3%), avail 417.9 GiB (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 19x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:35:57 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) INFO 06-06 16:36:00 [loggers.py:259] Engine 000: Avg prompt throughput: 27.3 tokens/s, Avg generation throughput: 445.8 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 30.9%, Prefix cache hit rate: 85.6% [repeated 47x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 39x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:36:05 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 440.9 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 51.7%, Prefix cache hit rate: 87.3% [repeated 49x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:36:06] OK: 359 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:36:06] OK: RSS 1.23 GiB | node mem 432.1/858.0 GiB used (50.4%), avail 425.9 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1427133, ip=10.128.16.36) ERROR 06-06 16:36:09 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 29x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:36:10 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 439.0 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 52.3%, Prefix cache hit rate: 87.3% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:36:14] OK: 375 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:36:14] OK: RSS 1.21 GiB | node mem 407.4/858.0 GiB used (47.5%), avail 450.6 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 36x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:36:15 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 439.2 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 52.9%, Prefix cache hit rate: 87.3% [repeated 49x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:36:20 [loggers.py:259] Engine 000: Avg prompt throughput: 79.2 tokens/s, Avg generation throughput: 447.8 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 53.7%, Prefix cache hit rate: 87.2% [repeated 47x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:36:25 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 467.6 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 45.2%, Prefix cache hit rate: 87.2% [repeated 49x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:36:29] OK: 564 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:36:29] OK: RSS 1.80 GiB | node mem 195.7/858.0 GiB used (22.8%), avail 662.2 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:36:30 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 393.8 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 45.7%, Prefix cache hit rate: 87.2% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:36:35 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 392.2 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 46.3%, Prefix cache hit rate: 87.2% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:36:40 [loggers.py:259] Engine 000: Avg prompt throughput: 104.0 tokens/s, Avg generation throughput: 426.7 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 47.0%, Prefix cache hit rate: 87.1% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:36:41 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:36:45 [loggers.py:259] Engine 000: Avg prompt throughput: 113.7 tokens/s, Avg generation throughput: 442.1 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 47.9%, Prefix cache hit rate: 87.0% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:36:41 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:36:50 [loggers.py:259] Engine 000: Avg prompt throughput: 64.4 tokens/s, Avg generation throughput: 481.9 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 48.6%, Prefix cache hit rate: 86.9% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:36:53 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:36:55 [loggers.py:259] Engine 000: Avg prompt throughput: 10.6 tokens/s, Avg generation throughput: 490.7 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 49.2%, Prefix cache hit rate: 87.0% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:36:57 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:36:58 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:37:00 [loggers.py:259] Engine 000: Avg prompt throughput: 10.2 tokens/s, Avg generation throughput: 495.5 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 50.0%, Prefix cache hit rate: 87.0% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 21x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:36:59 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:37:05 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:37:05 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 450.2 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 41.1%, Prefix cache hit rate: 87.0% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 14x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:37:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:37:08 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:37:10 [loggers.py:259] Engine 000: Avg prompt throughput: 3.2 tokens/s, Avg generation throughput: 367.1 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.2%, Prefix cache hit rate: 87.0% [repeated 49x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 6x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 6x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:37:12 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:37:15 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 346.8 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.7%, Prefix cache hit rate: 87.0% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:37:20 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 299.1 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.9%, Prefix cache hit rate: 87.0% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:37:21 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:37:25 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 295.5 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 33.3%, Prefix cache hit rate: 87.0% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:25 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:28 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) ERROR 06-06 16:37:29 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:37:30 [loggers.py:259] Engine 000: Avg prompt throughput: 34.6 tokens/s, Avg generation throughput: 341.0 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.0%, Prefix cache hit rate: 87.0% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 6x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:37:35 [loggers.py:259] Engine 000: Avg prompt throughput: 11.6 tokens/s, Avg generation throughput: 401.2 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.7%, Prefix cache hit rate: 87.1% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:37:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:37:40 [loggers.py:259] Engine 000: Avg prompt throughput: 48.3 tokens/s, Avg generation throughput: 434.9 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.0%, Prefix cache hit rate: 87.1% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:37:45 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 360.9 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 26.6%, Prefix cache hit rate: 87.1% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:37:47 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:37:49] OK: 367 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:37:49] OK: RSS 1.24 GiB | node mem 441.7/858.0 GiB used (51.5%), avail 416.3 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) INFO 06-06 16:37:50 [loggers.py:259] Engine 000: Avg prompt throughput: 46.3 tokens/s, Avg generation throughput: 358.1 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 21.0%, Prefix cache hit rate: 87.9% [repeated 47x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:50 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:37:53 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) INFO 06-06 16:37:55 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 327.4 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 21.5%, Prefix cache hit rate: 87.9% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:37:57] OK: 366 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:37:57] OK: RSS 1.22 GiB | node mem 440.1/858.0 GiB used (51.3%), avail 417.9 GiB (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:37:54 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) ERROR 06-06 16:37:59 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:38:00 [loggers.py:259] Engine 000: Avg prompt throughput: 236.9 tokens/s, Avg generation throughput: 385.3 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.2%, Prefix cache hit rate: 86.9% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 26x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:37:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 6x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] Error in preprocessing prompt inputs [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] Traceback (most recent call last): [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] return await asyncio.gather( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] raise VLLMValidationError( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 3x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:38:05 [loggers.py:259] Engine 000: Avg prompt throughput: 34.9 tokens/s, Avg generation throughput: 407.6 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 30.0%, Prefix cache hit rate: 86.9% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:38:06] OK: 358 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:38:06] OK: RSS 1.24 GiB | node mem 432.1/858.0 GiB used (50.4%), avail 425.9 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:38:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:38:10 [loggers.py:259] Engine 000: Avg prompt throughput: 130.3 tokens/s, Avg generation throughput: 472.6 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.1%, Prefix cache hit rate: 86.8% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:38:14] OK: 367 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:38:14] OK: RSS 1.22 GiB | node mem 407.4/858.0 GiB used (47.5%), avail 450.6 GiB (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:38:15 [loggers.py:259] Engine 000: Avg prompt throughput: 45.3 tokens/s, Avg generation throughput: 488.8 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.6%, Prefix cache hit rate: 86.8% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:38:19 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:38:20 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 492.4 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.3%, Prefix cache hit rate: 86.8% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:38:25 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:38:25 [loggers.py:259] Engine 000: Avg prompt throughput: 43.3 tokens/s, Avg generation throughput: 472.9 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.7%, Prefix cache hit rate: 86.8% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:38:29] OK: 563 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:38:29] OK: RSS 1.80 GiB | node mem 195.7/858.0 GiB used (22.8%), avail 662.2 GiB (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:38:30 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 443.6 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.3%, Prefix cache hit rate: 86.8% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:38:32 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) INFO 06-06 16:38:36 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 441.4 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.9%, Prefix cache hit rate: 86.8% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=2121841, ip=10.128.16.185) INFO 06-06 16:38:41 [loggers.py:259] Engine 000: Avg prompt throughput: 34.9 tokens/s, Avg generation throughput: 222.5 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 9.6%, Prefix cache hit rate: 91.1% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 16:38:45 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) INFO 06-06 16:38:46 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 264.1 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.5%, Prefix cache hit rate: 88.1% [repeated 49x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:38:48 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) INFO 06-06 16:38:51 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 264.5 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.9%, Prefix cache hit rate: 88.1% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) INFO 06-06 16:38:56 [loggers.py:259] Engine 000: Avg prompt throughput: 135.9 tokens/s, Avg generation throughput: 302.8 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.0%, Prefix cache hit rate: 88.2% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 19x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:39:01 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) INFO 06-06 16:39:01 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 334.8 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.5%, Prefix cache hit rate: 88.2% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 27x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:39:03 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) INFO 06-06 16:39:06 [loggers.py:259] Engine 000: Avg prompt throughput: 41.8 tokens/s, Avg generation throughput: 321.5 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 27.7%, Prefix cache hit rate: 88.4% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:39:08 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) INFO 06-06 16:39:11 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 348.6 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 28.2%, Prefix cache hit rate: 88.4% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) ERROR 06-06 16:39:09 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) INFO 06-06 16:39:16 [loggers.py:259] Engine 000: Avg prompt throughput: 146.9 tokens/s, Avg generation throughput: 348.5 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.6%, Prefix cache hit rate: 88.3% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 11x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) INFO 06-06 16:39:21 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 393.4 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 30.2%, Prefix cache hit rate: 88.3% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:39:23 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) INFO 06-06 16:39:26 [loggers.py:259] Engine 000: Avg prompt throughput: 118.2 tokens/s, Avg generation throughput: 638.1 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 59.9%, Prefix cache hit rate: 88.8% [repeated 49x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) INFO 06-06 16:39:31 [loggers.py:259] Engine 000: Avg prompt throughput: 75.3 tokens/s, Avg generation throughput: 657.2 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 57.6%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 6x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) INFO 06-06 16:39:36 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 442.2 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 51.8%, Prefix cache hit rate: 88.7% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 14x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) INFO 06-06 16:39:41 [loggers.py:259] Engine 000: Avg prompt throughput: 318.8 tokens/s, Avg generation throughput: 478.4 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 57.2%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) INFO 06-06 16:39:46 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 485.0 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 57.8%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:39:49] OK: 358 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:39:49] OK: RSS 1.26 GiB | node mem 441.7/858.0 GiB used (51.5%), avail 416.2 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) INFO 06-06 16:39:51 [loggers.py:259] Engine 000: Avg prompt throughput: 56.9 tokens/s, Avg generation throughput: 502.0 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 59.0%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) INFO 06-06 16:39:56 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 529.3 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 59.7%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:39:57] OK: 368 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:39:57] OK: RSS 1.23 GiB | node mem 440.1/858.0 GiB used (51.3%), avail 417.9 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) INFO 06-06 16:40:01 [loggers.py:259] Engine 000: Avg prompt throughput: 53.4 tokens/s, Avg generation throughput: 505.0 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 55.3%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 44x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 16:40:04 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:40:06] OK: 365 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:40:06] OK: RSS 1.24 GiB | node mem 432.2/858.0 GiB used (50.4%), avail 425.8 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) INFO 06-06 16:40:06 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 474.3 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 55.5%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 50x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:40:11 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) INFO 06-06 16:40:11 [loggers.py:259] Engine 000: Avg prompt throughput: 178.4 tokens/s, Avg generation throughput: 442.7 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 39.2%, Prefix cache hit rate: 88.9% [repeated 47x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:40:14] OK: 365 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:40:14] OK: RSS 1.22 GiB | node mem 407.4/858.0 GiB used (47.5%), avail 450.6 GiB (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) INFO 06-06 16:40:16 [loggers.py:259] Engine 000: Avg prompt throughput: 435.1 tokens/s, Avg generation throughput: 465.8 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 57.6%, Prefix cache hit rate: 88.4% [repeated 49x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 29x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:40:19 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) INFO 06-06 16:40:21 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 527.9 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 58.4%, Prefix cache hit rate: 88.4% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:21 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) INFO 06-06 16:40:26 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 529.6 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 59.1%, Prefix cache hit rate: 88.4% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (skyrl_entrypoint pid=1407951) [fd-monitor] [16:40:29] OK: 564 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:40:29] OK: RSS 1.82 GiB | node mem 195.8/858.0 GiB used (22.8%), avail 662.2 GiB (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:40:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) INFO 06-06 16:40:32 [loggers.py:259] Engine 000: Avg prompt throughput: 86.0 tokens/s, Avg generation throughput: 468.1 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 30.7%, Prefix cache hit rate: 88.6% [repeated 49x across cluster] (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:40:29 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 16:40:36 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) INFO 06-06 16:40:37 [loggers.py:259] Engine 000: Avg prompt throughput: 163.0 tokens/s, Avg generation throughput: 478.4 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.8%, Prefix cache hit rate: 88.6% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:40:37 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) INFO 06-06 16:40:42 [loggers.py:259] Engine 000: Avg prompt throughput: 200.0 tokens/s, Avg generation throughput: 546.9 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.7%, Prefix cache hit rate: 88.5% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:40:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) INFO 06-06 16:40:47 [loggers.py:259] Engine 000: Avg prompt throughput: 191.3 tokens/s, Avg generation throughput: 560.0 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 36.7%, Prefix cache hit rate: 88.3% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) INFO 06-06 16:40:52 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 596.3 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.6%, Prefix cache hit rate: 88.3% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) INFO 06-06 16:40:57 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 547.9 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.7%, Prefix cache hit rate: 88.3% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1089006, ip=10.128.16.43) ERROR 06-06 16:41:01 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) INFO 06-06 16:41:01 [loggers.py:259] Engine 000: Avg prompt throughput: 2.2 tokens/s, Avg generation throughput: 687.9 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 63.9%, Prefix cache hit rate: 88.9% [repeated 47x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 16:41:03 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) INFO 06-06 16:41:07 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 461.0 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.6%, Prefix cache hit rate: 88.3% [repeated 49x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) INFO 06-06 16:41:12 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 445.9 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.6%, Prefix cache hit rate: 88.3% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) INFO 06-06 16:41:17 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 413.5 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.2%, Prefix cache hit rate: 88.3% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 21x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) INFO 06-06 16:41:22 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 414.6 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.8%, Prefix cache hit rate: 88.3% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:41:23 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) INFO 06-06 16:41:27 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 414.5 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 36.4%, Prefix cache hit rate: 88.3% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 16:41:26 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) INFO 06-06 16:41:32 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 414.1 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.0%, Prefix cache hit rate: 88.3% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) INFO 06-06 16:41:37 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 414.1 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.5%, Prefix cache hit rate: 88.3% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:41:37 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) INFO 06-06 16:41:42 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 414.7 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.1%, Prefix cache hit rate: 88.3% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) INFO 06-06 16:41:47 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 415.2 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.7%, Prefix cache hit rate: 88.3% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 16:41:48 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:41:49] OK: 358 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:41:49] OK: RSS 1.27 GiB | node mem 441.8/858.0 GiB used (51.5%), avail 416.2 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) INFO 06-06 16:41:52 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 414.8 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 39.3%, Prefix cache hit rate: 88.3% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1089004, ip=10.128.16.43) ERROR 06-06 16:41:53 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) INFO 06-06 16:41:57 [loggers.py:259] Engine 000: Avg prompt throughput: 150.7 tokens/s, Avg generation throughput: 446.5 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 42.8%, Prefix cache hit rate: 88.5% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:41:57] OK: 366 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:41:57] OK: RSS 1.23 GiB | node mem 440.1/858.0 GiB used (51.3%), avail 417.9 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 30x across cluster] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) INFO 06-06 16:42:02 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 506.7 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 43.5%, Prefix cache hit rate: 88.5% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:42:06] OK: 362 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:42:06] OK: RSS 1.25 GiB | node mem 432.2/858.0 GiB used (50.4%), avail 425.8 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 34x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) INFO 06-06 16:42:07 [loggers.py:259] Engine 000: Avg prompt throughput: 178.8 tokens/s, Avg generation throughput: 509.4 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 43.9%, Prefix cache hit rate: 88.5% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 19x across cluster] (AsyncVLLMInferenceEngine pid=1446015, ip=10.128.16.46) INFO 06-06 16:42:12 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 389.7 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 26.8%, Prefix cache hit rate: 92.2% [repeated 49x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:42:14] OK: 367 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:42:14] OK: RSS 1.23 GiB | node mem 407.5/858.0 GiB used (47.5%), avail 450.5 GiB (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 29x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1446015, ip=10.128.16.46) INFO 06-06 16:42:17 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 390.5 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 27.4%, Prefix cache hit rate: 92.2% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 57x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1446015, ip=10.128.16.46) INFO 06-06 16:42:22 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 394.7 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 27.9%, Prefix cache hit rate: 92.2% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1426879, ip=10.128.16.36) ERROR 06-06 16:42:25 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1446015, ip=10.128.16.46) INFO 06-06 16:42:27 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 391.7 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 28.5%, Prefix cache hit rate: 92.2% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:42:29] OK: 564 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:42:29] OK: RSS 1.82 GiB | node mem 195.9/858.0 GiB used (22.8%), avail 662.1 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1446015, ip=10.128.16.46) INFO 06-06 16:42:32 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 389.7 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.0%, Prefix cache hit rate: 92.2% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1446015, ip=10.128.16.46) INFO 06-06 16:42:37 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 390.3 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.5%, Prefix cache hit rate: 92.2% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 16x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1446015, ip=10.128.16.46) INFO 06-06 16:42:42 [loggers.py:259] Engine 000: Avg prompt throughput: 40.9 tokens/s, Avg generation throughput: 401.4 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.4%, Prefix cache hit rate: 92.2% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1446015, ip=10.128.16.46) INFO 06-06 16:42:47 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 433.3 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.0%, Prefix cache hit rate: 92.2% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) INFO 06-06 16:42:52 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 459.6 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 55.4%, Prefix cache hit rate: 88.3% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1449219, ip=10.128.16.42) ERROR 06-06 16:42:56 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) INFO 06-06 16:42:57 [loggers.py:259] Engine 000: Avg prompt throughput: 174.0 tokens/s, Avg generation throughput: 460.1 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 57.5%, Prefix cache hit rate: 88.2% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) INFO 06-06 16:43:02 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 505.4 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 58.2%, Prefix cache hit rate: 88.2% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:43:05 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) INFO 06-06 16:43:07 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 479.1 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 57.3%, Prefix cache hit rate: 88.2% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 4x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) INFO 06-06 16:43:12 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 460.8 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 58.0%, Prefix cache hit rate: 88.2% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1378447, ip=10.128.16.37) ERROR 06-06 16:43:14 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) INFO 06-06 16:43:17 [loggers.py:259] Engine 000: Avg prompt throughput: 117.3 tokens/s, Avg generation throughput: 468.1 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 61.9%, Prefix cache hit rate: 88.5% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 16x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) INFO 06-06 16:43:22 [loggers.py:259] Engine 000: Avg prompt throughput: 89.4 tokens/s, Avg generation throughput: 491.6 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 59.4%, Prefix cache hit rate: 88.4% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:43:24 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 14x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) INFO 06-06 16:43:27 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 507.4 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 60.1%, Prefix cache hit rate: 88.4% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) INFO 06-06 16:43:32 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 506.9 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 60.8%, Prefix cache hit rate: 88.4% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) ERROR 06-06 16:43:35 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) INFO 06-06 16:43:37 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 506.3 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 61.5%, Prefix cache hit rate: 88.4% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2121843, ip=10.128.16.185) ERROR 06-06 16:43:36 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) INFO 06-06 16:43:42 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 505.9 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 62.2%, Prefix cache hit rate: 88.4% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) INFO 06-06 16:43:47 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 505.7 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 62.9%, Prefix cache hit rate: 88.4% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:43:49] OK: 365 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:43:49] OK: RSS 1.27 GiB | node mem 441.8/858.0 GiB used (51.5%), avail 416.1 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) INFO 06-06 16:43:52 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 505.8 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 63.6%, Prefix cache hit rate: 88.4% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:43:53 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 11x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 11x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:43:57] OK: 364 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:43:57] OK: RSS 1.23 GiB | node mem 440.1/858.0 GiB used (51.3%), avail 417.9 GiB (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) INFO 06-06 16:43:57 [loggers.py:259] Engine 000: Avg prompt throughput: 211.1 tokens/s, Avg generation throughput: 537.7 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 66.6%, Prefix cache hit rate: 88.4% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:43:55 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 16x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) INFO 06-06 16:44:02 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 598.4 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 67.5%, Prefix cache hit rate: 88.4% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:44:06] OK: 363 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:44:06] OK: RSS 1.26 GiB | node mem 432.2/858.0 GiB used (50.4%), avail 425.8 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) INFO 06-06 16:44:07 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 574.1 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 67.7%, Prefix cache hit rate: 88.4% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 16:44:12 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) INFO 06-06 16:44:12 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 552.9 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 68.4%, Prefix cache hit rate: 88.4% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:44:14] OK: 360 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:44:14] OK: RSS 1.24 GiB | node mem 407.5/858.0 GiB used (47.5%), avail 450.5 GiB (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) INFO 06-06 16:44:18 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 272.3 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.1%, Prefix cache hit rate: 88.8% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) INFO 06-06 16:44:23 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 272.8 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.5%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:26 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) INFO 06-06 16:44:28 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 272.7 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.9%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:44:29] OK: 564 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:44:29] OK: RSS 1.83 GiB | node mem 195.9/858.0 GiB used (22.8%), avail 662.0 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 8x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 8x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] Error in preprocessing prompt inputs [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] Traceback (most recent call last): [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] return await asyncio.gather( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] raise VLLMValidationError( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:44:30 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 3x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) INFO 06-06 16:44:33 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 272.3 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 28.9%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 7x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) ERROR 06-06 16:44:36 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) INFO 06-06 16:44:38 [loggers.py:259] Engine 000: Avg prompt throughput: 72.0 tokens/s, Avg generation throughput: 266.8 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.1%, Prefix cache hit rate: 88.9% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 31x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455862, ip=10.128.16.48) INFO 06-06 16:44:42 [loggers.py:259] Engine 000: Avg prompt throughput: 79.5 tokens/s, Avg generation throughput: 518.1 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 72.6%, Prefix cache hit rate: 88.7% [repeated 47x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) INFO 06-06 16:44:48 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 272.9 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.8%, Prefix cache hit rate: 88.9% [repeated 49x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) INFO 06-06 16:44:53 [loggers.py:259] Engine 000: Avg prompt throughput: 78.7 tokens/s, Avg generation throughput: 294.7 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.4%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 16:44:53 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 19x across cluster] (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) INFO 06-06 16:44:58 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 317.6 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.8%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 27x across cluster] (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) INFO 06-06 16:45:03 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 318.7 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 33.3%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 74x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) INFO 06-06 16:45:08 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 318.1 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 33.7%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 58x across cluster] (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) INFO 06-06 16:45:13 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 317.8 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.1%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:45:15 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 58x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:45:18 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 215.9 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 16.8%, Prefix cache hit rate: 93.8% [repeated 49x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:45:17 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 45x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:45:23 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 192.9 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 17.1%, Prefix cache hit rate: 93.8% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 16:45:28 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:45:28 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 192.8 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 17.3%, Prefix cache hit rate: 93.8% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:45:33 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 193.0 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 17.6%, Prefix cache hit rate: 93.8% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:45:38 [loggers.py:259] Engine 000: Avg prompt throughput: 141.6 tokens/s, Avg generation throughput: 228.5 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 21.7%, Prefix cache hit rate: 93.8% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:45:43 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 207.0 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 13.1%, Prefix cache hit rate: 93.8% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:45:44 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:45:48 [loggers.py:259] Engine 000: Avg prompt throughput: 41.1 tokens/s, Avg generation throughput: 229.8 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 14.1%, Prefix cache hit rate: 93.8% [repeated 49x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:45:49] OK: 364 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:45:49] OK: RSS 1.27 GiB | node mem 443.6/858.0 GiB used (51.7%), avail 414.4 GiB (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:45:52 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:45:53 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 195.4 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 10.4%, Prefix cache hit rate: 93.8% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:45:57] OK: 365 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:45:57] OK: RSS 1.24 GiB | node mem 440.1/858.0 GiB used (51.3%), avail 417.8 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:45:58 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 193.4 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 10.7%, Prefix cache hit rate: 93.8% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 16:45:58 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:46:03 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 193.7 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 11.0%, Prefix cache hit rate: 93.8% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 16:46:00 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:46:06] OK: 359 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:46:06] OK: RSS 1.26 GiB | node mem 432.2/858.0 GiB used (50.4%), avail 425.8 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:46:08 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 193.7 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 11.2%, Prefix cache hit rate: 93.8% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:46:13 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 193.8 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 11.5%, Prefix cache hit rate: 93.8% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:46:14] OK: 369 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:46:14] OK: RSS 1.25 GiB | node mem 407.5/858.0 GiB used (47.5%), avail 450.5 GiB (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1089005, ip=10.128.16.43) ERROR 06-06 16:46:17 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 19x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:46:18 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 193.8 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 11.8%, Prefix cache hit rate: 93.8% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:46:23 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 193.0 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 12.0%, Prefix cache hit rate: 93.8% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:46:28 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 194.0 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 12.3%, Prefix cache hit rate: 93.8% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:46:29] OK: 564 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:46:29] OK: RSS 1.84 GiB | node mem 196.0/858.0 GiB used (22.8%), avail 662.0 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:46:33 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 193.3 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 12.6%, Prefix cache hit rate: 93.8% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) INFO 06-06 16:46:38 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 193.5 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 12.9%, Prefix cache hit rate: 93.8% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) INFO 06-06 16:46:43 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 458.8 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.0%, Prefix cache hit rate: 87.7% [repeated 49x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) INFO 06-06 16:46:48 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 457.9 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.6%, Prefix cache hit rate: 87.7% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:46:49 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) INFO 06-06 16:46:53 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 458.5 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.3%, Prefix cache hit rate: 87.7% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) INFO 06-06 16:46:58 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 458.1 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.9%, Prefix cache hit rate: 87.7% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) INFO 06-06 16:47:03 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 459.4 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 36.5%, Prefix cache hit rate: 87.7% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) INFO 06-06 16:47:08 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 457.7 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.2%, Prefix cache hit rate: 87.7% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) INFO 06-06 16:47:13 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 458.2 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.8%, Prefix cache hit rate: 87.7% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:47:14 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1420131, ip=10.128.16.41) INFO 06-06 16:47:18 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 459.3 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.5%, Prefix cache hit rate: 87.7% [repeated 49x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) INFO 06-06 16:47:23 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 631.8 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 75.1%, Prefix cache hit rate: 90.7% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) INFO 06-06 16:47:28 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 615.0 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 75.0%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 16:47:33 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) INFO 06-06 16:47:33 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 579.2 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 75.9%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) INFO 06-06 16:47:38 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 574.5 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 76.7%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:47:40 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) INFO 06-06 16:47:43 [loggers.py:259] Engine 000: Avg prompt throughput: 21.8 tokens/s, Avg generation throughput: 618.3 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 77.7%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) INFO 06-06 16:47:48 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 642.5 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 78.3%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:47:49] OK: 387 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:47:49] OK: RSS 1.27 GiB | node mem 443.6/858.0 GiB used (51.7%), avail 414.4 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) INFO 06-06 16:47:54 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 603.0 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 70.2%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:47:57] OK: 366 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:47:57] OK: RSS 1.24 GiB | node mem 440.1/858.0 GiB used (51.3%), avail 417.9 GiB (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) INFO 06-06 16:47:59 [loggers.py:259] Engine 000: Avg prompt throughput: 90.8 tokens/s, Avg generation throughput: 625.4 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 73.3%, Prefix cache hit rate: 90.8% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) INFO 06-06 16:48:04 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 502.4 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 59.2%, Prefix cache hit rate: 93.7% [repeated 49x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:48:06] OK: 351 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:48:06] OK: RSS 1.26 GiB | node mem 432.2/858.0 GiB used (50.4%), avail 425.8 GiB (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:48:06 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) INFO 06-06 16:48:09 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 503.4 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 59.9%, Prefix cache hit rate: 93.7% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) INFO 06-06 16:48:14 [loggers.py:259] Engine 000: Avg prompt throughput: 518.3 tokens/s, Avg generation throughput: 491.9 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 64.2%, Prefix cache hit rate: 93.6% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:48:14] OK: 355 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:48:14] OK: RSS 1.25 GiB | node mem 407.5/858.0 GiB used (47.5%), avail 450.5 GiB (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) INFO 06-06 16:48:19 [loggers.py:259] Engine 000: Avg prompt throughput: 45.4 tokens/s, Avg generation throughput: 500.7 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 61.1%, Prefix cache hit rate: 93.6% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) INFO 06-06 16:48:24 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 499.6 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 61.8%, Prefix cache hit rate: 93.6% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) INFO 06-06 16:48:29 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 481.9 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 53.6%, Prefix cache hit rate: 93.6% [repeated 48x across cluster] (skyrl_entrypoint pid=1407951) [fd-monitor] [16:48:29] OK: 564 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:48:29] OK: RSS 1.85 GiB | node mem 196.0/858.0 GiB used (22.8%), avail 662.0 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) INFO 06-06 16:48:34 [loggers.py:259] Engine 000: Avg prompt throughput: 13.4 tokens/s, Avg generation throughput: 517.5 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 55.5%, Prefix cache hit rate: 93.6% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) INFO 06-06 16:48:39 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 545.4 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 56.2%, Prefix cache hit rate: 93.6% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:48:41 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 11x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) INFO 06-06 16:48:44 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 502.3 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 56.3%, Prefix cache hit rate: 93.6% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) INFO 06-06 16:48:49 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 499.3 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 57.1%, Prefix cache hit rate: 93.6% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 16x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) INFO 06-06 16:48:54 [loggers.py:259] Engine 000: Avg prompt throughput: 49.5 tokens/s, Avg generation throughput: 504.5 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 58.6%, Prefix cache hit rate: 93.6% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) ERROR 06-06 16:48:57 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 59x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) INFO 06-06 16:48:59 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 485.1 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.4%, Prefix cache hit rate: 88.9% [repeated 49x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:00 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 58x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) INFO 06-06 16:49:04 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 383.2 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.9%, Prefix cache hit rate: 88.9% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 30x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) INFO 06-06 16:49:09 [loggers.py:259] Engine 000: Avg prompt throughput: 66.8 tokens/s, Avg generation throughput: 416.1 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.3%, Prefix cache hit rate: 89.0% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:49:12 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) INFO 06-06 16:49:14 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 430.6 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.9%, Prefix cache hit rate: 89.0% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) INFO 06-06 16:49:19 [loggers.py:259] Engine 000: Avg prompt throughput: 40.5 tokens/s, Avg generation throughput: 456.1 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 36.3%, Prefix cache hit rate: 89.0% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) INFO 06-06 16:49:24 [loggers.py:259] Engine 000: Avg prompt throughput: 127.5 tokens/s, Avg generation throughput: 510.6 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.4%, Prefix cache hit rate: 89.0% [repeated 49x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:49:28 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) INFO 06-06 16:49:29 [loggers.py:259] Engine 000: Avg prompt throughput: 205.2 tokens/s, Avg generation throughput: 572.2 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 42.4%, Prefix cache hit rate: 89.1% [repeated 47x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 16:49:31 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) INFO 06-06 16:49:34 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 553.4 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.0%, Prefix cache hit rate: 89.1% [repeated 50x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) INFO 06-06 16:49:39 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 479.1 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.7%, Prefix cache hit rate: 89.1% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) INFO 06-06 16:49:44 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 424.5 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 36.9%, Prefix cache hit rate: 89.1% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) INFO 06-06 16:49:49 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 384.4 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.4%, Prefix cache hit rate: 89.1% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:49:49] OK: 359 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:49:49] OK: RSS 1.28 GiB | node mem 443.6/858.0 GiB used (51.7%), avail 414.4 GiB (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) ERROR 06-06 16:49:54 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) INFO 06-06 16:49:54 [loggers.py:259] Engine 000: Avg prompt throughput: 163.1 tokens/s, Avg generation throughput: 321.5 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 26.9%, Prefix cache hit rate: 89.9% [repeated 49x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:49:57] OK: 372 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:49:57] OK: RSS 1.25 GiB | node mem 440.1/858.0 GiB used (51.3%), avail 417.8 GiB (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 24x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:49:57 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) INFO 06-06 16:49:59 [loggers.py:259] Engine 000: Avg prompt throughput: 27.9 tokens/s, Avg generation throughput: 386.9 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 25.7%, Prefix cache hit rate: 90.0% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) INFO 06-06 16:50:04 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 380.3 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 26.1%, Prefix cache hit rate: 90.0% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:50:06] OK: 352 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:50:06] OK: RSS 1.27 GiB | node mem 432.2/858.0 GiB used (50.4%), avail 425.8 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 65x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2121712, ip=10.128.16.185) INFO 06-06 16:50:09 [loggers.py:259] Engine 000: Avg prompt throughput: 9.8 tokens/s, Avg generation throughput: 345.6 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.3%, Prefix cache hit rate: 90.1% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 85x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:50:14] OK: 351 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:50:14] OK: RSS 1.28 GiB | node mem 407.5/858.0 GiB used (47.5%), avail 450.5 GiB (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) INFO 06-06 16:50:14 [loggers.py:259] Engine 000: Avg prompt throughput: 223.5 tokens/s, Avg generation throughput: 652.5 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 59.0%, Prefix cache hit rate: 88.6% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 67x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) INFO 06-06 16:50:19 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 718.3 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 60.0%, Prefix cache hit rate: 88.6% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 57x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) INFO 06-06 16:50:24 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 720.0 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 61.0%, Prefix cache hit rate: 88.6% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster] (skyrl_entrypoint pid=1407951) [fd-monitor] [16:50:29] OK: 564 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:50:29] OK: RSS 1.85 GiB | node mem 196.1/858.0 GiB used (22.9%), avail 661.9 GiB (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) INFO 06-06 16:50:29 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 690.2 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 52.0%, Prefix cache hit rate: 88.6% [repeated 49x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 21x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) INFO 06-06 16:50:34 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 580.7 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 48.8%, Prefix cache hit rate: 88.6% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) INFO 06-06 16:50:39 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 576.4 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 49.6%, Prefix cache hit rate: 88.6% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:50:43 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) INFO 06-06 16:50:44 [loggers.py:259] Engine 000: Avg prompt throughput: 9.4 tokens/s, Avg generation throughput: 603.5 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 50.6%, Prefix cache hit rate: 88.7% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) INFO 06-06 16:50:49 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 600.5 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 51.2%, Prefix cache hit rate: 88.7% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:50:53 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) INFO 06-06 16:50:54 [loggers.py:259] Engine 000: Avg prompt throughput: 105.9 tokens/s, Avg generation throughput: 591.6 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 53.2%, Prefix cache hit rate: 88.7% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 31x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) INFO 06-06 16:50:59 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 622.6 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 54.1%, Prefix cache hit rate: 88.7% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:02 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 36x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) INFO 06-06 16:51:04 [loggers.py:259] Engine 000: Avg prompt throughput: 8.0 tokens/s, Avg generation throughput: 629.2 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 55.0%, Prefix cache hit rate: 88.7% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:51:08 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) INFO 06-06 16:51:09 [loggers.py:259] Engine 000: Avg prompt throughput: 35.1 tokens/s, Avg generation throughput: 701.9 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 56.2%, Prefix cache hit rate: 88.7% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 16x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) INFO 06-06 16:51:14 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 718.3 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 57.2%, Prefix cache hit rate: 88.7% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 16:51:17 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 30x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) INFO 06-06 16:51:19 [loggers.py:259] Engine 000: Avg prompt throughput: 78.3 tokens/s, Avg generation throughput: 694.8 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 57.9%, Prefix cache hit rate: 88.6% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 9x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) INFO 06-06 16:51:24 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 717.5 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 58.9%, Prefix cache hit rate: 88.6% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) INFO 06-06 16:51:29 [loggers.py:259] Engine 000: Avg prompt throughput: 75.7 tokens/s, Avg generation throughput: 647.4 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 58.5%, Prefix cache hit rate: 88.6% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) INFO 06-06 16:51:34 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 646.8 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 59.0%, Prefix cache hit rate: 88.6% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) INFO 06-06 16:51:39 [loggers.py:259] Engine 000: Avg prompt throughput: 365.4 tokens/s, Avg generation throughput: 647.7 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 65.4%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 16x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=2121712, ip=10.128.16.185) INFO 06-06 16:51:44 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 362.2 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.2%, Prefix cache hit rate: 90.4% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 30x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:51:49] OK: 386 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:51:49] OK: RSS 1.28 GiB | node mem 443.5/858.0 GiB used (51.7%), avail 414.4 GiB (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) INFO 06-06 16:51:50 [loggers.py:259] Engine 000: Avg prompt throughput: 69.3 tokens/s, Avg generation throughput: 354.9 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 46.3%, Prefix cache hit rate: 91.1% [repeated 49x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:51:55 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 541.5 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 51.1%, Prefix cache hit rate: 90.3% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:51:57] OK: 346 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:51:57] OK: RSS 1.27 GiB | node mem 443.0/858.0 GiB used (51.6%), avail 415.0 GiB (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:52:00 [loggers.py:259] Engine 000: Avg prompt throughput: 43.7 tokens/s, Avg generation throughput: 553.6 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 52.1%, Prefix cache hit rate: 90.3% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 51x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:52:05 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 562.1 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 52.9%, Prefix cache hit rate: 90.3% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:52:06] OK: 346 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:52:06] OK: RSS 1.28 GiB | node mem 432.2/858.0 GiB used (50.4%), avail 425.8 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) ERROR 06-06 16:52:06 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 54x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:52:10 [loggers.py:259] Engine 000: Avg prompt throughput: 84.1 tokens/s, Avg generation throughput: 564.7 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 48.6%, Prefix cache hit rate: 90.4% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) ERROR 06-06 16:52:07 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 71x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:52:14] OK: 363 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:52:14] OK: RSS 1.28 GiB | node mem 407.5/858.0 GiB used (47.5%), avail 450.5 GiB (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:52:15 [loggers.py:259] Engine 000: Avg prompt throughput: 125.0 tokens/s, Avg generation throughput: 551.2 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 51.0%, Prefix cache hit rate: 90.4% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 30x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:52:20 [loggers.py:259] Engine 000: Avg prompt throughput: 90.4 tokens/s, Avg generation throughput: 507.1 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 45.9%, Prefix cache hit rate: 90.4% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 21x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:52:25 [loggers.py:259] Engine 000: Avg prompt throughput: 11.6 tokens/s, Avg generation throughput: 551.9 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 46.7%, Prefix cache hit rate: 90.4% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster] (skyrl_entrypoint pid=1407951) [fd-monitor] [16:52:29] OK: 565 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:52:29] OK: RSS 1.86 GiB | node mem 196.1/858.0 GiB used (22.9%), avail 661.9 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:52:30 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 555.4 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 47.5%, Prefix cache hit rate: 90.4% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:52:35 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 556.8 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 48.2%, Prefix cache hit rate: 90.4% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:52:40 [loggers.py:259] Engine 000: Avg prompt throughput: 113.6 tokens/s, Avg generation throughput: 535.1 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 40.8%, Prefix cache hit rate: 90.4% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:52:45 [loggers.py:259] Engine 000: Avg prompt throughput: 21.8 tokens/s, Avg generation throughput: 567.1 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 45.1%, Prefix cache hit rate: 90.5% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:52:50 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 601.7 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 46.0%, Prefix cache hit rate: 90.5% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:52:55 [loggers.py:259] Engine 000: Avg prompt throughput: 143.2 tokens/s, Avg generation throughput: 568.6 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.6%, Prefix cache hit rate: 90.5% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 26x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:53:00 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 556.9 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.4%, Prefix cache hit rate: 90.5% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 4x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:53:05 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 556.8 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 36.1%, Prefix cache hit rate: 90.5% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:53:10 [loggers.py:259] Engine 000: Avg prompt throughput: 46.6 tokens/s, Avg generation throughput: 580.6 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 42.9%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:53:15 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 619.5 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 43.1%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:53:20 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 603.3 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 43.9%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 30x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:53:25 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 540.2 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.4%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:53:30 [loggers.py:259] Engine 000: Avg prompt throughput: 13.2 tokens/s, Avg generation throughput: 552.3 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 40.0%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 26x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:53:35 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 581.2 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 39.9%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:53:40 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 534.0 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 40.4%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 32x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:53:45 [loggers.py:259] Engine 000: Avg prompt throughput: 10.6 tokens/s, Avg generation throughput: 541.4 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 41.3%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 4x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:53:49] OK: 356 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:53:49] OK: RSS 1.28 GiB | node mem 444.0/858.0 GiB used (51.8%), avail 413.9 GiB (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:53:50 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 557.7 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 42.1%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:53:55 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 555.6 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 42.9%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:53:57] OK: 364 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:53:57] OK: RSS 1.28 GiB | node mem 442.9/858.0 GiB used (51.6%), avail 415.1 GiB (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 30x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:54:00 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 555.7 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 43.6%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:54:05 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 558.4 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 44.4%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:54:06] OK: 358 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:54:06] OK: RSS 1.28 GiB | node mem 432.2/858.0 GiB used (50.4%), avail 425.8 GiB (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:54:10 [loggers.py:259] Engine 000: Avg prompt throughput: 27.3 tokens/s, Avg generation throughput: 723.3 tokens/s, Running: 19 reqs, Waiting: 0 reqs, GPU KV cache usage: 87.3%, Prefix cache hit rate: 89.7% [repeated 49x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:54:14] OK: 358 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:54:14] OK: RSS 1.28 GiB | node mem 407.5/858.0 GiB used (47.5%), avail 450.5 GiB (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:54:15 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 558.2 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 46.2%, Prefix cache hit rate: 90.7% [repeated 47x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1420003, ip=10.128.16.41) ERROR 06-06 16:54:17 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:54:20 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 554.9 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 47.0%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 16:54:17 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:54:25 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 676.0 tokens/s, Running: 17 reqs, Waiting: 0 reqs, GPU KV cache usage: 80.5%, Prefix cache hit rate: 89.6% [repeated 49x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 21x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:54:29] OK: 564 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:54:29] OK: RSS 1.87 GiB | node mem 196.1/858.0 GiB used (22.9%), avail 661.9 GiB (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:54:30 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 664.3 tokens/s, Running: 17 reqs, Waiting: 0 reqs, GPU KV cache usage: 81.4%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:54:35 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 642.3 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 50.1%, Prefix cache hit rate: 90.7% [repeated 47x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 16:54:39 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:54:40 [loggers.py:259] Engine 000: Avg prompt throughput: 46.8 tokens/s, Avg generation throughput: 636.6 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 51.2%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:54:45 [loggers.py:259] Engine 000: Avg prompt throughput: 220.5 tokens/s, Avg generation throughput: 677.7 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 52.6%, Prefix cache hit rate: 90.6% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:54:50 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 666.0 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 53.0%, Prefix cache hit rate: 90.6% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 16:54:54 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:54:55 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 647.2 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 53.9%, Prefix cache hit rate: 90.6% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) INFO 06-06 16:55:00 [loggers.py:259] Engine 000: Avg prompt throughput: 67.0 tokens/s, Avg generation throughput: 328.7 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 27.6%, Prefix cache hit rate: 90.9% [repeated 47x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1455863, ip=10.128.16.48) INFO 06-06 16:55:05 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 602.0 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 55.1%, Prefix cache hit rate: 90.6% [repeated 49x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:55:10 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 705.8 tokens/s, Running: 18 reqs, Waiting: 0 reqs, GPU KV cache usage: 82.0%, Prefix cache hit rate: 89.7% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:55:15 [loggers.py:259] Engine 000: Avg prompt throughput: 0.8 tokens/s, Avg generation throughput: 736.4 tokens/s, Running: 19 reqs, Waiting: 0 reqs, GPU KV cache usage: 83.1%, Prefix cache hit rate: 89.7% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:55:20 [loggers.py:259] Engine 000: Avg prompt throughput: 61.5 tokens/s, Avg generation throughput: 738.9 tokens/s, Running: 20 reqs, Waiting: 0 reqs, GPU KV cache usage: 84.2%, Prefix cache hit rate: 89.7% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1427008, ip=10.128.16.36) ERROR 06-06 16:55:23 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:55:25 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 748.3 tokens/s, Running: 19 reqs, Waiting: 0 reqs, GPU KV cache usage: 83.9%, Prefix cache hit rate: 89.7% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:55:30 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 738.5 tokens/s, Running: 19 reqs, Waiting: 0 reqs, GPU KV cache usage: 84.9%, Prefix cache hit rate: 89.7% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 33x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:55:35 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 738.2 tokens/s, Running: 19 reqs, Waiting: 0 reqs, GPU KV cache usage: 85.9%, Prefix cache hit rate: 89.7% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:55:40 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 738.9 tokens/s, Running: 19 reqs, Waiting: 0 reqs, GPU KV cache usage: 87.0%, Prefix cache hit rate: 89.7% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:55:45 [loggers.py:259] Engine 000: Avg prompt throughput: 21.3 tokens/s, Avg generation throughput: 757.4 tokens/s, Running: 19 reqs, Waiting: 0 reqs, GPU KV cache usage: 89.8%, Prefix cache hit rate: 89.7% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:55:49] OK: 351 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:55:49] OK: RSS 1.29 GiB | node mem 444.6/858.0 GiB used (51.8%), avail 413.3 GiB (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:55:50 [loggers.py:259] Engine 000: Avg prompt throughput: 369.3 tokens/s, Avg generation throughput: 741.9 tokens/s, Running: 21 reqs, Waiting: 0 reqs, GPU KV cache usage: 91.4%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:55:55 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 779.6 tokens/s, Running: 20 reqs, Waiting: 0 reqs, GPU KV cache usage: 90.2%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:55:57] OK: 360 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:55:57] OK: RSS 1.28 GiB | node mem 442.9/858.0 GiB used (51.6%), avail 415.1 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:56:00 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 772.5 tokens/s, Running: 20 reqs, Waiting: 0 reqs, GPU KV cache usage: 91.3%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:56:05 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 741.0 tokens/s, Running: 19 reqs, Waiting: 0 reqs, GPU KV cache usage: 91.7%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:56:06] OK: 345 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:56:06] OK: RSS 1.28 GiB | node mem 432.2/858.0 GiB used (50.4%), avail 425.8 GiB (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:56:10 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 729.2 tokens/s, Running: 19 reqs, Waiting: 0 reqs, GPU KV cache usage: 92.7%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:56:14] OK: 355 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:56:14] OK: RSS 1.28 GiB | node mem 407.5/858.0 GiB used (47.5%), avail 450.5 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:56:15 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 724.7 tokens/s, Running: 19 reqs, Waiting: 0 reqs, GPU KV cache usage: 93.7%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 21x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:56:20 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 719.7 tokens/s, Running: 19 reqs, Waiting: 0 reqs, GPU KV cache usage: 94.7%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 21x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:56:25 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 714.5 tokens/s, Running: 19 reqs, Waiting: 0 reqs, GPU KV cache usage: 95.7%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:56:29] OK: 564 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:56:29] OK: RSS 1.87 GiB | node mem 196.1/858.0 GiB used (22.9%), avail 661.9 GiB (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:56:30 [loggers.py:259] Engine 000: Avg prompt throughput: 115.1 tokens/s, Avg generation throughput: 733.3 tokens/s, Running: 20 reqs, Waiting: 0 reqs, GPU KV cache usage: 96.9%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 22x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:56:35 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 734.2 tokens/s, Running: 19 reqs, Waiting: 0 reqs, GPU KV cache usage: 89.0%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:56:40 [loggers.py:259] Engine 000: Avg prompt throughput: 316.8 tokens/s, Avg generation throughput: 736.4 tokens/s, Running: 20 reqs, Waiting: 0 reqs, GPU KV cache usage: 90.5%, Prefix cache hit rate: 89.4% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:56:46 [loggers.py:259] Engine 000: Avg prompt throughput: 574.1 tokens/s, Avg generation throughput: 759.7 tokens/s, Running: 21 reqs, Waiting: 0 reqs, GPU KV cache usage: 92.5%, Prefix cache hit rate: 89.2% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:56:51 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 763.8 tokens/s, Running: 20 reqs, Waiting: 0 reqs, GPU KV cache usage: 93.0%, Prefix cache hit rate: 89.2% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:56:56 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 752.6 tokens/s, Running: 20 reqs, Waiting: 0 reqs, GPU KV cache usage: 94.1%, Prefix cache hit rate: 89.2% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:57:01 [loggers.py:259] Engine 000: Avg prompt throughput: 10.8 tokens/s, Avg generation throughput: 769.1 tokens/s, Running: 21 reqs, Waiting: 0 reqs, GPU KV cache usage: 95.2%, Prefix cache hit rate: 89.2% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:57:06 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 777.2 tokens/s, Running: 21 reqs, Waiting: 0 reqs, GPU KV cache usage: 96.2%, Prefix cache hit rate: 89.2% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:57:11 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 760.4 tokens/s, Running: 19 reqs, Waiting: 0 reqs, GPU KV cache usage: 94.5%, Prefix cache hit rate: 89.2% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:57:16 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 705.7 tokens/s, Running: 19 reqs, Waiting: 0 reqs, GPU KV cache usage: 95.5%, Prefix cache hit rate: 89.2% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:57:21 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 700.0 tokens/s, Running: 19 reqs, Waiting: 0 reqs, GPU KV cache usage: 96.5%, Prefix cache hit rate: 89.2% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 35x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:57:26 [loggers.py:259] Engine 000: Avg prompt throughput: 3.2 tokens/s, Avg generation throughput: 725.4 tokens/s, Running: 20 reqs, Waiting: 0 reqs, GPU KV cache usage: 97.5%, Prefix cache hit rate: 89.3% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 36x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:57:31 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 719.5 tokens/s, Running: 20 reqs, Waiting: 0 reqs, GPU KV cache usage: 98.5%, Prefix cache hit rate: 89.3% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 34x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:57:36 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 719.8 tokens/s, Running: 19 reqs, Waiting: 0 reqs, GPU KV cache usage: 90.7%, Prefix cache hit rate: 89.3% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 24x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:57:41 [loggers.py:259] Engine 000: Avg prompt throughput: 643.8 tokens/s, Avg generation throughput: 727.1 tokens/s, Running: 20 reqs, Waiting: 0 reqs, GPU KV cache usage: 92.6%, Prefix cache hit rate: 89.0% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 22x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:57:46 [loggers.py:259] Engine 000: Avg prompt throughput: 453.8 tokens/s, Avg generation throughput: 758.9 tokens/s, Running: 21 reqs, Waiting: 0 reqs, GPU KV cache usage: 94.3%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:57:49] OK: 348 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:57:49] OK: RSS 1.29 GiB | node mem 445.9/858.0 GiB used (52.0%), avail 412.0 GiB (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:57:51 [loggers.py:259] Engine 000: Avg prompt throughput: 1650.8 tokens/s, Avg generation throughput: 738.8 tokens/s, Running: 21 reqs, Waiting: 0 reqs, GPU KV cache usage: 96.6%, Prefix cache hit rate: 88.2% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 21x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:57:56 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 759.1 tokens/s, Running: 21 reqs, Waiting: 0 reqs, GPU KV cache usage: 97.7%, Prefix cache hit rate: 88.2% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 29x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:57:57] OK: 357 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:57:57] OK: RSS 1.28 GiB | node mem 442.9/858.0 GiB used (51.6%), avail 415.1 GiB (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:58:01 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 754.5 tokens/s, Running: 21 reqs, Waiting: 0 reqs, GPU KV cache usage: 98.7%, Prefix cache hit rate: 88.2% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 26x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:58:06] OK: 350 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [16:58:06] OK: RSS 1.28 GiB | node mem 432.2/858.0 GiB used (50.4%), avail 425.8 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:58:06 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 748.7 tokens/s, Running: 21 reqs, Waiting: 0 reqs, GPU KV cache usage: 99.8%, Prefix cache hit rate: 88.2% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:58:11 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 701.9 tokens/s, Running: 19 reqs, Waiting: 0 reqs, GPU KV cache usage: 99.4%, Prefix cache hit rate: 88.2% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 14x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:58:14] OK: 360 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [16:58:14] OK: RSS 1.29 GiB | node mem 407.5/858.0 GiB used (47.5%), avail 450.5 GiB (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:58:16 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 678.4 tokens/s, Running: 18 reqs, Waiting: 1 reqs, GPU KV cache usage: 97.8%, Prefix cache hit rate: 88.2% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) INFO 06-06 16:58:21 [loggers.py:259] Engine 000: Avg prompt throughput: 25.0 tokens/s, Avg generation throughput: 425.5 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 47.4%, Prefix cache hit rate: 91.5% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) INFO 06-06 16:58:26 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 453.7 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 48.0%, Prefix cache hit rate: 91.5% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:58:29] OK: 564 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1407951) [fd-monitor] [16:58:29] OK: RSS 1.87 GiB | node mem 196.0/858.0 GiB used (22.9%), avail 661.9 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:58:31 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 734.6 tokens/s, Running: 18 reqs, Waiting: 0 reqs, GPU KV cache usage: 86.6%, Prefix cache hit rate: 87.8% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:58:36 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 694.0 tokens/s, Running: 18 reqs, Waiting: 0 reqs, GPU KV cache usage: 87.5%, Prefix cache hit rate: 87.8% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:58:41 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 680.1 tokens/s, Running: 17 reqs, Waiting: 0 reqs, GPU KV cache usage: 79.6%, Prefix cache hit rate: 87.8% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:58:46 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 680.9 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 79.2%, Prefix cache hit rate: 87.8% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:58:51 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 672.1 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 80.2%, Prefix cache hit rate: 87.8% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 6x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:58:56 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 667.5 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 72.2%, Prefix cache hit rate: 87.8% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 16x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:59:01 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 667.7 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 73.1%, Prefix cache hit rate: 87.8% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:59:06 [loggers.py:259] Engine 000: Avg prompt throughput: 657.2 tokens/s, Avg generation throughput: 698.1 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 75.0%, Prefix cache hit rate: 87.6% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 6x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:59:11 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 689.6 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 74.9%, Prefix cache hit rate: 87.6% [repeated 49x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 11x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:59:16 [loggers.py:259] Engine 000: Avg prompt throughput: 46.8 tokens/s, Avg generation throughput: 672.8 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 76.1%, Prefix cache hit rate: 87.6% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:59:21 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 715.7 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 77.1%, Prefix cache hit rate: 87.6% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 19x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:59:26 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 715.7 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 78.1%, Prefix cache hit rate: 87.6% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 24x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:59:31 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 696.8 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 78.7%, Prefix cache hit rate: 87.6% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 30x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:59:36 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 672.0 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 79.6%, Prefix cache hit rate: 87.6% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 26x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1420133, ip=10.128.16.41) ERROR 06-06 16:59:41 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) INFO 06-06 16:59:41 [loggers.py:259] Engine 000: Avg prompt throughput: 67.3 tokens/s, Avg generation throughput: 314.0 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 25.3%, Prefix cache hit rate: 91.5% [repeated 47x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 16x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) INFO 06-06 16:59:46 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 305.8 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 22.6%, Prefix cache hit rate: 91.5% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:59:49] OK: 352 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [16:59:49] OK: RSS 1.29 GiB | node mem 447.6/858.0 GiB used (52.2%), avail 410.4 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:59:51 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 670.2 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 75.2%, Prefix cache hit rate: 87.5% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1092252, ip=10.128.16.38) ERROR 06-06 16:59:52 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 16:59:56 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 669.6 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 76.1%, Prefix cache hit rate: 87.5% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:59:57] OK: 358 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [16:59:57] OK: RSS 1.28 GiB | node mem 448.4/858.0 GiB used (52.3%), avail 409.5 GiB (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 17:00:01 [loggers.py:259] Engine 000: Avg prompt throughput: 22.6 tokens/s, Avg generation throughput: 690.4 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 78.1%, Prefix cache hit rate: 87.6% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [17:00:06] OK: 358 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [17:00:06] OK: RSS 1.28 GiB | node mem 432.2/858.0 GiB used (50.4%), avail 425.8 GiB (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 17:00:06 [loggers.py:259] Engine 000: Avg prompt throughput: 81.1 tokens/s, Avg generation throughput: 683.2 tokens/s, Running: 17 reqs, Waiting: 0 reqs, GPU KV cache usage: 79.6%, Prefix cache hit rate: 87.6% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 11x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 17:00:11 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 656.3 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 76.8%, Prefix cache hit rate: 87.6% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) ERROR 06-06 17:00:12 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 14x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [17:00:14] OK: 353 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [17:00:14] OK: RSS 1.29 GiB | node mem 407.5/858.0 GiB used (47.5%), avail 450.4 GiB (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 17:00:16 [loggers.py:259] Engine 000: Avg prompt throughput: 98.9 tokens/s, Avg generation throughput: 617.8 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 77.8%, Prefix cache hit rate: 87.5% [repeated 49x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] Error in preprocessing prompt inputs [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] Traceback (most recent call last): [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] return await asyncio.gather( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] raise VLLMValidationError( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:15 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 3x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 34x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) INFO 06-06 17:00:21 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 647.5 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 78.7%, Prefix cache hit rate: 87.5% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1379081, ip=10.128.16.44) INFO 06-06 17:00:26 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 397.9 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 30.7%, Prefix cache hit rate: 90.2% [repeated 49x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (skyrl_entrypoint pid=1407951) [fd-monitor] [17:00:29] OK: 564 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1407951) [fd-monitor] [17:00:29] OK: RSS 1.88 GiB | node mem 196.2/858.0 GiB used (22.9%), avail 661.8 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1379081, ip=10.128.16.44) INFO 06-06 17:00:31 [loggers.py:259] Engine 000: Avg prompt throughput: 11.0 tokens/s, Avg generation throughput: 410.2 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.3%, Prefix cache hit rate: 90.2% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1379081, ip=10.128.16.44) INFO 06-06 17:00:36 [loggers.py:259] Engine 000: Avg prompt throughput: 10.2 tokens/s, Avg generation throughput: 481.6 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.0%, Prefix cache hit rate: 90.2% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:00:37 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 22x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1379081, ip=10.128.16.44) INFO 06-06 17:00:41 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 461.6 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.0%, Prefix cache hit rate: 90.2% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 30x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) INFO 06-06 17:00:46 [loggers.py:259] Engine 000: Avg prompt throughput: 30.1 tokens/s, Avg generation throughput: 493.1 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 53.2%, Prefix cache hit rate: 91.2% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1236954, ip=10.128.16.45) INFO 06-06 17:00:51 [loggers.py:259] Engine 000: Avg prompt throughput: 13.0 tokens/s, Avg generation throughput: 489.1 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 51.2%, Prefix cache hit rate: 91.3% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 8x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 8x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 26x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1943929, ip=10.128.16.187) ERROR 06-06 17:00:55 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) INFO 06-06 17:00:56 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 408.6 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 26.7%, Prefix cache hit rate: 90.0% [repeated 49x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 27x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1446014, ip=10.128.16.46) ERROR 06-06 17:00:59 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) INFO 06-06 17:01:01 [loggers.py:259] Engine 000: Avg prompt throughput: 58.3 tokens/s, Avg generation throughput: 306.5 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 26.5%, Prefix cache hit rate: 90.0% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 31x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) INFO 06-06 17:01:06 [loggers.py:259] Engine 000: Avg prompt throughput: 92.6 tokens/s, Avg generation throughput: 314.5 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 27.7%, Prefix cache hit rate: 89.9% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) INFO 06-06 17:01:12 [loggers.py:259] Engine 000: Avg prompt throughput: 76.7 tokens/s, Avg generation throughput: 338.8 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 28.3%, Prefix cache hit rate: 90.0% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:01:12 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) INFO 06-06 17:01:17 [loggers.py:259] Engine 000: Avg prompt throughput: 86.2 tokens/s, Avg generation throughput: 367.2 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.7%, Prefix cache hit rate: 89.9% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1088873, ip=10.128.16.43) ERROR 06-06 17:01:12 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 22x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455735, ip=10.128.16.48) INFO 06-06 17:01:22 [loggers.py:259] Engine 000: Avg prompt throughput: 33.2 tokens/s, Avg generation throughput: 386.7 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.5%, Prefix cache hit rate: 89.9% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) INFO 06-06 17:01:27 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 656.3 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 63.9%, Prefix cache hit rate: 89.7% [repeated 49x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 32x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1092382, ip=10.128.16.38) ERROR 06-06 17:01:28 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) INFO 06-06 17:01:32 [loggers.py:259] Engine 000: Avg prompt throughput: 212.3 tokens/s, Avg generation throughput: 626.2 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 65.2%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) INFO 06-06 17:01:37 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 632.4 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 64.9%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 21x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) INFO 06-06 17:01:42 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 579.8 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 65.7%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) INFO 06-06 17:01:47 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 578.6 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 66.5%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [17:01:49] OK: 349 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [17:01:49] OK: RSS 1.29 GiB | node mem 447.6/858.0 GiB used (52.2%), avail 410.4 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) INFO 06-06 17:01:52 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 528.6 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 65.0%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) INFO 06-06 17:01:57 [loggers.py:259] Engine 000: Avg prompt throughput: 24.2 tokens/s, Avg generation throughput: 525.6 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 66.3%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [17:01:57] OK: 355 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [17:01:57] OK: RSS 1.28 GiB | node mem 448.4/858.0 GiB used (52.3%), avail 409.5 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) INFO 06-06 17:02:02 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 535.4 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 67.0%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [17:02:06] OK: 332 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [17:02:06] OK: RSS 1.28 GiB | node mem 432.1/858.0 GiB used (50.4%), avail 425.8 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) INFO 06-06 17:02:07 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 535.9 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 67.8%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 30x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) INFO 06-06 17:02:12 [loggers.py:259] Engine 000: Avg prompt throughput: 74.9 tokens/s, Avg generation throughput: 544.4 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 69.1%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [17:02:14] OK: 345 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [17:02:14] OK: RSS 1.29 GiB | node mem 407.6/858.0 GiB used (47.5%), avail 450.4 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1092380, ip=10.128.16.38) INFO 06-06 17:02:17 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 581.6 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 70.0%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 30x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) INFO 06-06 17:02:22 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 382.2 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 42.2%, Prefix cache hit rate: 89.7% [repeated 49x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 32x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) INFO 06-06 17:02:27 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 382.5 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 42.7%, Prefix cache hit rate: 89.7% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 44x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (skyrl_entrypoint pid=1407951) [fd-monitor] [17:02:29] OK: 564 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1407951) [fd-monitor] [17:02:29] OK: RSS 1.88 GiB | node mem 196.1/858.0 GiB used (22.9%), avail 661.8 GiB (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) INFO 06-06 17:02:32 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 376.1 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 36.5%, Prefix cache hit rate: 89.4% [repeated 47x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 41x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) INFO 06-06 17:02:37 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 365.2 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.0%, Prefix cache hit rate: 89.4% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 45x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1427007, ip=10.128.16.36) ERROR 06-06 17:02:42 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) INFO 06-06 17:02:42 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 365.7 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.5%, Prefix cache hit rate: 89.4% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 43x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) INFO 06-06 17:02:47 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 366.1 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.0%, Prefix cache hit rate: 89.4% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) INFO 06-06 17:02:52 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 366.3 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.5%, Prefix cache hit rate: 89.4% [repeated 49x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) INFO 06-06 17:02:57 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 366.7 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 39.0%, Prefix cache hit rate: 89.4% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1236956, ip=10.128.16.45) ERROR 06-06 17:03:02 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1445887, ip=10.128.16.46) INFO 06-06 17:03:02 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 366.2 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 39.5%, Prefix cache hit rate: 89.4% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) INFO 06-06 17:03:07 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 381.5 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 46.9%, Prefix cache hit rate: 89.7% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) INFO 06-06 17:03:12 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 382.3 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 47.5%, Prefix cache hit rate: 89.7% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) INFO 06-06 17:03:17 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 382.7 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 48.0%, Prefix cache hit rate: 89.7% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1379208, ip=10.128.16.44) ERROR 06-06 17:03:17 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) INFO 06-06 17:03:22 [loggers.py:259] Engine 000: Avg prompt throughput: 74.4 tokens/s, Avg generation throughput: 395.0 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 48.7%, Prefix cache hit rate: 89.7% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) INFO 06-06 17:03:27 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 421.0 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 49.1%, Prefix cache hit rate: 89.7% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) INFO 06-06 17:03:32 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 438.9 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 44.2%, Prefix cache hit rate: 90.4% [repeated 49x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) INFO 06-06 17:03:37 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 439.1 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 44.8%, Prefix cache hit rate: 90.4% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 16x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) INFO 06-06 17:03:42 [loggers.py:259] Engine 000: Avg prompt throughput: 237.3 tokens/s, Avg generation throughput: 459.4 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 49.2%, Prefix cache hit rate: 90.4% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 24x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) INFO 06-06 17:03:47 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 439.4 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 41.0%, Prefix cache hit rate: 90.4% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [17:03:49] OK: 359 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [17:03:49] OK: RSS 1.29 GiB | node mem 447.6/858.0 GiB used (52.2%), avail 410.4 GiB (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) INFO 06-06 17:03:52 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 439.1 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 41.6%, Prefix cache hit rate: 90.4% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) INFO 06-06 17:03:57 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 438.3 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 42.2%, Prefix cache hit rate: 90.4% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [17:03:57] OK: 388 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [17:03:57] OK: RSS 1.29 GiB | node mem 448.4/858.0 GiB used (52.3%), avail 409.6 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) INFO 06-06 17:04:02 [loggers.py:259] Engine 000: Avg prompt throughput: 45.0 tokens/s, Avg generation throughput: 446.6 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 43.6%, Prefix cache hit rate: 90.4% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [17:04:06] OK: 370 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1237433, ip=10.128.16.45) [fd-monitor] [17:04:06] OK: RSS 1.29 GiB | node mem 432.1/858.0 GiB used (50.4%), avail 425.8 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) INFO 06-06 17:04:07 [loggers.py:259] Engine 000: Avg prompt throughput: 29.9 tokens/s, Avg generation throughput: 484.6 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 46.0%, Prefix cache hit rate: 90.5% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) INFO 06-06 17:04:12 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 493.7 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 45.8%, Prefix cache hit rate: 90.5% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [17:04:14] OK: 363 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=2122385, ip=10.128.16.185) [fd-monitor] [17:04:14] OK: RSS 1.29 GiB | node mem 407.6/858.0 GiB used (47.5%), avail 450.4 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) INFO 06-06 17:04:17 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 456.2 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 44.6%, Prefix cache hit rate: 90.5% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1236826, ip=10.128.16.45) ERROR 06-06 17:04:18 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) INFO 06-06 17:04:22 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 443.3 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 45.3%, Prefix cache hit rate: 90.5% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1092381, ip=10.128.16.38) INFO 06-06 17:04:27 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 444.9 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 45.9%, Prefix cache hit rate: 90.5% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (skyrl_entrypoint pid=1407951) [fd-monitor] [17:04:29] OK: 564 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1407951) [fd-monitor] [17:04:29] OK: RSS 1.88 GiB | node mem 196.2/858.0 GiB used (22.9%), avail 661.8 GiB (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) INFO 06-06 17:04:32 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 558.5 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 53.4%, Prefix cache hit rate: 86.6% [repeated 49x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) INFO 06-06 17:04:37 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 526.1 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 52.4%, Prefix cache hit rate: 86.6% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) INFO 06-06 17:04:42 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 466.6 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 50.4%, Prefix cache hit rate: 86.6% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) ERROR 06-06 17:04:43 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1943928, ip=10.128.16.187) INFO 06-06 17:04:47 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 464.7 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 51.0%, Prefix cache hit rate: 86.6% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 29x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2121842, ip=10.128.16.185) ERROR 06-06 17:04:44 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) INFO 06-06 17:04:53 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 462.9 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.1%, Prefix cache hit rate: 88.5% [repeated 49x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 14x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) INFO 06-06 17:04:58 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 459.0 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.7%, Prefix cache hit rate: 88.5% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:02 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) INFO 06-06 17:05:03 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 457.3 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.4%, Prefix cache hit rate: 88.5% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) INFO 06-06 17:05:08 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 457.3 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 39.0%, Prefix cache hit rate: 88.5% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) INFO 06-06 17:05:13 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 456.9 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 39.7%, Prefix cache hit rate: 88.5% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1378448, ip=10.128.16.37) ERROR 06-06 17:05:15 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) INFO 06-06 17:05:18 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 449.6 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 42.9%, Prefix cache hit rate: 89.9% [repeated 49x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) INFO 06-06 17:05:23 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 420.8 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 43.4%, Prefix cache hit rate: 89.9% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) INFO 06-06 17:05:28 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 417.6 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 44.0%, Prefix cache hit rate: 89.9% [repeated 48x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster] (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1379210, ip=10.128.16.44) ERROR 06-06 17:05:33 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) INFO 06-06 17:05:33 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 374.5 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.6%, Prefix cache hit rate: 89.9% [repeated 48x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 22x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1943930, ip=10.128.16.187) ERROR 06-06 17:05:36 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) INFO 06-06 17:05:38 [loggers.py:259] Engine 000: Avg prompt throughput: 101.3 tokens/s, Avg generation throughput: 421.8 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.0%, Prefix cache hit rate: 89.9% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1237433, ip=10.128.16.45) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1237433, ip=10.128.16.45) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1379209, ip=10.128.16.44) ERROR 06-06 17:05:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) INFO 06-06 17:05:43 [loggers.py:259] Engine 000: Avg prompt throughput: 34.6 tokens/s, Avg generation throughput: 458.9 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.9%, Prefix cache hit rate: 89.9% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 2x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) INFO 06-06 17:05:48 [loggers.py:259] Engine 000: Avg prompt throughput: 133.5 tokens/s, Avg generation throughput: 450.7 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.2%, Prefix cache hit rate: 89.8% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [17:05:49] OK: 347 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1449700, ip=10.128.16.42) [fd-monitor] [17:05:49] OK: RSS 1.29 GiB | node mem 447.6/858.0 GiB used (52.2%), avail 410.4 GiB (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) INFO 06-06 17:05:53 [loggers.py:259] Engine 000: Avg prompt throughput: 120.1 tokens/s, Avg generation throughput: 490.5 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.1%, Prefix cache hit rate: 89.8% [repeated 48x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [17:05:58] OK: 353 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1427486, ip=10.128.16.36) [fd-monitor] [17:05:58] OK: RSS 1.29 GiB | node mem 448.4/858.0 GiB used (52.3%), avail 409.6 GiB (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (AsyncVLLMInferenceEngine pid=1449220, ip=10.128.16.42) INFO 06-06 17:05:58 [loggers.py:259] Engine 000: Avg prompt throughput: 2.0 tokens/s, Avg generation throughput: 502.8 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.1%, Prefix cache hit rate: 89.8% [repeated 48x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 146x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 146x across cluster] (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1449700, ip=10.128.16.42) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=1237433, ip=10.128.16.45) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=2122385, ip=10.128.16.185) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (RolloutCoordinator pid=1427486, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455988, ip=10.128.16.48) ERROR 06-06 17:06:02 [core_client.py:616] Engine core proc EngineCore_DP0 died unexpectedly, shutting down client. (AsyncVLLMInferenceEngine pid=1943797, ip=10.128.16.187) INFO 06-06 17:06:02 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 413.8 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 39.6%, Prefix cache hit rate: 89.7% [repeated 43x across cluster] (RolloutCoordinator pid=2122385, ip=10.128.16.185) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 96x across cluster] (RolloutCoordinator pid=1427486, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 96x across cluster] (AsyncVLLMInferenceEngine pid=1236955, ip=10.128.16.45) ERROR 06-06 17:06:02 [core_client.py:616] Engine core proc EngineCore_DP0 died unexpectedly, shutting down client. [repeated 2x across cluster] Error executing job with overrides: ['+terminal_bench_config=terminal_bench', 'trainer.strategy=fsdp2', 'trainer.algorithm.advantage_estimator=rloo_n', 'trainer.algorithm.use_kl_loss=false', 'trainer.algorithm.kl_loss_coef=0.0', 'trainer.algorithm.eps_clip_low=0.2', 'trainer.algorithm.eps_clip_high=0.05', 'trainer.algorithm.loss_reduction=seq_mean_token_sum_norm_global', 'trainer.epochs=2', 'trainer.max_steps=80', 'trainer.update_epochs_per_batch=1', 'trainer.train_batch_size=64', 'trainer.policy_mini_batch_size=64', 'trainer.eval_batch_size=64', 'trainer.micro_forward_batch_size_per_gpu=4', 'trainer.micro_train_batch_size_per_gpu=1', 'trainer.max_prompt_length=999999', 'trainer.eval_interval=999999', 'trainer.eval_before_train=false', 'trainer.ckpt_interval=2', 'trainer.resume_mode=latest', 'trainer.hf_save_interval=5', '++trainer.hf_hub_repo_id=laion/ablation-pymethods2test-seqnorm', '++trainer.hf_hub_private=false', '++trainer.hf_hub_revision=main', '++trainer.enable_db_registration=false', 'trainer.project_name=OpenThoughts-Agent', 'trainer.log_level=INFO', 'trainer.tracker_commit_each_step=true', 'trainer.logger=console', 'trainer.run_name=ablation-pymethods2test-seqnorm', 'trainer.ckpt_path=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm/checkpoints', 'trainer.export_path=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm/exports', 'trainer.policy.optimizer_config.lr=8e-6', 'trainer.policy.optimizer_config.weight_decay=0.0', 'trainer.policy.optimizer_config.adam_betas=[0.9,0.999]', 'trainer.policy.optimizer_config.max_grad_norm=0.9', 'trainer.policy.fsdp_config.cpu_offload=false', 'trainer.policy.fsdp_config.reshard_after_forward=true', 'trainer.policy.fsdp_config.fsdp_size=4', 'trainer.policy.model.path=/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', 'trainer.ref.fsdp_config.cpu_offload=false', 'trainer.ref.fsdp_config.reshard_after_forward=true', 'trainer.ref.fsdp_config.fsdp_size=4', 'trainer.placement.colocate_all=false', 'trainer.placement.policy_num_nodes=2', 'trainer.placement.ref_num_nodes=2', 'trainer.placement.policy_num_gpus_per_node=4', 'trainer.placement.ref_num_gpus_per_node=4', 'trainer.fully_async.max_staleness_steps=16', 'trainer.fully_async.num_parallel_generation_workers=338', 'generator.backend=vllm', 'generator.timeout_multiplier=1.0', 'generator.model_dtype=bfloat16', 'generator.inference_engine_tensor_parallel_size=1', 'generator.num_inference_engines=48', 'generator.n_samples_per_prompt=8', 'generator.eval_n_samples_per_prompt=8', 'generator.gpu_memory_utilization=0.75', 'generator.max_num_seqs=24', 'generator.max_num_batched_tokens=65536', 'generator.enable_prefix_caching=true', 'generator.enable_chunked_prefill=true', 'generator.run_engines_locally=true', 'generator.weight_sync_backend=nccl', 'generator.async_engine=true', 'generator.batched=false', 'generator.enable_http_endpoint=true', 'generator.enable_ray_prometheus_stats=false', 'generator.vllm_stats_interval=1', 'generator.append_eos_token_after_stop_str_in_multi_turn=true', 'generator.max_turns=999999', 'generator.sampling_params.max_generate_length=4096', 'generator.sampling_params.temperature=0.7', 'generator.sampling_params.top_p=0.95', 'generator.sampling_params.top_k=20', '++generator.engine_init_kwargs.max_model_len=32768', '++generator.engine_init_kwargs.custom_chat_template_chat_completion_path=chat_templates/qwen3_thinking_acc.jinja2', '++generator.engine_init_kwargs.served_model_name=0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', 'data.train_data=["/e/scratch/jureap59/feuer1/tasks/exp_rpt_pymethods2test-large"]', 'data.val_data=[]', '+terminal_bench_config.trials_dir=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm/trace_jobs', '+terminal_bench_config.harbor.name=terminus-2', '+terminal_bench_config.harbor.max_episodes=999999', '+terminal_bench_config.harbor.enable_summarize=false', '+terminal_bench_config.harbor.store_all_messages=true', '+terminal_bench_config.harbor.trajectory_config.raw_content=true', '+terminal_bench_config.harbor.enable_episode_logging=false', '+terminal_bench_config.harbor.record_terminal_session=false', '+terminal_bench_config.harbor.enable_pane_logging=false', '+terminal_bench_config.harbor.strict_json_parser=true', '+terminal_bench_config.harbor.interleaved_thinking=true', '+terminal_bench_config.harbor.extra_body.chat_template_kwargs.enable_thinking=true', '+terminal_bench_config.harbor.override_timeout_sec=900', '+terminal_bench_config.harbor.override_cpus=1', '+terminal_bench_config.harbor.override_memory_mb=2048', '+terminal_bench_config.harbor.override_storage_mb=2048', '+terminal_bench_config.harbor.auto_snapshot=true', '+terminal_bench_config.harbor.verifier_override_timeout_sec=120', '+terminal_bench_config.harbor.max_retries=3', '+terminal_bench_config.harbor.min_wait_sec=60.0', '+terminal_bench_config.harbor.max_wait_sec=600.0', '+terminal_bench_config.harbor.wait_multiplier=2.0', '+terminal_bench_config.harbor.exclude_exceptions=["VerifierTimeoutError","VerifierRuntimeError","RewardFileNotFoundError","RewardFileEmptyError","VerifierOutputParseError"]', '+terminal_bench_config.harbor.n_concurrent_trials=675', '+terminal_bench_config.harbor.log_level=INFO', '+terminal_bench_config.harbor.enable_reward_shaping=false', '+terminal_bench_config.harbor.enable_error_classification=true', '+terminal_bench_config.harbor.mask_exceptions=["DaytonaError","EnvironmentStartTimeoutError","NetworkError","ConnectionError","RewardFileNotFoundError","RewardFileEmptyError","AgentEnvironmentTimeoutError","ContextLengthExceededError"]', '+terminal_bench_config.harbor.default_error_treatment=zero', '+terminal_bench_config.harbor.passthrough_exceptions=["AgentTimeoutError"]', '+terminal_bench_config.harbor.zero_exceptions=[]', '+terminal_bench_config.model_info.max_input_tokens=32000', '+terminal_bench_config.model_info.max_output_tokens=4096', '+terminal_bench_config.archiving.enabled=false', '+terminal_bench_config.trace_upload.enabled=true', '+terminal_bench_config.trace_upload.repo_org=DCAgent', '+terminal_bench_config.trace_upload.episodes=last', '+terminal_bench_config.trace_upload.dataset_type=SFT', '+terminal_bench_config.trace_upload.cleanup=true'] Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/examples/terminal_bench/entrypoints/main_tbench.py", line 142, in main() File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/main.py", line 94, in decorated_main _run_hydra( File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/_internal/utils.py", line 394, in _run_hydra _run_app( File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/_internal/utils.py", line 457, in _run_app run_and_report( File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/_internal/utils.py", line 223, in run_and_report raise ex File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/_internal/utils.py", line 220, in run_and_report return func() ^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/_internal/utils.py", line 458, in lambda: hydra.run( ^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/_internal/hydra.py", line 132, in run _ = ret.return_value ^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/core/utils.py", line 260, in return_value raise self._return_value File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/core/utils.py", line 186, in run_job ret.return_value = task_function(task_cfg) ^^^^^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/examples/terminal_bench/entrypoints/main_tbench.py", line 132, in main ray.get(skyrl_entrypoint.remote(cfg)) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/ray/_private/auto_init_hook.py", line 22, in auto_init_wrapper return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/ray/_private/client_mode_hook.py", line 104, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/ray/_private/worker.py", line 2967, in get values, debugger_breakpoint = worker.get_objects( ^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/ray/_private/worker.py", line 1015, in get_objects raise value.as_instanceof_cause() ray.exceptions.RayTaskError(ConfigAttributeError): ray::skyrl_entrypoint() (pid=1407951, ip=10.128.16.35) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/examples/terminal_bench/entrypoints/main_tbench.py", line 105, in skyrl_entrypoint exp.run() File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/skyrl_train/entrypoints/main_base.py", line 472, in run asyncio.run(trainer.train()) File "/e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/asyncio/runners.py", line 195, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/asyncio/base_events.py", line 691, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/skyrl_train/fully_async_trainer.py", line 491, in train await self._train_loop() File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/skyrl_train/fully_async_trainer.py", line 642, in _train_loop status = await self._run_training(training_input) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/skyrl_train/fully_async_trainer.py", line 868, in _run_training status = await asyncio.to_thread(self.train_critic_and_policy, training_input) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/asyncio/threads.py", line 25, in to_thread return await loop.run_in_executor(None, func_call) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/concurrent/futures/thread.py", line 59, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/skyrl_train/trainer.py", line 1434, in train_critic_and_policy policy_statuses = ray.get(self.policy_model.async_run_ray_method("mesh", "ppo_train", data)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^ ray.exceptions.RayTaskError(ConfigAttributeError): ray::FSDPPolicyWorkerBase.ppo_train() (pid=1408117, ip=10.128.16.35, actor_id=b46e5c16ece895cee8fda57602000000, repr=) File "/e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/concurrent/futures/_base.py", line 449, in result return self.__get_result() ^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result raise self._exception ^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/skyrl_train/workers/worker.py", line 752, in ppo_train self.cfg.trainer.algorithm.global_loss_denom = ( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 337, in __setattr__ raise e File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 334, in __setattr__ self.__set_impl(key, value) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 318, in __set_impl self._set_item_impl(key, value) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/basecontainer.py", line 549, in _set_item_impl self._validate_set(key, value) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 180, in _validate_set target = self._get_node(key) if key is not None else self ^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 475, in _get_node self._validate_get(key) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 164, in _validate_get self._format_and_raise( File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/base.py", line 231, in _format_and_raise format_and_raise( File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/_utils.py", line 899, in format_and_raise _raise(ex, cause) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/omegaconf/_utils.py", line 797, in _raise raise ex.with_traceback(sys.exc_info()[2]) # set env var OC_CAUSE=1 for full trace ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ omegaconf.errors.ConfigAttributeError: Key 'global_loss_denom' is not in struct full_key: trainer.algorithm.global_loss_denom object_type=dict Stopping Ray cluster... Ray cluster stopped [RLJobRunner] Crash detected (exit!=0) — preserving Ray logs to /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm/ray_logs BEFORE trace upload (so a wall-clock kill can't lose crash evidence)... Collecting Ray logs from worker jpbo-001-36... Collecting Ray logs from worker jpbo-001-37... Collecting Ray logs from worker jpbo-001-38... Collecting Ray logs from worker jpbo-001-41... Collecting Ray logs from worker jpbo-001-42... Collecting Ray logs from worker jpbo-001-43... Collecting Ray logs from worker jpbo-001-44... Collecting Ray logs from worker jpbo-001-45... Collecting Ray logs from worker jpbo-001-46... Collecting Ray logs from worker jpbo-001-47... Collecting Ray logs from worker jpbo-001-48... Collecting Ray logs from worker jpbo-004-41... Collecting Ray logs from worker jpbo-004-43... [RLJobRunner] Crash-time Ray log preservation complete. [RLJobRunner] Launching trace upload (training exit code: 1): repo_id: DCAgent/ablation-pymethods2test-seqnorm job_dir: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm episodes: last log: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/logs/ablation-pymethods2test-seqnorm_trace_upload.log [RLJobRunner] Waiting for trace upload to complete... [RLJobRunner] Trace upload failed with exit code 1. Preserving Ray logs to /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ray_logs/ Collecting Ray logs from worker jpbo-001-36... Collecting Ray logs from worker jpbo-001-37... Collecting Ray logs from worker jpbo-001-38... Collecting Ray logs from worker jpbo-001-41... Collecting Ray logs from worker jpbo-001-42... Collecting Ray logs from worker jpbo-001-43... Collecting Ray logs from worker jpbo-001-44... Collecting Ray logs from worker jpbo-001-45... Collecting Ray logs from worker jpbo-001-46... Collecting Ray logs from worker jpbo-001-47... Collecting Ray logs from worker jpbo-001-48... Collecting Ray logs from worker jpbo-004-41... Collecting Ray logs from worker jpbo-004-43... Ray log preservation complete